« Back to Glossary Index

A significant challenge in training Artificial Neural Networks, particularly with gradient-based learning methods and backpropagation. This issue arises when the gradients, or weight updates, become either too small (vanishing) or too large (exploding), causing instability in the network’s training process. It occurs because the weights are updated based on the partial derivative of the error function with respect to the current weight, and in some cases, these updates can diminish or grow excessively with each iteration.