This is problem in deep learning techniques. After the forward pass and calculation of the error, during backpropogation if the gradients are small, then the back propogation would reduce keep reducing the gradients as it travels back the network. As per google it happens due to some older activation functions. There were many solutions

1. Relu- its derivative resulted in 0 or 1. So the gradients would never vanish
2. ResNet Residual connections- Skip a layer so that gradients are protected.
3. LSTM
4. careful weight initialiation

How needs to be researched.