Taylor Expansion of the RMS Prop Update Algorithm
RMSprop is similar to gradient descent with momentum, but takes into consideration the sign of the gradient like this:
Recent Posts
The Dance of Forward Pass and Backprop
In order to implement a neural network, a solid understanding of Linear Algebra is needed for the forward pass, whereas vector calculus is needed for backpro...
Week of May 24th
What I did
Week of May 17
What I did
WSL2: VM’s in Windows are now Obsolete
In my previous miscellaneous post, I wrote about why I gave up on WSL1 and switched to running ubuntu in a virtual machine. After looking into WSL2 more, I c...