18 Jul 2022

A Detour to the Imaginary has its Benefits

Two examples of using complex numbers for real-function optimization.

#numerics #math #ml #gradient #code #python

29 Jun 2022

The Back-Gradient Trick

Stochastic Gradient Descent can be (kind of) reversed and can be used to compute gradients with respect to its hyperparameters.

#math #ml #gradient #graph #code #jax #deep-learning

18 Jun 2020

A Cleverer Trick on top of the Reparametrization Trick

Implicit differentiation can lead to an efficient computation of the gradient of reparametrized samples.

#math #ml #gradient #deep-learning