Optimization for Deep Learning

Research directions

Training and adapting neural networks places demands on optimizer memory, update geometry, and parameter efficiency. I study algorithms that address these constraints alongside a theoretical analysis of their convergence.

This includes compressed adaptive optimizers, low-rank adaptation, sparsified training, and adaptive scaling for Muon. The aim is to connect practical training methods with a clearer understanding of when and why they converge.

Research focus

  • Reducing the memory cost of adaptive optimization.
  • Analyzing low-rank adaptation and sparsified training.
  • Adapting step scales in normalized optimizers.

Selected papers

View all publications