Splitting Adam Info

Published in 2025, this paper "splits" the problem of in LLM embeddings.

It argues that Adam's second moment actually causes word representations to become narrow and directional (anisotropic). Splitting Adam

A more recent and highly regarded paper (2025) investigates what happens when Adam "wanders" around the manifold of minimizers. Published in 2025, this paper "splits" the problem