Add GC and LinearWarmupCosineAnnealingLRScheduler#439

Merged

le1nux merged 7 commits intomainfrom

nccl_compute_overlap_investigation

Mar 30, 2026

Commits on Mar 20, 2026

chore: temporarily reverted combination of fsdp units
le1nux
committed

Commits on Mar 26, 2026

feat: added garbage collection
le1nux
committed
chore: setting cycle_momentum to False in OneCycleLRSchedulerConfig
le1nux
committed
refactor: reintroducing layers_per_fsdp_unit in model_factory for better FSDP sharding control
le1nux
committed

Commits on Mar 27, 2026

feat: added LinearWarmupCosineAnnealingLRScheduler
le1nux
committed

Commits on Mar 30, 2026