AI Researcher — Training Optimization

About the Role

We’re looking for anAI Researcher focused on training optimizationto help us push the efficiency, stability, and scalability of large-scale model training. You’ll work at the intersection ofresearch and systems, developing novel techniques to reduce training cost, accelerate convergence, and improve model quality—while validating ideas through rigorous experiments and publications.

This role is ideal for someone who enjoys turningresearch insights into practical training wins, and who has a track record (or strong ambition) ofpublishing applied ML research.

What You’ll Work On
* Design and evaluatetraining optimization techniquesfor large models (e.g. optimization algorithms, schedulers, normalization, curriculum strategies)
Design and evaluatetraining optimization techniquesfor large models (e.g. optimization algorithms, schedulers, normalization, curriculum strategies)
* Improvetraining efficiency and stabilityacross long runs and large datasets
Improvetraining efficiency and stabilityacross long runs and large datasets
* Research and implement methods such as:Optimizer and scheduler innovationsMixed-precision, low-precision, and memory-efficient trainingGradient noise reduction, scaling laws, and convergence analysisTraining-time regularization and robustness techniques

Research and implement methods such as:
* Optimizer and scheduler innovations

Optimizer and scheduler innovations
* Mixed-precision, low-precision, and memory-efficient training
Mixed-precision, low-precision, and memory-efficient training
* Gradient noise reduction, scaling laws, and convergence analysis
Gradient noise reduction, scaling laws, and convergence analysis
* Training-time regularization and robustness techniques
Training-time regularization and robustness techniques
* Run large-scale experiments, analyze results, and translate findings into actionable improvements
Run large-scale experiments, analyze results, and translate findings into actionable improvements
* Author or co-authorresearch papers, technical reports, or blog posts
Author or co-authorresearch papers, technical reports, or blog posts
* Collaborate closely with infrastructure and inference teams to ensure training decisions translate to real-world performance
Collaborate closely with infrastructure and inference teams to ensure training decisions translate to real-world performance

What We’re Looking For
* Strong background inmachine learning research, with emphasis ontraining dynamics and optimization
Strong background inmachine learning research, with emphasis ontraining dynamics and optimization
* Experience traininglarge neural networks(LLMs, multimodal models, or large sequence models)
Experience traininglarge neural networks(LLMs, multimodal models, or large sequence models)
* Publication experiencein ML venues (e.g. NeurIPS, ICML, ICLR, ACL, EMNLP, COLM, arXiv) or equivalent high-quality open research
Publication experiencein ML venues (e.g. NeurIPS, ICML, ICLR, ACL, EMNLP, COLM, arXiv) or equivalent high-quality open research
* Solid understanding of:Optimization theory and practiceBackpropagation, gradient flow, and training stabilityDistributed and large-batch training

Solid understanding of:
* Optimization theory and practice

Optimization theory and practice
* Backpropagation, gradient flow, and training stability
Backpropagation, gradient flow, and training stability
* Distributed and large-batch training

Distributed and large-batch training
* Proficiency inPythonand modern ML frameworks (PyTorch preferred)
Proficiency inPythonand modern ML frameworks (PyTorch preferred)
* Ability to independently design experiments and reason from data
Ability to independently design experiments and reason from data

Nice to Have
* Experience withnon-standard architectures(e.g. RNN variants, long-context models, hybrid systems)
Experience withnon-standard architectures(e.g. RNN variants, long-context models, hybrid systems)
* Experience optimizing training on GPUs at scale (FSDP, ZeRO, custom kernels)
Experience optimizing training on GPUs at scale (FSDP, ZeRO, custom kernels)
* Contributions toopen-source ML or research codebases
Contributions toopen-source ML or research codebases
* Comfort operating in fast-moving, ambiguous startup environments
Comfort operating in fast-moving, ambiguous startup environments

Why This Role
* Real influence overcore model training decisions
Real influence overcore model training decisions
* Freedom to pursue and publishnovel research
Freedom to pursue and publishnovel research
* Direct access to large-scale experiments and real production constraints
Direct access to large-scale experiments and real production constraints
* A small, senior team that valuesthinking deeply and shipping thoughtfully
A small, senior team that valuesthinking deeply and shipping thoughtfully

Originally posted onHimalayas