MTS: Pretraining

Master the Future of AI: Lead Pretraining as a Research Engineer in Palo Alto, CA

What We're Building We're at the forefront of AI innovation, focusing on pretraining models to meet the unique demands of our commercial partners. Our success in developing, aligning, and deploying state-of-the-art models has established a strong foundation. Equipped with ample H100 resources, we've built a robust infrastructure to support best-in-class finetuning and pretraining. Join us to contribute your expertise to a dynamic organization that values innovation and collaboration.

About Us As a small, interdisciplinary AI studio, we’ve trained several state-of-the-art language models and built a personal assistant. Our focus is on pretraining and deploying models for specific commercial use cases. We believe that AI represents the beginning of an era of exponential change, and our work reflects this moment of transformation. Our status as a public benefit corporation ensures that we prioritize the well-being and happiness of our partners, users, and wider stakeholders.

About the Role Our pretraining team is responsible for creating and refining the foundational models that power our AI capabilities for enterprise applications. Research engineers in this role focus on developing large-scale training datasets, optimizing pretraining processes, and innovating on model architectures to push the boundaries of what our models can achieve in enterprise environments.

This Role is for You If You:

• Have experience in pretraining large-scale language models from scratch or on extensive datasets.
• Are skilled in managing and utilizing large compute resources efficiently for pretraining purposes.
• Have a strong background in modern deep learning techniques and architectures, especially transformer models, and are proficient with PyTorch.
• Enjoy experimenting with new pretraining methodologies and hyperparameter tuning to achieve state-of-the-art results.
• Are familiar with distributed pretraining frameworks and tools such as Horovod or DeepSpeed.

How We Work We value excellence and ownership, with a structure focused on individual responsibilities rather than management hierarchies. Our team thrives on the unreasonable effectiveness of talented Individual Contributors who are given the resources, space, and ownership to move fast and deliver outstanding results. Teamwork, generosity, and constructive disagreement are at our core, driving our success in pretraining and beyond.

Engineering As a vertically integrated AI studio, we build our entire technology stack in-house, from large foundational model pretraining to user interface. We believe in scale as the engine of progress in AI and are building one of the largest supercomputers in the world to develop and deploy the next generation of AI through advanced pretraining techniques.

Our Benefits We offer generous benefits to ensure a positive, safe, inclusive, and inspiring work environment:

• Unlimited paid time off
• Parental leave and flexibility for all parents and caregivers
• Generous medical, dental, and vision plans for US employees
• Visa sponsorship for new hires
• Avenues for personal growth such as coaching, conference attendance, or specific trainings

Diversity & Inclusion We are committed to representing the full extent of the human experience inside our AI studio. Everyone from any walk of life is welcome, provided you have the right skills. We ensure diverse candidate pools for all open roles.