Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Tailor your resume to this Research Engineer - Pre-training role.
Rezi rewrites your resume against Pluralis Research's job description. Free.
Tailor your resume to this Research Engineer - Pre-training role.
Rezi rewrites your resume against Pluralis Research's job description. Free.
Don't guess if your resume is good enough.
See how it scores against the Research Engineer - Pre-training posting at Pluralis Research — free, in seconds.
Don't guess if your resume is good enough.
See how it scores against the Research Engineer - Pre-training posting at Pluralis Research — free, in seconds.
About the Role
Pluralis Research is pioneering Protocol Learning, enabling decentralized training and serving of large models on consumer-grade devices. This role focuses on building the training system to scale Protocol Learning from 8B to frontier models, addressing challenges in distributed pretraining, performance optimization, elasticity, and fault tolerance across heterogeneous hardware and networks.
Responsibilities
- Implement and optimize model-parallel training, including data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links.
- Implement techniques to reduce communication overhead while maintaining model convergence in challenging network environments.
- Ensure runs survive node churn through robust checkpointing, state synchronization, and recovery mechanisms.
- Build monitoring systems to track throughput, bottlenecks, and model quality across hundreds of devices.
Requirements
- Hands-on distributed training experience with PyTorch (FSDP, DeepSpeed, Megatron, or custom implementation).
- Understanding of data, tensor, and pipeline parallelism.
- Strong production-quality Python engineering skills.
- Proficiency in concurrency, failure handling, and profiling.
- Demonstrated evidence of shipping systems, research code, open-source work, or significant personal projects.
- Belief in Protocol Learning as a viable path for collective, trustless, and sovereign AI.
- Professional-level English proficiency (written and spoken).
Skills
- Distributed training
- Model parallelism
- Data parallelism
- Pipeline parallelism
- Tensor parallelism
- PyTorch
- FSDP
- DeepSpeed
- Megatron
- Python
- Concurrency
- Failure handling
- Profiling
- Large language models
- P2P networking
- NAT traversal
- Post-training
- RL
- Inference
- Serving systems
Location
- Remote
- Australia
- US
Work Type
- Remote-First
- Full-time
Experience Level
- Key technical contributors
Salary/Compensations
- High base salary
Benefits
- Significant ownership (equity-heavy package)
- Flexible work environment
- Optional full visa sponsorship and relocation support to Australia or the US
About the Company
- Pluralis Research works on Protocol Learning: training and serving large models in a fully decentralized way on small consumer-grade devices connected via the internet.
- We have made significant advances, including Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs spread over the internet.
- Our published methods include Subspace Networks, Factored Gossip DiLoCo, AsyncMesh, and Sentinel.
- We work remotely across the world, with main teams in Australia and North America.
- We are backed by Union Square Ventures and other tier-1 investors.
- We are a world-class, deeply technical team of ML researchers.
- Pluralis is unapologetically ideological and believes AI, and the world, end up on a better path if we succeed in implementing the protocol for intelligence.