ColossalAI

HPC-AI Tech's distributed training framework — ZeRO + tensor/pipeline parallel + heterogeneous training (CPU offload). HPC-AI raised $50M.

At a glance

Optimised for: searched not found

Anti-fit: searched not found

searched not found

searched not found

~40k★; HPC-AI Tech raised $50M+.

Other training infrastructure in the catalog, ranked by inbound references.

Azure Machine Learning T1
Microsoft Azure's managed ML platform — pipelines, AutoML, Designer, prompt flow (LLM). Heavy integration with Azure OpenAI and Azure AI Studio.
LoRA T3
Microsoft Research's Low-Rank Adapters method — trains rank-r matrices added to attention weights; 10000x parameter reduction. Foundation of all adapter-style fine-tuning.
DPO T3
Stanford method that converts RLHF into a supervised-learning loss over preference pairs — no separate reward model needed; widely adopted alignment recipe.
GRPO T3
DeepSeek-Math's GRPO — group-relative advantage estimation replacing PPO's critic. Used in DeepSeek-R1 reasoning post-training; widely adopted in OSS reasoning RLHF.
Accelerate (Hugging Face) T1
Hugging Face's abstraction over distributed-training backends (DDP, FSDP, DeepSpeed, Megatron). Minimal code change to scale a PyTorch script.
Argilla T2
Open-source data-quality platform for LLM training data — human-in-the-loop labelling, RLHF data collection, dataset curation. Acquired by Hugging Face 2024.