Catalog Systems Training infrastructure GRPO

GRPO

https://arxiv.org/abs/2402.03300

DeepSeek-Math's GRPO — group-relative advantage estimation replacing PPO's critic. Used in DeepSeek-R1 reasoning post-training; widely adopted in OSS reasoning RLHF.

Active · Verified 2026-10-27 · fresh as of 2026-10-27 · 7 cited facts, 0 with a third-party source · Suggest a correction

At a glance

Type
Group Relative Policy Optimization vendor fix
Tier
T3
Section
Training infrastructure
Created
2024-02 vendor fix
Latest release
searched not found fix
License
searched not found fix
GitHub
searched not found vendor fix
Pricing
not applicable — research paper fix
Funding
not applicable — research paper fix
Freshness
2026-10-27 fix
Layer
model-infra repo fix

When to use

Optimised for: searched not found

Anti-fit: searched not found

Pros & cons

Pros

searched not found

Cons

searched not found

Claims & capabilities

Foundational to DeepSeek-R1 reasoning model.

Technical surface

API surface
not applicable — research paper
Backend storage
not applicable — research paper
Deployment
not applicable — research paper
Embedding model
not applicable — research paper
Multi-tenancy
not applicable — research paper
MCP
not applicable — research paper
A2A
not applicable — research paper
OpenTelemetry
not applicable — research paper

Similar systems

Other training infrastructure in the catalog, ranked by inbound references.

  • MLflow T1

    Databricks-incubated open platform for ML lifecycle — tracking, projects, models, model registry. Standard ML-ops baseline. MLflow 3.0 (2024) added LLM-eval features.

  • AWS SageMaker T1

    AWS's managed ML platform — Studio, Training, HyperPod (large-scale clusters), Endpoints. Industry-standard for enterprise AWS-hosted training.

  • Weights & Biases T1

    Experiment-tracking and ML-ops platform (metrics, artifacts, sweeps, Weave tracing/evals). Acquired by CoreWeave (completed 2025-05-05, ~$1.7B); now part of CoreWeave Forge.

  • Comet ML T2

    ML experiment tracking + LLM evaluation platform (Opik for LLM eval). Comet competes head-to-head with Weights & Biases.

  • Azure Machine Learning T1

    Microsoft Azure's managed ML platform — pipelines, AutoML, Designer, prompt flow (LLM). Heavy integration with Azure OpenAI and Azure AI Studio.

  • LoRA T3

    Microsoft Research's Low-Rank Adapters method — trains rank-r matrices added to attention weights; 10000x parameter reduction. Foundation of all adapter-style fine-tuning.

Sources (6 cited facts, 0 third-party)

Each fact links to the page it came from; the badge says who published it (how to read sources).

Related systems

References (1)

  • DPO cites — S2 isInfluential citation

Referenced by (1)

  • SEAgent cites — S2 isInfluential citation