Promptfoo
OSS (MIT) prompt-engineering eval and red-team CLI; supports OWASP LLM Top-10 vulnerability tests, jailbreak generation, and CI/CD pipeline integration. $5M seed Mar-2024 (Andreessen Horowitz). Founders ex-Discord + ex-Smile Identity.
Acquired · Verified 2026-10-28 · fresh as of 2026-10-28 · 28 cited facts, 1 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Type
- OSS LLM eval + red-team toolkit vendor fix
- Tier
- T2
- Created
- 2023 (initial release) vendor fix
- Latest release
- 0.125.1 (2026-10-25) registry fix
- Pricing
- Community: free (open source, incl. 10k red-team probes/month); Enterprise: custom (managed cloud); On-Premise: custom vendor fix
- Funding
- Acquired by OpenAI (agreement announced Mar-2026); previously VC-backed incl. $5M seed Mar-2024 (Andreessen Horowitz) vendor fix
- Freshness
- 2026-10-28 fix
- Layer
- agent-infra repo fix
- SDK languages
- typescript repo fix
- Protocols
- mcp-server vendor fix
- Compliance
- iso27001, soc2-type2 vendor fix
- Developer activity
- OpenRank 25.48 (12-mo avg 31.5) · 135 participants — 2026-09 third-party fix
When to use
Optimised for: developer-led LLM eval + red-team in CI
Anti-fit: not applicable — wrong section
Pros & cons
Pros
OSS-first; CLI-driven (CI/CD-friendly); strong OWASP LLM Top-10 alignment; a16z-backed.
Cons
Self-host is the default; less polished UI than commercial rivals; smaller dataset of canned attacks than enterprise tools.
Claims & capabilities
$5M seed Mar-2024 (a16z); OWASP LLM Top-10 coverage
Technical surface
- API surface
- JS / Python CLI + YAML config
- Backend storage
- not applicable — wrong section
- Deployment
- Both (self-hosted + managed cloud) — OSS CLI runs locally; Enterprise managed cloud or on-premise
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- not applicable — wrong section
- MCP
- yes — MCP Proxy product for securing MCP traffic, plus MCP testing integrations
- A2A
- not applicable — wrong section
- OpenTelemetry
- not applicable — wrong section
Alternatives to Promptfoo → live options ranked by shared design, licence, hosting and MCP support
Similar systems
Other use-case-specific agent harnesses in the catalog, ranked by inbound references.
- Calypso AI (now F5 AI Guardrails) T2
Dublin / NYC founded 2018; GenAI security and content scanning. $23M Series A-1 Aug-2024 (Paladin Capital + Lockheed Martin Ventures). Pivoted from defence ML to enterprise GenAI guardrails 2023-24.
- Salesforce Agentforce T1
Cross-listing: Salesforce Agentforce already exists under Vertical / Customer-support / voice memory; this row is the Agent-harness framing — Agentforce as the consumer-facing agent IDE / builder on Data Cloud. Agentforce launched Sep-2024; Agentforce 2.0 announced Dec-2024.
- Lasso Security T2
Tel-Aviv-founded LLM security platform — adversarial testing, prompt-injection defence, data-leakage prevention. $6M seed Nov-2023 (Entrée Capital + Samsung Next).
- Pillar Security T2
Tel-Aviv-founded (Dor Sarig + Ziv Karliner, both ex-IDF cyber); end-to-end LLM security platform — discovery, red-teaming, runtime guardrails. $9M seed Apr-2024 (Shield Capital + Greenfield).
- Protect AI T2
Founded 2022 by Ian Swanson + Daryan Dehghanpisheh (ex-AWS); AI/ML supply-chain + GenAI security. Acquired by Palo Alto Networks (2025-07-22; announced Apr-2025, reported $700M); its capabilities now ship in Prisma AIRS. Products: Recon (red-team), ModelScan (model artifact scanner), Sightline (vulnerability DB).
- Garak T3
OSS (Apache 2.0) LLM vulnerability scanner — 'nmap for LLMs.' Originally Leon Derczynski (Univ Sheffield / ITU Copenhagen); NVIDIA now stewards the project. Tests for jailbreaks, prompt injection, hallucination, data leakage. Apache-2.0.
Sources (27 cited facts, 1 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is OSS LLM eval + red-team toolkit vendor
- What it does & what's distinct OSS (MIT) prompt-engineering eval and red-team CLI; supports OWASP LLM Top-10 vulnerabil… vendor
- Claims / benefits $5M seed Mar-2024 (a16z); OWASP LLM Top-10 coverage vendor
- Created 2023 (initial release) vendor
- Latest release 0.125.1 (2026-10-25) registry
- License MIT vendor
- GitHub promptfoo/promptfoo — ~26.5k stars, last commit 2026-10 repo
- Funding Acquired by OpenAI (agreement announced Mar-2026); previously VC-backed incl. $5M seed M… vendor
- Pricing Community: free (open source, incl. 10k red-team probes/month); Enterprise: custom (mana… vendor
- Deployment Both (self-hosted + managed cloud) — OSS CLI runs locally; Enterprise managed cloud or o… vendor
- HQ San Francisco vendor
- Founders / pedigree Ian Webster (CEO; ex-Discord); Michael D'Angelo (ex-Smile Identity) vendor
- Code/weights release MIT — OSS vendor
- API surface JS / Python CLI + YAML config vendor
- MCP support yes — MCP Proxy product for securing MCP traffic, plus MCP testing integrations vendor
- Optimised for developer-led LLM eval + red-team in CI vendor
- Pros OSS-first; CLI-driven (CI/CD-friendly); strong OWASP LLM Top-10 alignment; a16z-backed. vendor
- Cons Self-host is the default; less polished UI than commercial rivals; smaller dataset of ca… vendor
- Project & sources www.promptfoo.dev/ vendor
- architecture-deployment-locus hybrid vendor
- licensing-bucket open-core vendor
- positioning-platform-vs-product platform vendor
- taxonomy-three-layer-rollup agent-infra repo
- developer-activity-opendigger {"activity":526.62,"as_of":"2026-09","openrank_12mo":31.5,"openrank_current":25.48,"part… third-party
- compliance-certifications iso27001,soc2-type2 vendor
- sdk-languages typescript repo
- protocol-support mcp-server vendor
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
- 2026-10-28
- Latest release “not applicable — wrong section” → “0.125.1 (2026-10-25)” source
- Pricing “not applicable — wrong section” → “Community: free (open source, incl. 10k red-team probes/month); Enterprise: custom (managed cloud); On-Premise: custom” source
- Funding / ownership “$5M seed Mar-2024 (Andreessen Horowitz)” → “Acquired by OpenAI (agreement announced Mar-2026); previously VC-backed incl. $5M seed Mar-2024 (Andreessen Horowitz)” source
- Deployment “not applicable — wrong section” → “Both (self-hosted + managed cloud) — OSS CLI runs locally; Enterprise managed cloud or on-premise” source
- MCP support “not applicable — wrong section” → “yes — MCP Proxy product for securing MCP traffic, plus MCP testing integrations” source
Related systems
Referenced by (2)
- Langfuse integrates with — Langfuse's integrations directory has a Promptfoo integration page. (verified 2026-10-29)
- Portkey AI integrates with — Portkey docs have a Promptfoo app/library integration page. (verified 2026-10-29)