Promptfoo
OSS (MIT) prompt-engineering eval and red-team CLI; supports OWASP LLM Top-10 vulnerability tests, jailbreak generation, and CI/CD pipeline integration. $5M seed Mar-2024 (Andreessen Horowitz). Founders ex-Discord + ex-Smile Identity.
At a glance
- Type
- OSS LLM eval + red-team toolkit
- Tier
- T2
- Created
- 2023 (initial release)
- Latest release
- not applicable — wrong section
- License
- MIT
- Pricing
- not applicable — wrong section
- Funding
- $5M seed Mar-2024 (Andreessen Horowitz)
Taxonomy
- storage
- none-trivial
- retrieval
- none
- persistence
- session-only
- update
- agent-controlled
- unit
- turn
- governance
- opaque
- conflict
- out-of-scope
When to use
Optimised for: developer-led LLM eval + red-team in CI
Anti-fit: not applicable — wrong section
Pros & cons
Pros
OSS-first; CLI-driven (CI/CD-friendly); strong OWASP LLM Top-10 alignment; a16z-backed.
Cons
Self-host is the default; less polished UI than commercial rivals; smaller dataset of canned attacks than enterprise tools.
Claims & capabilities
$5M seed Mar-2024 (a16z); 8k+ GitHub stars; OWASP LLM Top-10 coverage
Technical surface
- API surface
- JS / Python CLI + YAML config
- Backend storage
- not applicable — wrong section
- Deployment
- not applicable — wrong section
- Embedding model
- not applicable — not a memory product
- Multi-tenancy
- not applicable — wrong section
- MCP
- not applicable — wrong section
- A2A
- not applicable — wrong section
- OpenTelemetry
- not applicable — wrong section
Similar systems
Other use-case-specific agent harnesses in the catalog, ranked by inbound references.
- 11x.ai T1
Founded 2022 by Hasan Sukkar; AI sales agents Alice (outbound SDR), Mike (voice SDR), Jordan (RevOps). $50M Series B Oct-2024 (Benchmark + Andreessen Horowitz; $350M val). Among the most-discussed AI-employee startups of 2024.
- 6sense T2
Founded 2013; ABM/intent platform with Revenue AI agents. $200M Series E Jan-2022 ($5.2B val; Blue Owl). 6sense Revenue AI agents launched 2024 — autonomous account research + email drafting.
- AlphaSense T2
Founded 2010; AI-driven market intelligence — semantic search over earnings calls, broker research, regulatory filings. $4B val Jun-2024 ($650M growth round; BDT + MSP). **Acquired Tegus Jun-2024 for $930M** (expert-call transcripts). Genesis = autonomous research agent launched 2024.
- Anvilogic Forge T3
Founded 2019 by ex-Yahoo CISO Karthik Kannan; AI-driven detection-engineering platform. $45M Series C Jul-2024 (Evolution + G Squared; ~$200M val). Forge is the AI agent that auto-generates SIEM detections.
- Apex Security T2
Founded 2023 by Tomer Avni + Matan Getz (ex-Microsoft); generative-AI security platform spanning red-teaming, DLP, and runtime. $7M seed Apr-2024 (Sequoia + Index Ventures).
- Apollo.io T2
Founded 2015 (rebranded from ZenProspect); GTM platform — prospecting database + outreach + AI. $100M Series D Aug-2023 ($1.6B val; Bain Capital Ventures + Sequoia). $200M+ ARR; Apollo AI launched 2024.