# BIG-Bench

**Canonical:** https://apis.io/apis/evals/big-bench/  
**Provider:** Evals — https://apis.io/providers/evals/  
**Base URL:** https://github.com/google/BIG-bench  
**Documentation:** https://github.com/google/BIG-bench

BIG-Bench is one of 20 APIs that [Evals](https://apis.io/providers/evals/) publishes on the [APIs.io](https://apis.io/) network. Tagged areas include Benchmarks, Collaborative, Multitask, Google, and BIG-Bench Lite. The published artifact set on APIs.io includes a GitHub repository and API documentation.

The Beyond the Imitation Game Benchmark (BIG-Bench) is "a collaborative benchmark intended to probe large language models and extrapolate their future capabilities." It contains more than 200 tasks across JSON-based simplified tasks and programmatic tasks; a curated subset (BIG-Bench Lite) of 24 tasks is provided as the canonical headline measurement. Maintained on GitHub by Google with open community task submissions.

## Machine-readable artifacts (3)

- **GitHubRepository** — https://github.com/google/BIG-bench
- **Paper** — https://arxiv.org/abs/2206.04615
- **Documentation** — https://github.com/google/BIG-bench/blob/main/README.md

## Other Evals APIs (12)

- [OpenAI Evals](https://apis.io/apis/evals/openai-evals/)
- [Inspect AI](https://apis.io/apis/evals/inspect-ai/)
- [Braintrust](https://apis.io/apis/evals/braintrust/)
- [LangSmith Evaluation](https://apis.io/apis/evals/langsmith-evaluation/)
- [Promptfoo](https://apis.io/apis/evals/promptfoo/)
- [Helicone](https://apis.io/apis/evals/helicone/)
- [Patronus AI](https://apis.io/apis/evals/patronus-ai/)
- [DeepEval (Confident AI)](https://apis.io/apis/evals/deepeval-confident-ai/)
- [Arize AI (Phoenix)](https://apis.io/apis/evals/arize-ai-phoenix/)
- [Galileo](https://apis.io/apis/evals/galileo/)
- [Humanloop](https://apis.io/apis/evals/humanloop/)
- [TruLens](https://apis.io/apis/evals/trulens/)

## Tags

Benchmarks, Collaborative, Multitask, Google, BIG-Bench Lite

---

Profiled by [API Evangelist](https://apievangelist.com) and published on [APIs.io](https://apis.io/apis/evals/big-bench/). The API's provider profile, Kin Score and agent-readiness rating are at https://apis.io/providers/evals/.