onlinecompiler-io Judge API

openapi: 3.2.0
info:
  title: OnlineCompiler.io Judge API
  version: '2026-10-24'
  summary: Run code in a sandbox, judge it against test cases, and read limits and usage.
  description: 'Execute source code in isolated, network-less sandboxes in dozens of languages and get stdout, stderr, exit code, time and memory back as JSON. This file describes the native REST API: synchronous runs, asynchronous runs with a callback, judging a program against test cases, the compiler list, execution limits, usage and health.


    **Authentication.** Create an API key (type API) in the dashboard at https://api.onlinecompiler.io/compilers/api-keys/ and send the bare key in the `Authorization` header, for example `Authorization: YOUR_API_KEY`. A `Bearer ` prefix is not accepted on this API. Widget keys cannot call it. `GET /api/compilers/`, `GET /api/limits/` and `GET /api/health/` need no key.


    Paths are written with a trailing slash; every path except `/api/files/{file_id}/` also works without it.


    **Limits.** Per-run limits (memory, CPUs, timeout, output size, file counts) and your plan''s allowance and rate limits change over time and are not repeated here: read `GET /api/limits/` for the defaults and `GET /api/usage/` for your account. A refusal for a limit answers HTTP 429 with `{"error": "..."}` and nothing runs or is counted.


    **Compatible APIs.** Existing clients can also talk to OnlineCompiler.io through compatibility layers that are not part of this description: Judge0 CE (https://onlinecompiler.io/docs/judge0), Piston v2 (https://onlinecompiler.io/docs/piston) and Jobe for Moodle CodeRunner (https://onlinecompiler.io/docs/jobe). The MCP server and WebSocket widget are documented at https://onlinecompiler.io/docs.


    Full documentation: https://onlinecompiler.io/docs'
  termsOfService: https://onlinecompiler.io/terms
  contact:
    name: OnlineCompiler.io
    url: https://onlinecompiler.io/contact
servers:
- url: https://api.onlinecompiler.io
  description: Production
tags:
- name: Judge
  description: Run one program against many tests and score it.
paths:
  /api/run-tests/:
    post:
      operationId: runTests
      tags:
      - Judge
      summary: Run a program against test cases and score it
      description: Runs one program against a list of tests, each in its own fresh sandbox, and answers with a verdict per test, an overall verdict and a score from 0 to 100. Always waits for the result. Each test counts as one run against the monthly allowance and the per-minute and pending limits, and the number of tests is capped by your plan (see `GET /api/usage/`, `pending_submissions`). The request has a time budget, so many tests with a long `time_limit` are refused with 400 and the largest `time_limit` that fits. Hidden and skipped tests return only `verdict` and `hidden`.
      security:
      - ApiKey: []
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/RunTestsRequest'
            example:
              compiler: python-3.14
              code: 'a, b = map(int, input().split())

                print(a + b)'
              tests:
              - input: '1 2

                  '
                expected_output: '3

                  '
              - input: '5 7

                  '
                expected_output: '12

                  '
                hidden: true
                weight: 2
              scoring: weighted
      responses:
        '200':
          description: 'Every test that could start has run. Check `verdict`: `incomplete` means some tests were skipped by the start cut-off; retry.'
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/RunTestsResponse'
              example:
                verdict: accepted
                score: 100
                passed: 2
                total: 2
                time_limit: 2
                compile_error: null
                tests:
                - verdict: accepted
                  hidden: false
                  time: '0.0213'
                  memory: '9000'
                  exit_code: 0
                  signal: null
                  output: '3

                    '
                  error: ''
                  truncated: false
                  files: []
                  files_truncated: false
                - verdict: accepted
                  hidden: true
        '400':
          $ref: '#/components/responses/BadRequest'
        '403':
          $ref: '#/components/responses/Forbidden'
        '404':
          $ref: '#/components/responses/InvalidKey'
        '429':
          $ref: '#/components/responses/RateLimited'
components:
  schemas:
    CompilerId:
      type: string
      description: A compiler id from GET /api/compilers/. Retired ids (the dotnet-*-9 ids, now dotnet-*-10) are still accepted and run on their successor, but are not listed.
      examples:
      - python-3.14
      - gcc-15
      - nodejs-24
      enum:
      - python-3.14
      - python-3.14-pandas2
      - duckdb-1.5
      - gcc-15
      - g++-15
      - openjdk-25
      - dotnet-csharp-10
      - dotnet-fsharp-10
      - php-8.5
      - ruby-4.0
      - haskell-9.12
      - go-1.26
      - rust-1.93
      - typescript-deno
      - nodejs-24
      - sqlite-3
      - bash-5.2
      - gfortran-15
      - dotnet-vbnet-10
      - kotlin-2.4
      - r-4.6
      - swift-6.4
      - gnucobol-3.2
      - perl-5.44
      - julia-1.13
      - fpc-3.2
      - octave-11
      - nasm-3
      - lua-5.5
      - ocaml-5.4
      - sbcl-2.6
      - erlang-29
      - elixir-1.20
      - swipl-10
      - ldc-1.43
      - fbc-1.10
      - dart-3.13
      - clojure-1.12
      - groovy-6.0
      - scala-3.10
      - zig-0.17
      - nim-2.2
      - crystal-1.22
      - gnat-15
      - powershell-7.6
      - racket-9.3
      - clang-objc-21
    OutputFile:
      type: object
      required:
      - name
      - mime
      - size
      - content_base64
      - url
      properties:
        name:
          type: string
        mime:
          type: string
        size:
          type: integer
          description: Bytes.
        content_base64:
          type: string
          description: The file's bytes, base64 encoded.
        url:
          type:
          - string
          - 'null'
          format: uri
          description: Where the account that ran it can fetch the file again with its API key (GET /api/files/{file_id}/). Null when the file was not stored, always in run-tests results.
    RunTestsRequest:
      type: object
      required:
      - compiler
      - code
      - tests
      properties:
        compiler:
          $ref: '#/components/schemas/CompilerId'
        code:
          type: string
          minLength: 1
          description: Source code, at most 100,000 bytes. Compiled in every test, or once per request for some compiled languages when there are more tests than run at once.
        tests:
          type: array
          minItems: 1
          items:
            $ref: '#/components/schemas/Test'
          description: The tests, from 1 up to your plan's pending-submissions limit. Inputs and expected outputs together may hold at most 1 MB.
        time_limit:
          type: number
          exclusiveMinimum: 0
          maximum: 30
          default: 2
          description: Seconds each test may run. Java, Groovy and F# get a start-up allowance on top.
        files:
          type: array
          maxItems: 8
          items:
            $ref: '#/components/schemas/InputFile'
          description: Input files shared by every test; each test's sandbox gets its own copy.
        checker:
          $ref: '#/components/schemas/Checker'
        tolerance:
          type: number
          minimum: 0
          maximum: 1
          default: 1.0e-06
          description: 'For the float and rows checkers: two numbers match when they differ by at most tolerance times the larger of 1 and the expected value.'
        scoring:
          $ref: '#/components/schemas/Scoring'
        stop_on_failure:
          type: boolean
          description: Stop starting tests after the first one that is not accepted. Defaults to true with all_or_nothing scoring and false otherwise.
    Checker:
      type: string
      enum:
      - trim
      - exact
      - tokens
      - float
      - rows
      default: trim
      description: 'How output is compared. trim: strip trailing whitespace per line and trailing blank lines, then compare. exact: byte for byte. tokens: compare whitespace-separated tokens. float: as tokens, numbers within tolerance. rows: as an unordered set of lines, fields split at | , or tab, numbers within tolerance.'
    InputFile:
      type: object
      required:
      - name
      - content
      properties:
        name:
          type: string
          maxLength: 100
          pattern: ^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$
          description: A plain file name. It cannot be input, run.sh, build, build.tar or go-trim.txt, its part before the first dot cannot be code, and names in one request must differ.
        content:
          type: string
          description: 'The file''s content: UTF-8 text without NUL characters, or base64 when encoding is base64.'
        encoding:
          type: string
          enum:
          - utf8
          - base64
          default: utf8
      description: A file placed in the program's working directory.
    RunTestsResponse:
      type: object
      required:
      - verdict
      - score
      - passed
      - total
      - time_limit
      - compile_error
      - tests
      properties:
        verdict:
          $ref: '#/components/schemas/OverallVerdict'
        score:
          type: number
          minimum: 0
          maximum: 100
          description: 0 to 100, rounded to 2 decimals. Skipped tests count as not accepted.
        passed:
          type: integer
          description: Tests accepted.
        total:
          type: integer
          description: Tests sent.
        time_limit:
          type: number
          description: The time limit used, in seconds, without any start-up allowance.
        compile_error:
          type:
          - string
          - 'null'
          description: The compiler output (first 4096 characters) when the code failed to compile and at least one test is visible; otherwise null.
        tests:
          type: array
          items:
            $ref: '#/components/schemas/TestResult'
          description: One entry per test, in the order sent.
    Error:
      type: object
      required:
      - error
      properties:
        error:
          type: string
          description: A human-readable reason.
    OverallVerdict:
      type: string
      enum:
      - internal_error
      - compilation_error
      - time_limit_exceeded
      - memory_limit_exceeded
      - output_limit_exceeded
      - runtime_error
      - wrong_answer
      - accepted
      - incomplete
      description: accepted when every test is; otherwise the verdict of the first test (in your order) that failed. incomplete when tests were skipped by the start cut-off and none failed; retry the request.
    Scoring:
      type: string
      enum:
      - all_or_nothing
      - percent
      - weighted
      default: all_or_nothing
      description: 'all_or_nothing: 100 when every test is accepted, else 0. percent: share of accepted tests. weighted: share of accepted weight.'
    Test:
      type: object
      required:
      - expected_output
      properties:
        input:
          type: string
          default: ''
          description: Standard input for this test, at most 100,000 bytes.
        expected_output:
          type: string
          description: 'What the program should print: UTF-8 without NUL, at most 64 KB. Stderr is never compared.'
        hidden:
          type: boolean
          default: false
          description: A hidden test returns only its verdict.
        weight:
          type: number
          exclusiveMinimum: 0
          default: 1
          description: Counts only for weighted scoring.
    TestResult:
      type: object
      required:
      - verdict
      - hidden
      properties:
        verdict:
          $ref: '#/components/schemas/Verdict'
        hidden:
          type: boolean
        time:
          type: string
        memory:
          type: string
        exit_code:
          type: integer
        signal:
          type:
          - integer
          - 'null'
        output:
          type: string
          description: The first 4096 characters of stdout.
        error:
          type: string
          description: The first 4096 characters of stderr.
        truncated:
          type: boolean
          description: True when output or error was cut.
        files:
          type: array
          items:
            $ref: '#/components/schemas/OutputFile'
        files_truncated:
          type: boolean
      description: Hidden tests and tests that did not run carry only verdict and hidden. A visible test that ran also has time, memory, exit_code, signal, output, error, truncated, files and files_truncated.
    Verdict:
      type: string
      enum:
      - internal_error
      - compilation_error
      - time_limit_exceeded
      - memory_limit_exceeded
      - output_limit_exceeded
      - runtime_error
      - wrong_answer
      - accepted
      - skipped
      description: 'internal_error: we could not run the test. compilation_error: compiling failed. time_limit_exceeded: exit code 124 or past the limit plus any start-up allowance. memory_limit_exceeded: killed at the memory limit. output_limit_exceeded: signal 25, or stdout was cut. runtime_error: any other non-zero exit or signal. wrong_answer: exit 0 and the checker rejects the output. accepted: exit 0 and the checker accepts it. skipped: the test never started.'
  responses:
    Forbidden:
      description: The Authorization header is missing, the key is not an API key (for example a widget key), or the request comes from an origin the key does not allow.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          examples:
            missing:
              summary: No header
              value:
                error: Missing Authorization header
            type:
              summary: Wrong key type
              value:
                error: This key is not authorized for the REST API
            origin:
              summary: Origin restriction
              value:
                error: Request origin not allowed for this API key
    InvalidKey:
      description: No active API key matches the Authorization header (unknown, deleted or deactivated).
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            error: Invalid or inactive API key
    RateLimited:
      description: A limit was reached or the sandboxes are busy. Nothing ran and nothing was counted. A spent monthly allowance or key cap will not clear until the month resets; the other causes clear shortly, so retry with backoff. Where a paid plan raises the limit, the message names it with a link.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          examples:
            month:
              summary: Monthly allowance spent
              value:
                error: Monthly submission limit of 100,000 reached. It resets on 1 November 2026 (UTC).
            key:
              summary: The key's own cap spent
              value:
                error: Monthly limit of 5,000 submissions for this API key reached. It resets on 1 November 2026 (UTC).
            rate:
              summary: Per-minute rate
              value:
                error: Submission rate limit exceeded. Try again shortly.
            pending:
              summary: Too many pending
              value:
                error: Too many pending submissions for this account.
            account:
              summary: The account's runs at once are busy
              value:
                error: Too many runs in progress for this account (limit 2 at once). Please try again shortly.
            host:
              summary: The host's sandboxes are busy
              value:
                error: Too many concurrent requests. Please try again shortly.
    BadRequest:
      description: 'The request is invalid and nothing ran or was counted: not a JSON object, a missing or bad field, an unsupported compiler, a bad file, an over-budget run-tests request, or (for /api/run-code/) a key with no usable callback URL.'
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          examples:
            compiler:
              summary: Unknown compiler
              value:
                error: Unsupported or missing compiler
            code:
              summary: No code
              value:
                error: Missing code field
            json:
              summary: Not JSON
              value:
                error: Invalid JSON body
            time_limit:
              summary: Bad time limit
              value:
                error: time_limit must be a number of seconds above 0 and at most 30
            tests:
              summary: run-tests without tests
              value:
                error: tests must be a list of 1 to 8 tests on this plan
            callback:
              summary: No callback URL on the key (/api/run-code/)
              value:
                error: This API key has no callback URL configured. Set one in your dashboard, or use /api/run-code-sync/ instead.
  securitySchemes:
    ApiKey:
      type: apiKey
      in: header
      name: Authorization
      description: Your API key (key type API) from https://api.onlinecompiler.io/compilers/api-keys/, sent as the bare value of the Authorization header. Do not add a Bearer prefix. A key may be restricted to an origin in the dashboard.
externalDocs:
  description: OnlineCompiler.io documentation
  url: https://onlinecompiler.io/docs