Figure Eight Job Results API
Request rows, judgments, and reports.
OpenAPI Specification
openapi: 3.0.0
info:
title: Appen Platform Account Info Job Results API
version: 1.0.0
description: "# Introduction\nHello, and welcome to Appen, the leading data annotation platform for Machine Learning. Our API enables developers to build applications that can programmatically create, edit, and launch Appen jobs, download results, and more! \n\nAppen uses a RESTful API that accepts data requests as URL-encoded key-value pairs. Responses are returned in JSON and authentication is key-based.\n\nBefore integrating with the API, we highly recommend you first build and run a job using our [graphical user interface](https://client.appen.com/sessions/new). Once you have run a job successfully and are satisfied with the results, you can automate the process using the Appen API.\n\n# Authentication\nYou will need to supply your Appen API key to access the API. To find your API key:\n1. Click the Account button located in the bottom corner of the Global Navigation bar on the left side of any page.\n![Appen Authentication](/assets/img/authentication1.png \"Appen Authentication\")\n2. Click the API tab and find your key listed under the Your API Key section. \n![Appen API](/assets/img/authentication2.png \"Appen API key\")\n\nFor requests listed below, the API Key will be listed as `{api_key}`. \n\nThe API key needs to be included in the header for every request: \n```\n headers: { \n\n \"AUTHORIZATION\" => \"Token token={api_key}\" \n\n } \n```\n\nYou may also see other variables, such as `{job_id}` for operations pertaining to a specific job, or `{team_id}` when completing operations relevant to an entire team (like listing all available jobs). Job ID's can be determined from the job listing page or the URL. A team id is visible in the URL of the Teams tab in the Account page. \n# Resources\nThere are three primary data objects on Appen that correspond to resources on the API:\n## Jobs\nA **Job** is an interface that connects your data to an online workforce.\n Each job on the Appen platform has data, instructions, customizable questions for your use case\n (written in **CML** [](https://success.appen.com/hc/en-us/articles/202817989-CML-Custom-Markup-Language-Overview)),\n [test questions](https://success.appen.com/hc/en-us/articles/202702985-How-to-Create-Test-Questions),\n and is worked on by *Contributors*. Contributors submit *Judgments* on the rows of data via a worker interface. \n * All jobs in a single account can be found [on your jobs page](https://client.appen.com/jobs) and will be\n identified by a unique numeric id.\n## Units (also known as 'Rows')\nA **Unit** of data is uploaded to the job from your source data file via API or GUI. \n## Judgments \nA **Judgment** is the set of responses submitted by a contributor on a particular unit of data. It is\n recommended to collect multiple judgments and compare them to one another or aggregate to the top response. For\n each job, you can specify the number of judgments you would like each row to receive.\n# Responses and Messaging\nEach request to the Appen API returns an HTTP status code response and one or both of the following:\n* An application response message (in JSON) \n* A JSON representation of the object (job, unit, or judgments) requested\nSee the below sections of different API Requests, Responses and Messaging for more information:\n1. [Job Create/Update](/#tag/Job-CreateUpdate)\n2. [Manage Job Data](/#tag/Manage-Job-Data)\n3. [Job Status](/#tag/Job-Status)\n4. [Monitor Contributors](/#tag/Monitor-Contributors)\n5. [Manage Job Settings](/#tag/Manage-Job-Settings)\n6. [Job Results](/#tag/Job-Results)\n7. [Account Info](/#tag/Account-Info)\n"
license:
name: Licensed under Apache 2.0
url: http://www.apache.org/licenses/LICENSE-2.0.html
termsOfService: https://www.appen.com/privacy-statement/
contact:
email: help@appen.com
servers:
- url: https://api.appen.com/v1
description: Production Server
tags:
- name: Job Results
description: Request rows, judgments, and reports.
externalDocs:
url: https://success.appen.com/hc/en-us/articles/202703435-Jobs-Resource-Attributes
paths:
/jobs/{job_id}/units/{unit_id}.json:
get:
tags:
- Job Results
summary: Get data for a Unit
description: 'Get source data, all judgments and aggregate result for a specified
row.
'
parameters:
- $ref: '#/components/parameters/job_id'
- $ref: '#/components/parameters/api_key'
- $ref: '#/components/parameters/unit_id'
responses:
'200':
description: 'JSON object comprising judgment count, row state, and timestamp
values.
'
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/unit'
x-code-samples:
- lang: Shell
label: cURL
source: 'curl -X GET \
"https://api.appen.com/v1/jobs/{job_id}/units/{unit_id}.json" \
-H "Authorization: Token token={api_key}"
'
/jobs/{job_id}/regenerate.json:
post:
tags:
- Job Results
summary: Regenerate a job report.
description: "Regenerate a report of job output. See below for available report types. \n"
parameters:
- $ref: '#/components/parameters/job_id'
- $ref: '#/components/parameters/api_key'
- $ref: '#/components/parameters/type'
responses:
'302':
description: redirect
x-code-samples:
- lang: Shell
label: cURL
source: 'curl -X POST \
"https://api.appen.com/v1/jobs/{job_id}/regenerate" \
-d "type=aggregated" \
-H "Authorization: Token token={api_key}" \
'
/jobs/{job_id}.csv:
post:
tags:
- Job Results
summary: Download a job report.
description: Download a report from a job.
parameters:
- $ref: '#/components/parameters/job_id'
- $ref: '#/components/parameters/api_key'
- $ref: '#/components/parameters/type'
- in: query
name: filename
required: true
description: Name the output file
schema:
type: string
responses:
'201':
description: created
x-code-samples:
- lang: Shell
label: cURL
source: 'curl -L -X GET \
"https://api.appen.com/v1/jobs/{job_id}.csv" \
-d "type=aggregated" \
-H "Authorization: Token token={api_key}" \ \
-o "output_filename.zip"
'
/jobs/{job_id}/units/{unit_id}/judgments.json:
get:
tags:
- Job Results
summary: Get all judgments w/aggregated responses.
description: Get all judgments and aggregated responses per row
parameters:
- $ref: '#/components/parameters/job_id'
- $ref: '#/components/parameters/api_key'
- $ref: '#/components/parameters/unit_id'
responses:
'200':
description: 'JSON array of all Judgments received for the Row indicated in the
request.
'
x-code-samples:
- lang: Shell
label: cURL
source: 'curl -X GET \
"https://api.appen.com/v1/jobs/{job_id}/units/{unit_id}/judgments.json" \
-H "Authorization: Token token={api_key}" \
'
/jobs/{job_id}/judgments.json:
get:
tags:
- Job Results
summary: Get rows with judgments.
description: Get rows and their judgments
parameters:
- $ref: '#/components/parameters/job_id'
- $ref: '#/components/parameters/api_key'
- in: query
name: page
required: false
schema:
type: integer
responses:
'200':
description: 'JSON array of 100 rows with received judgments for the Job indicated
in the request. In order to get the next 100 rows, you must
increment the value of page. Keep doing so until an empty hash is
returned.
'
x-code-samples:
- lang: Shell
label: cURL
source: 'curl -X GET \
"https://api.appen.com/v1/jobs/{job_id}/judgments.json" \
-H "Authorization: Token token={api_key}" \
-d "page=1"
'
components:
schemas:
new_job_id:
description: Integer identifier of the requested resource (Job, Unit, Workflow) used in API request paths and response bodies.
type: integer
format: int64
readOnly: true
minimum: 1
maximum: 9223372036854776000
example: 1234567
unit:
properties:
id:
type: integer
format: int64
readOnly: true
example: 1234
data:
type: object
example: {}
judgments_count:
type: integer
format: int32
state:
type: string
enum:
- new
- judgable
- judging
- judged
- ordering
- golden
- hidden_gold
- canceled
- finalized
- preprocessing
agreement:
type: number
multipleOf: 0.01
missed_count:
type: integer
format: int32
contested_count:
type: integer
format: int32
gold_pool:
type: string
created_at:
type: string
format: date-time
readOnly: true
updated_at:
type: string
format: date-time
readOnly: true
job_id:
type: integer
format: int32
readOnly: true
example: 425765
parameters:
type:
name: type
in: query
required: true
description: 'Specifies the type of report to generate or download, valid values include:
- **full** - Returns the Full report containing every judgment.
- **aggregated** - Returns the Aggregated report containing the aggregated response for each row.
- **json** - Returns the JSON report containing the aggregated response, as well as the individual judgments.
- **gold_report** - Returns the Test Question report.
- **workset** - Returns the Contributor report.
- **source** - Returns a CSV of the source data uploaded to the job.
'
schema:
type: string
api_key:
name: key
in: query
required: true
description: "Your personal Appen API key; used to access the Appen platform. Can be found in your [account settings](https://client.appen.com/account/api). Variable name: `{api_key}`. \n"
schema:
type: string
unit_id:
name: unit_id
in: path
required: true
description: "A unique identifier for a row of data that has been loaded into the system. Variable name: `{unit_id}`. \n"
schema:
type: integer
format: int32
job_id:
in: path
name: job_id
description: "A unique identifier for a job or task that is distributed to contributors. Variable name: `{job_id}`. \n"
required: true
schema:
$ref: '#/components/schemas/new_job_id'