| task_key |
string |
A unique key for the task within the job. Used to reference the task in dependencies and logging. |
| description |
string |
A description of the task. |
| depends_on |
array |
An array of objects specifying the task dependencies. Each dependency is identified by its task_key. |
| existing_cluster_id |
string |
An existing cluster to run the task on. |
| job_cluster_key |
string |
Reference to a job_clusters entry. |
| notebook_task |
object |
|
| spark_jar_task |
object |
|
| spark_python_task |
object |
|
| spark_submit_task |
object |
|
| pipeline_task |
object |
|
| python_wheel_task |
object |
|
| sql_task |
object |
|
| dbt_task |
object |
|
| run_if |
string |
Condition to run this task. |
| timeout_seconds |
integer |
Timeout for this individual task. |
| max_retries |
integer |
Maximum number of retries for a failed task. |
| min_retry_interval_millis |
integer |
Minimum interval between retry attempts. |
| retry_on_timeout |
boolean |
Whether to retry when the task times out. |
| libraries |
array |
Libraries to install on the cluster running this task. |