Jobs
A job is an arbitrary task with some stored state, pertaining to the governing workspace. Typically, jobs are orchestrated by pipeline operations, but can also represent tasks for the user to complete.
Objects
Job
| Path | JSON Type | Format | Description |
|---|---|---|---|
id | string | Version 4 UUID | The job ID |
created | string | ISO 8601 timestamp | The instant at which the job was created |
type | string | Job Type | The descriptor for the process undertaken by the job |
exitCode | number | Process exit status | The job exit code |
status | string | Job Status | The job status |
startTime | string | ISO 8601 timestamp | The instant at which the job run started |
endTime | string | ISO 8601 timestamp | The instant at which the job run ended |
Formats
Job Status
string
| Value | Description |
|---|---|
QUEUED | The job is queued |
RUNNING | The job is running |
COMPLETE | The job finished with no errors |
ERROR | The job finished with errors |
STOPPED | The job timed out or was manually stopped |
Job Type
string
| Value | Description |
|---|---|
WORKSPACE_INIT | A system task to create a Meltano project in a workspace repository - automatically run when a workspace is created |
PIPELINE_CONFIG | A system task to configure the Meltano project and publish datasets with reference to a pipeline - automatically run when a pipeline is created, or a pipeline with a status of FAILED is updated |
PIPELINE_VERIFY | A system task to display and test the configuration of a pipeline |
PIPELINE_RUN | A system task to run a pipeline to load data into the workspace default datastore, or some other destination external to the platform - manually run by the user or automatically run on the defined schedule |
PROFILE_COLLABORATE | A user task to send an invitation |
PROFILE_IMPORT | A user task to create a pipeline |
Requests
View all running or completed jobs for a workspace
GET /api/workspaces/{workspace-id}/jobs
Returns all running or completed jobs for the workspace {workspace-id}.
Prerequisites
- Workspace
{workspace-id}must exist
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/workspaces/65279a9b-0b3d-40f5-a69d-607d9f3abe87/jobs' -i -X GET
import requests
url = "https://app.meltano.com:443/api/workspaces/65279a9b-0b3d-40f5-a69d-607d9f3abe87/jobs"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Job collection with HAL links.
View all running or completed jobs for a pipeline
GET /api/pipelines/{pipeline-id}/jobs
Returns all running or completed jobs for the pipeline {pipeline-id}.
Prerequisites
- Pipeline
{pipeline-id}must exist
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/pipelines/f2003fc1-49d5-451c-b8ad-8c2134c7b31d/jobs' -i -X GET
import requests
url = "https://app.meltano.com:443/api/pipelines/f2003fc1-49d5-451c-b8ad-8c2134c7b31d/jobs"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Job collection with HAL links.
View a job
GET /api/jobs/{job-id}
Returns the job {job-id}.
Prerequisites
- Job
{job-id}must exist
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/jobs/69de2f7a-1c2c-4fdd-91c2-397180f8c738' -i -X GET
import requests
url = "https://app.meltano.com:443/api/jobs/69de2f7a-1c2c-4fdd-91c2-397180f8c738"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Job with HAL links.
View the logs of a job
GET /api/jobs/{job-id}/logs?sequence={sequence}
Returns the logs of the job {job-id}.
Prerequisites
- Job
{job-id}must exist
Query Parameters
| Query Parameter | Format | Default Value | Description |
|---|---|---|---|
sequence | Unsigned integer | 0 | The line number in the logs to read from |
Headers
Accept
| Media Type(s) | Description |
|---|---|
text/plain */* | Sets the response content type format to plain text |
application/stream+json application/x-ndjson | Sets the response content type format to NDJSON |
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/jobs/69de2f7a-1c2c-4fdd-91c2-397180f8c738/logs?sequence=0' -i -X GET \
-H 'Accept: application/x-ndjson'
import requests
url = "https://app.meltano.com:443/api/jobs/69de2f7a-1c2c-4fdd-91c2-397180f8c738/logs?sequence=0"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200: The job logs in the format specified by associated requestAcceptheader
204: No response body provided.
Create a job from a pipeline
POST /api/pipelines/{pipeline-id}/jobs
Creates a new job from the pipeline {pipeline-id}.
Prerequisites
- Pipeline
{pipeline-id}must exist and not already be running
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/pipelines/f2003fc1-49d5-451c-b8ad-8c2134c7b31d/jobs' -i -X POST \
-H 'Content-Type: application/json'
import requests
url = "https://app.meltano.com:443/api/pipelines/f2003fc1-49d5-451c-b8ad-8c2134c7b31d/jobs"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("POST", url, headers=headers)
print(response.text.encode('utf8'))
Response
201 Created
Job with HAL links.
Stop a job
PUT /api/jobs/{job-id}/stopped
Stops the execution of the job {job-id}.
Prerequisites
- Job
{job-id}must exist - Job
{job-id}must have statusRUNNING
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/jobs/3baf5ce5-76bd-4f24-98c5-ed0c846ec0e8/stopped' -i -X PUT \
-H 'Content-Type: application/json'
import requests
url = "https://app.meltano.com:443/api/jobs/3baf5ce5-76bd-4f24-98c5-ed0c846ec0e8/stopped"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers)
print(response.text.encode('utf8'))
Response
202 Accepted
Job stop acceptance message.
Delete a job
DELETE /api/jobs/{job-id}
Deletes and stops the execution of the job {job-id}.
Prerequisites
- Job
{job-id}must exist
Examples
- cURL
- Python (requests)
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.meltano.com:443/api/jobs/69de2f7a-1c2c-4fdd-91c2-397180f8c738' -i -X DELETE
import requests
url = "https://app.meltano.com:443/api/jobs/69de2f7a-1c2c-4fdd-91c2-397180f8c738"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("DELETE", url, headers=headers)
print(response.text.encode('utf8'))
Response
204 No Content
No response body provided.