Skip to Content
Jobs

Jobs

A job is the unit of work you submit. Computalot expands it into tasks — one per input, chunk, or candidate — runs them on matching capacity, and hands back structured results.

Submit with POST /api/v1/jobs: your project name, your command, and a payload.

Choosing a job type

I want to…Use
Run one script with JSON input/outputstructured_runner
Run code across a list of inputsstructured_runner + fan_out.by
Evaluate many tiny inputs per worker taskstructured_runner + fan_out.by / fan_out.items + batch_size
Evaluate CMA/evolutionary candidatesstructured_runner + fan_out.items
Train a model on a GPUstructured_runner + profile: "gpu"
Search a parameter gridsweep
Run simulations and reduce resultsmap_reduce
Compare named strategiesbenchmark
Submit many jobs at oncePOST /api/v1/jobs/batch

When in doubt, use structured_runner.

Submitting a job

curl -sS https://computalot.com/api/v1/jobs \ -X POST \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "type": "structured_runner", "runner_command": ["python3", "evaluate.py"], "payload": {"model": "gpt-4", "dataset": "test_v3"}, "project": "my-project", "timeout_s": 600 }'

The response carries the job id, its status, and summary.billing_estimate — the authoritative cost quote for this run (see Pricing). timeout_s caps each task’s runtime; queue time is never charged.

The runner protocol

Your script talks to Computalot through files and exit codes — nothing to import inside the task:

  1. Computalot writes the task’s JSON payload to a temp file and points $COMPUTALOT_TASK_PAYLOAD at it
  2. Your script reads it, does the work, and writes JSON to $COMPUTALOT_TASK_RESULT
  3. Exit 0 marks the task completed; any non-zero exit marks it failed

The exit code is the final word — result JSON cannot override it. (result_quality / result_warnings are reserved and return null / []; non-empty result_schema values return 422.)

To report progress, print a line to stdout:

print(f"COMPUTALOT_PROGRESS:{json.dumps({'step': 42, 'loss': 0.05})}")

Stdout/stderr streams into the task’s live_feedback.output_tail and the SSE job stream while the task is still running. If your runner wraps a child process, run it unbuffered or flush explicitly so logs show up as they happen.

Fan-out

Three shapes, for three kinds of parallelism:

  • {"fan_out": {"by": "models"}} — split one array field in payload into one task per item
  • {"fan_out": {"items": [{...}, {...}]}} — provide the exact payload object for each task
  • {"fan_out": {"chunks": 20, "range_field": "total_seeds", "total": 10000}} — split a numeric range into chunk tasks

Pick exactly one — mixing by, items, or chunks + total in one request returns 422. When each item is tiny, add batch_size (or batch_per_task) so one dispatched task processes several items locally; batched tasks receive payload._batch metadata. For parameter grids, use the sweep job type instead of fan-out.

Watching a job

You wantUse
Poll statusGET /api/v1/jobs/:id every 2–5s
Live stream, one jobGET /api/v1/jobs/:id/stream — SSE, includes running-task output tails
Live stream, many jobsGET /api/v1/jobs/watch?ids=... — one SSE stream; frames carry client_ref, tags, meta, variant, summary fields, and persistence flags
Live stream, whole projectGET /api/v1/projects/:name/stream
Per-task detailGET /api/v1/jobs/:id/taskslive_feedback, latest_progress, checkpoint/resume state
Final resultsGET /api/v1/results/:job_id — per-task results, aggregates, completeness, artifact IDs
Stdout/stderrGET /api/v1/jobs/:id/output — during retries, keeps the last failed attempt’s output until the new attempt writes its own
Files your job producedGET /api/v1/artifacts, then GET /api/v1/artifacts/:id
CancelPUT /api/v1/jobs/:id/cancel

Lifecycle

planningqueuedrunningcompleted | partial | failed | cancelled

  • completed: every task succeeded
  • partial: at least one task succeeded and at least one failed or was cancelled
  • failed: no task succeeded and the work ended in failure
  • cancelled: the job was cancelled before all work completed

Options

  • Retries"max_retries": 0..10 re-runs failed tasks after the initial attempt. The submit-time hold covers the whole retry budget, and infrastructure failures (lost worker, OOM kill) requeue without spending it.
  • Dependencies"depends_on": ["job_id", ...] accepts up to 50 of your job IDs. Dispatch waits until every dependency is completed or partial; a failed or cancelled dependency cancels the blocked job.
  • Artifact handoff between stages — read the concrete artifact ID from the upstream GET /api/v1/results/:job_id, then pass it in payload._artifacts.download. Ownership is verified and the reference recorded before any work or billing hold is created.
  • Tags"tags": ["experiment_42"] (max 20), then filter with GET /api/v1/jobs?tag=experiment_42.
  • Client label"client_ref": "batch_7" (max 255 bytes) groups related jobs; search finished work with GET /api/v1/results?client_ref=batch_7.
  • Batch submitPOST /api/v1/jobs/batch takes up to 200 jobs in one request; successful entries preserve index, payload, meta, and variant.
  • Shared coordination values — store small project-scoped JSON with PUT /api/v1/projects/:name/kv/:key, read it before submission, and pass the resolved value in payload. (payload._shared.resolve is not supported and returns 422.)
  • Webhooks — not yet available: non-empty callback_url values return 422. Use SSE, watch, or polling.

Sizing requirements

requirements are minimums, not machine picks — Computalot may place your task on anything at least that large. Size storage_gb for more than your dataset: the runtime image, writable caches, temp files, checkpoints, and sandbox overhead all share that disk, and heavy ML runtimes often need tens of GB free before any weights download.

If one project serves both light CPU jobs and heavyweight GPU training, split those into separate runtimes rather than shipping one oversized environment everywhere — smaller runtimes place faster.

What job responses expose

Public job, task, watch, and result payloads keep everything you submitted (payload, meta, variant, aggregates, artifact IDs) and redact placement internals (node identities, provider IDs, runtime paths, image digests). Placement is Computalot’s job, not part of yours.

Last updated on