Agent Swarms

Run the same evaluation script across many inputs in parallel.

Fan-out by list


curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  https://computalot.com/api/v1/jobs \
  -d '{
    "type": "structured_runner",
    "runner_command": ["python", "evaluate.py"],
    "payload": {
      "models": ["gpt-4", "claude-3", "llama-70b", "mixtral"],
      "dataset": "eval_v3"
    },
    "fan_out": {"by": "models"},
    "merge_strategy": "keyed",
    "project": "my-project",
    "timeout_s": 600
  }'

Creates 4 tasks. Each receives {"models": "gpt-4", "dataset": "eval_v3"}, etc.

Runner script


import json, os
 
payload = json.load(open(os.environ["COMPUTALOT_TASK_PAYLOAD"]))
score = run_evaluation(payload["model"], payload["dataset"])
 
with open(os.environ["COMPUTALOT_TASK_RESULT"], "w") as f:
    json.dump({"model": payload["model"], "score": score}, f)