Cheat sheet¶

Make, in workshop order¶

# Onboarding: key + gateway
make setup orq-doctor gateway-check

# Agent: create, inspect, and chat
make seed-workspace
make invoke-agent
make agent-chat-ui

# Simulation and prompt versions
make simulate-agent [VERSION=1.0.0]
make evaluatorq-dashboard
uv run eq sim runs  # saved reports: .evaluatorq/sim-runs/ (or $EVALUATORQ_DIR/sim-runs/)
# Reuse a specific v1 report's cases and opening messages against v2:
make simulate-agent VERSION=2.0.0 REPLAY=".evaluatorq/sim-runs/<your-v1-baseline>.json"
make publish-agent-version [FILE=...]
make compare-versions [VERSION=latest]

# Regression gate and cleanup
make seed-eval  # once: creates your dataset and judge
make regression-eval [VERSION=2.0.0]
make reset-workspace

The gateway¶

$ORQ_API_KEY and $MODEL come from your .env; export them once per terminal as in Onboarding.

curl -s https://my.orq.ai/v3/router/responses \
  -H "Authorization: Bearer $ORQ_API_KEY" \
  -H "Content-Type: application/json" \
  --data "{
    \"model\": \"$MODEL\",
    \"input\": \"hi\"
  }"

# The Model Garden
orq models list -o json

orq responses create \
  --model "$MODEL" \
  --input '"hi"' \
  -o json

from openai import OpenAI
client = OpenAI(
    api_key=ORQ_API_KEY,
    base_url="https://my.orq.ai/v3/router",
)
client.responses.create(
    model=MODEL,
    input="hi",
).output_text

Your agent¶

Studio shortcuts: Agents · Tools · Traces · Datasets · Evaluators · Experiments · Models.

Select the project from your email's ORQ_PROJECT, then search for your seat prefix.

What	Value
Agent key	`<WS_PREFIX>-refund-agent`, e.g. `ws07-refund-agent`
Model id	`agent/ws07-refund-agent` (latest), `agent/ws07-refund-agent@1.0.0` (pinned)
Responses endpoint	`https://my.orq.ai/v3/router/responses`
Code tools	`ws07-lookup_order`, `ws07-issue_refund`, `ws07-get_policy`
Dataset, judge	`ws07-refund-eval`, `ws07-refund-policy-judge`

from orq_ai_sdk import Orq
orq = Orq(api_key=ORQ_API_KEY)
r = orq.responses.create(
    model="agent/ws07-refund-agent@1.0.0",
    input="Refund ord_a1, I changed my mind.",
)
# r.output: function_call items, orq:<tool key> items with result,
# then a message item

orq responses create \
  --model agent/ws07-refund-agent \
  --input '"Refund ord_a1"' \
  -o json

orq agents retrieve ws07-refund-agent -o json
orq traces conversation <trace_id>

Docs¶

Topic	Page
Agents	Build agents · Run agents · Responses API
Quality	Agent simulations · Evaluators · Datasets · Experiments
Coding agents	orq.ai MCP server · orq.ai Skills · orq.ai CLI