Simulation (eq sim)¶
Agent simulation subcommand group. Always registered; the simulation extra (evaluatorq[simulation]) supplies the optional dependencies some of its commands need at runtime. sim is shorthand for convenience — the feature is agent simulation.
Three main verbs: generate (datapoints only), simulate (run against pre-built datapoints), run (generate then simulate in one shot).
The UI — eq dashboard
The way to browse saved simulation runs is the multi-run FastHTML dashboard, eq dashboard .evaluatorq/sim-runs (scopes to simulation) or eq dashboard (both stores). Passing a single JSON report file is an optional direct deep-link.
Going OpenAI-direct? Override the model roles
The default fast and smart models are openai/gpt-6-luna and openai/gpt-6-sol — provider-prefixed, because the default route is the Orq router, which resolves provider/model. With only OPENAI_API_KEY set, calls go straight to OpenAI, which does not know those ids and rejects them. Set EVALUATORQ_FAST_MODEL=gpt-6-luna and EVALUATORQ_SMART_MODEL=gpt-6-sol once, or pass the bare id on every model flag you use. The Models section of the Configuration page covers the roles.
eq sim run¶
Generate personas and scenarios, then run simulations.
Targets — provide exactly one:
| Flag | Description |
|---|---|
--target | agent:<key> or deployment:<key>. Bare values default to agent:<key>. |
--memory-entity | Memory entity_id sent with every agent:<key> (or bare <key>) target call, for agents with a memory store attached. Omit to mint a fresh id per conversation (parallel conversations never share memory); pass one to reuse a specific (e.g. seeded) entity, shared across the run. |
--vercel-url | Vercel AI SDK HTTP endpoint URL. |
--openai-model | OpenAI-compatible model name. Provider resolved from env: ORQ_API_KEY → Orq AI Router; OPENAI_API_KEY → OpenAI-compatible; ANTHROPIC_API_KEY → Anthropic. |
| Flag | Type / Default | Description |
|---|---|---|
--agent-description | str \| None / None | Free-text description of the agent. May be omitted when --target is an Orq agent (fetched automatically). |
--name / -n | str \| None / unset | Run name. Unset unless passed: the SDK then saves the run as sim and names an uploaded experiment simulation-<timestamp>-<id>. |
--sim-model | str \| None / unset | Model for the user-simulator, the judge, persona/scenario/first-message generation, the recommendations pass and the executive summary. Unset, the simulated user, recommendations, executive summary and persona/scenario/first-message generation run on the fast role (openai/gpt-6-luna by default) and the judge runs on the smart role (openai/gpt-6-sol by default); a value here applies to all of them. Sets llm_config.model. |
--max-turns | int / 10 | Maximum conversation turns. Unset unless passed; the SDK default is 10. |
--repetitions | int / 1 | Separate conversations to run for each case. Each is a fresh conversation with the same persona, scenario and opening message. Repeats the conversation, not the judge call. |
--datapoint-parallelism | int / 10 | Concurrent simulations. Unset unless passed; the SDK default is 10. --parallelism is a deprecated alias. |
--llm-parallelism | int / 10 | Ceiling on in-flight LLM requests for the whole run. -1 disables it. |
--num-personas | int / 5 | Number of personas to generate. |
--num-scenarios | int / 5 | Number of scenarios to generate. |
--persona-seed | str (repeatable) / None | Archetype seed for a persona, e.g. "angry retiree" (repeatable). Each seed becomes one persona the LLM fleshes out — overrides --num-personas. Omit to auto-generate. |
--scenario-seed | str (repeatable) / None | Situation seed for a scenario, e.g. "disputes a refund denial" (repeatable). Each seed becomes one scenario — overrides --num-scenarios. Omit to auto-generate. |
--generation-instructions | str / '' | Free-text steer applied to every generated persona AND scenario, e.g. "enterprise B2B buyers, replying in German". Stacks on top of any seeds and --edge-case-percentage. Empty leaves the built-in prompts unchanged. |
--target-reasoning-effort | str \| None / None | Reasoning effort pinned on the target agent under test (agent:<key> targets only). Distinct from the user-simulator's and judge's own reasoning effort, which comes from EVALUATORQ_REASONING_EFFORT — see Tuning. |
--evaluator | str (repeatable) / API defaults | Evaluator name(s). Repeatable. |
--save / --no-save | bool / True | Write the run to .evaluatorq/sim-runs/, with a run manifest. The SDK saves it after the recommendations and executive summary are attached. --no-save skips it. |
--recommendations / --no-recommendations | bool / True | Generate LLM remediation suggestions for failures, tied to their concrete cause. On by default; --no-recommendations skips the extra LLM call. Uses --sim-model. |
--datapoints / -d | Path \| None / None | Write generated datapoints to JSONL for reproducible re-runs. |
--results / -r | Path \| None / None | Path to write results JSONL, one SimulationResult per line. |
--report | Path \| None / None | Path to write full SimulationRun report JSON. The output reference describes this file and the --results rows. |
--report-md | Path \| None / None | Directory for an auto-named Markdown report. |
--report-html | Path \| None / None | Directory for an auto-named HTML report. |
--executive-summary / --no-executive-summary | bool / True | Generate an LLM narrative executive summary in the report. The SDK generates it before it saves the run. |
--yes / -y | bool / False | Skip interactive confirmation prompt. |
--verbose / -v | count / 0 | Increase verbosity. -v info; -vv debug. |
--quiet / -q | bool / False | Suppress non-error output. |
--config | JSON \| PATH \| - / None | JSON object (shell-quote it), file path, or - to read JSON from stdin. See Driving the CLI from a config file or inline JSON; eq sim schema --command run prints the accepted shape. |
--llm-config | JSON / None | LLMCallConfig for every simulation-side LLM call as a JSON object. Merged field by field into "llm_config" from --config. --sim-model wins over its "model" when passed. |
--json | bool / False | Print the final SimulationRun as JSON on stdout. Progress and messages go to stderr; exit codes are unchanged. |
eq sim simulate¶
Run simulations from a pre-built datapoints JSONL file.
Targets — same three flags as eq sim run. Provide exactly one input source: --input (-i), --dataset-id, --experiment-id (optionally narrowed by --experiment-run-id), --from-run, or inline datapoints (or personas with scenarios) in --config.
--target-reasoning-effort works here exactly as on eq sim run (see the flag table above), and "target_reasoning_effort" in --config sets the same value.
| Flag | Type / Default | Description |
|---|---|---|
--input / -i | Path \| None | Path to datapoints JSONL file. Mutually exclusive with the other input sources. |
--dataset-id | str \| None | Fetch datapoints from an Orq dataset instead of a local file. Requires ORQ_API_KEY. |
--experiment-id | str \| None | Fetch datapoints from an Orq experiment's rows instead of a local file. Requires ORQ_API_KEY. |
--experiment-run-id | str \| None | Specific run of --experiment-id to load. Latest run if omitted. |
--from-run | str \| None | Replay a previous run from .evaluatorq/sim-runs/: pass its file name, run id, path, or "latest". Re-runs the exact same personas, scenarios, and first messages; only the target/evaluators may differ. |
--memory-entity | str \| None / None | Memory entity_id sent with every agent:<key> (or bare <key>) target call, for agents with a memory store attached. Omit to mint a fresh id per conversation; pass one to reuse a specific (e.g. seeded) entity, shared across the run. |
--name / -n | str \| None / unset | Run name. Unset unless passed: the SDK then saves the run as sim and names an uploaded experiment simulation-<timestamp>-<id>. |
--sim-model | str \| None / unset | Model for the user-simulator, the judge, the recommendations pass and the executive summary. Unset, the simulated user, recommendations and executive summary run on the fast role (openai/gpt-6-luna by default) and the judge runs on the smart role (openai/gpt-6-sol by default); a value here applies to all of them. Sets llm_config.model. |
--max-turns | int / 10 | Maximum conversation turns. Unset unless passed; the SDK uses the replayed run's cap with --from-run, else 10. |
--repetitions | int / 1 | Separate conversations to run for each case. Each is a fresh conversation with the same persona, scenario and opening message. Repeats the conversation, not the judge call. Unset unless passed; the SDK uses the replayed run's count with --from-run, else 1. |
--datapoint-parallelism | int / 10 | Concurrent simulations. Unset unless passed; the SDK default is 10. --parallelism is a deprecated alias. |
--llm-parallelism | int / 10 | Ceiling on in-flight LLM requests for the whole run. -1 disables it. |
--evaluator | str (repeatable) / API defaults | Evaluator name(s). Repeatable. |
--save / --no-save | bool / True | Write the run to .evaluatorq/sim-runs/, with a run manifest. The SDK saves it after the recommendations and executive summary are attached. --no-save skips it. |
--recommendations / --no-recommendations | bool / True | Generate LLM remediation suggestions for failures, tied to their concrete cause. On by default; --no-recommendations skips the extra LLM call. Uses --sim-model. |
--results / -r | Path \| None / None | Path to write results JSONL. |
--report | Path \| None / None | Path to write full SimulationRun report JSON. The output reference describes this file and the --results rows. |
--report-md | Path \| None / None | Directory for an auto-named Markdown report. |
--report-html | Path \| None / None | Directory for an auto-named HTML report. |
--executive-summary / --no-executive-summary | bool / True | Generate an LLM narrative executive summary in the report. The SDK generates it before it saves the run. |
--yes / -y | bool / False | Skip interactive confirmation prompt. |
--verbose / -v | count / 0 | Increase verbosity. |
--quiet / -q | bool / False | Suppress non-error output. |
--config | JSON \| PATH \| - / None | JSON object (shell-quote it), file path, or - to read JSON from stdin. See Driving the CLI from a config file or inline JSON; eq sim schema prints the accepted shape. |
--llm-config | JSON / None | LLMCallConfig for every simulation-side LLM call as a JSON object. Merged field by field into "llm_config" from --config. --sim-model wins over its "model" when passed. |
--json | bool / False | Print the final SimulationRun as JSON on stdout. Progress and messages go to stderr; exit codes are unchanged. |
Driving the CLI from a config file or inline JSON¶
--config on eq sim simulate and eq sim run takes the keyword arguments of the Python simulate() and generate_and_simulate() functions as one JSON object, under their public names (run_name, experiment_description, raise_on_execution_failure). That reaches every data-shaped parameter, including the ones without a flag: inline personas and scenarios or datapoints, scoring, target_agent_timeout_ms, max_tool_result_chars, per_simulation_timeout_s, upload_results, the full llm_config, and on sim run edge_case_percentage. Pass a JSON file path, a shell-quoted inline JSON object, or - to read JSON from stdin. YAML is not accepted. Inline JSON is recognized after leading whitespace is stripped; a filename beginning with { can be passed as ./{name}.json to disambiguate it. Malformed inline JSON is reported as a JSON usage error and is not retried as a file path. For sensitive values, prefer a config file or stdin instead of placing them in shell history or process arguments.
The Python aliases are accepted too: evaluation_name → run_name, evaluation_description → experiment_description, orq_results_path → orq_folder_path, exit_on_failure → raise_on_execution_failure, report → report_path, and parallelism → datapoint_parallelism. Conflicts are rejected using the same rules as the Python API; datapoint_parallelism wins over parallelism. An explicit --name, --report or --datapoint-parallelism replaces the corresponding config alias. Use eq sim schema --command simulate or eq sim schema --command run for the complete accepted shape.
Inline datapoints, or personas with scenarios, count as the input source of eq sim simulate, so the file below needs no --input:
{
"target": "agent:my-agent",
"personas": [
{
"name": "Impatient customer",
"patience": 0.2,
"assertiveness": 0.8,
"politeness": 0.4,
"technical_level": 0.3,
"communication_style": "terse",
"background": "Wants a refund today"
}
],
"scenarios": [
{
"name": "Refund",
"goal": "Get a full refund",
"criteria": [{"description": "Agent asks for the order number", "type": "must_happen"}]
}
],
"max_turns": 6,
"llm_config": {"model": "openai/gpt-5.6-luna", "temperature": 0.2},
"target_reasoning_effort": "low",
"per_simulation_timeout_s": 300
}
eq sim simulate --config sim.json --json > run.json
eq sim simulate --input dp.jsonl --config '{"target":"agent:my-agent","max_turns":6,"save":false}' --json
eq sim run takes the generation keywords the same way:
{
"target": "agent:my-agent",
"num_personas": 4,
"edge_case_percentage": 0.25,
"generation_instructions": "customers replying in German",
"llm_config": {"model": "openai/gpt-5.6-luna"}
}
eq sim run --config gen.json --num-scenarios 3 --json
cat gen.json | eq sim run --config -
eq sim run --config '{"target":"agent:my-agent","num_personas":4,"num_scenarios":3,"save":false}' --json
A flag passed on the command line beats the config, and the config beats the default. The command checks where a value came from, not what it is, so --max-turns 10 wins over "max_turns": 6 even though 10 is the default. A null in the config means "not set" and is accepted only on fields that allow None, such as run_name, max_turns or datapoint_parallelism; the SDK default then applies. A value nobody sets (no flag, no config entry) is never sent: max_turns, datapoint_parallelism and run_name reach the SDK only when you pass them, so --from-run can still restore the replayed cap. "save": false in the config has the effect of --no-save. --llm-config and the config's "llm_config" merge field by field, and --sim-model wins for model.
The target and the input source are each one choice, so a command-line flag replaces the config's whole choice rather than adding a second one. --target, --vercel-url or --openai-model drops the config's "target". On eq sim simulate, --input, --dataset-id, --experiment-id or --from-run drops the config's "datapoints", "personas", "scenarios", "dataset_id", "experiment_id", "experiment_run_id" and "previous_run". --experiment-run-id alone narrows the config's "experiment_id" instead. Two flags for the same choice on the command line are still rejected.
Unknown keys fail the command before anything runs, at every depth, so "num_personas" in a sim simulate config or "patiense" inside a persona is rejected with the field path, regardless of whether the config came from inline JSON, a file, or stdin. "target" takes only the agent:<key> and deployment:<key> string forms; a callable or AgentTarget, user_simulator, judge, hooks and generation_client stay Python-only.
With --json, stdout carries the SimulationRun and nothing else; the run-store save, the report files and every progress line still happen, on stderr.
eq sim schema¶
Print a JSON schema: what --config accepts, or what --json prints.
| Flag | Type / Default | Description |
|---|---|---|
--input / --output | bool / --input | --input prints the --config input schema. --output prints the SimulationRun schema, which both commands share. |
--command | simulate \| run / simulate | Which command's config --input describes: SimulateCliConfig for eq sim simulate, GenerateAndSimulateCliConfig for eq sim run. |
eq sim upload-dataset¶
Upload simulation datapoints to an Orq dataset, or append them to an existing dataset.
eq sim upload-dataset -i cases.jsonl -n "Support simulation set"
eq sim upload-dataset -i more.jsonl --dataset-id <id>
The first command prints the new dataset ID. Use that ID, not the display name, with eq sim simulate --dataset-id or the Python extend_from_dataset() function. Here, upload-dataset --dataset-id adds rows to the stored dataset; extend_from_dataset() generates new cases without changing the stored dataset. Persona and scenario objects are JSON-stringified because the Orq dataset API accepts scalar inputs values. The simulation reader restores them when the dataset is used with eq sim simulate --dataset-id.
| Flag | Type / Default | Description |
|---|---|---|
--input / -i | Path (required) | Raw sim generate JSONL or a --dataset-format JSONL file. |
--name / -n | str \| None | Display name for a new dataset; required unless --dataset-id is provided. |
--path | str / Default | Orq folder path for a new dataset. |
--dataset-id | str \| None | Append to this existing dataset instead of creating one. |
eq sim generate¶
Generate simulation datapoints only — no simulation is run.
| Flag | Type / Default | Description |
|---|---|---|
--datapoints / -d | Path (required) | Path to write generated datapoints JSONL. |
--agent-description | str \| None / None | Free-text description of the agent. |
--target | str \| None / None | Agent target used to fetch the description when --agent-description is omitted. Accepts agent:<key>. |
--sim-model | str \| None / None | Model for persona/scenario/first-message generation. Unset resolves the fast role (openai/gpt-6-luna by default). |
--num-personas | int / 5 | Number of personas to generate. |
--num-scenarios | int / 5 | Number of scenarios to generate. |
--persona-seed | str (repeatable) / None | Archetype seed for a persona, e.g. "angry retiree" (repeatable). Each seed becomes one persona the LLM fleshes out — overrides --num-personas. Omit to auto-generate. |
--scenario-seed | str (repeatable) / None | Situation seed for a scenario, e.g. "disputes refund denial" (repeatable). Each seed becomes one scenario — overrides --num-scenarios. Omit to auto-generate. |
--generation-instructions | str / '' | Free-text steer applied to every generated persona AND scenario, e.g. "enterprise B2B buyers, replying in German". Stacks on top of any seeds and --edge-case-percentage. Empty leaves the built-in prompts unchanged. |
--dataset-format | bool / False | Write Orq dataset-row envelopes instead of raw simulation datapoints. |
--verbose / -v | count / 0 | Increase verbosity. |
--quiet / -q | bool / False | Suppress non-error output. |
eq sim from-traces¶
Build simulation datapoints from Orq production traces.
eq sim from-traces --output dp.jsonl --limit 50 --lookback-hours 24
eq sim from-traces --output dp.jsonl --extend 20 --agent-description "..."
Fetches recent traces from the Orq traces API (requires ORQ_API_KEY) and builds one datapoint per trace conversation: the persona and scenario are inferred from the conversation, and the opening message is written from them rather than replayed. Pass --extend N to additionally generate N new datapoints matching the traffic distribution of the fetched traces (extra LLM calls). Feed the output file to eq sim simulate --input to run it. Why the opening is generated, and what redaction covers, are the guide's: Why the opening message is generated, not replayed and What lands in the generated dataset.
| Flag | Type / Default | Description |
|---|---|---|
--output / -o | Path (required) | Path to write generated datapoints JSONL. |
--limit | int / 20 | Maximum number of traces to fetch. |
--lookback-hours | float \| None | Only fetch traces from the last N hours — a week is 168. Default: no time filter. |
--search | str / "" | Free-text search applied to the trace list. Empty means no search. |
--extend | int / 0 | Also generate N distribution-matched datapoints on top of the direct per-trace ones (extra LLM calls). 0 disables extension. |
--agent-description | str \| None | Agent description used for --extend generation. Optional; if omitted, a generic placeholder is used and the traffic profile alone steers generation. Read only with --extend. |
--sim-model | str \| None / None | Model for every LLM call the command makes: summarize, persona/scenario inference, first-message generation and --extend generation. Unset resolves the fast role (openai/gpt-6-luna by default). |
--generate-first-message / --replay-first-message | bool / True | Write each conversation's opening message from the inferred persona and scenario. --replay-first-message reuses the real user's recorded opening instead, verbatim — it never passes through the summarize step, so --redact-pii does not rewrite it. Sets TraceAnalysisConfig.generate_first_message. |
--redact-pii / --no-redact-pii | bool / True | Instruct the summarize prompt — the only step that reads a raw transcript — to replace identifying values such as names, emails and order numbers with placeholders like [CUSTOMER_NAME]. The persona/scenario inference and traffic-profile prompts see only summaries and are told to carry the placeholders through rather than invent concrete values; first-message and --extend generation get no such instruction. An instruction to a model, not a guarantee. --no-redact-pii turns it off. Sets TraceAnalysisConfig.redact_pii. |
--max-summaries | int / 50 | How many of the trace summaries the --extend traffic-profile call carries — the first N in fetch order, not a sample, with the rest dropped from the profile and a warning naming the counts. Read only with --extend, and it does not lower spend: every fetched trace is summarized before extension is considered. Sets TraceAnalysisConfig.max_reduce_summaries. |
--llm-parallelism | int / 10 | Ceiling on in-flight LLM requests while building datapoints. -1 disables it. |
--verbose / -v | count / 0 | Increase verbosity. |
--quiet / -q | bool / False | Suppress non-error output. |
--agent-description and --max-summaries are accepted and then ignored on a run without --extend, and nothing is logged when they are. --max-summaries also only bites when more conversations were fetched than its value, so with both defaults (--limit 20, --max-summaries 50) it never does.
Redaction has a gap the flag cannot close. The recorded opening is copied from the trace itself, not from the summary, so the redaction instruction never touches it. --replay-first-message writes that recorded text into every datapoint, and on the default a first-message generation that fails falls back to the same recorded opening with a warning rather than dropping the datapoint. Either way the output file can carry a real user's words into a file you commit.
Which datapoint fields can hold real text. Each JSONL row is a SimulationDatapoint: id (trace-<trace_id> for a per-trace row, dp_<hex> for an --extend row), persona (its name and free-text background), scenario (name, goal, and optional context, criteria and ground_truth), user_system_prompt (built from the persona and scenario), and first_message. The model writes the persona and scenario from the summary, so redaction covers them. first_message is the one field that holds recorded text, and only under --replay-first-message or after the fallback above.
Cost scales with traces. Each fetched trace costs one summarize call and one persona/scenario inference call, plus one first-message call on the default. --extend N adds one traffic-profile call, persona and scenario generation, and one first-message call per persona-by-scenario combination, which is normally at least N calls because the persona-by-scenario grid is rounded up to cover N and the surplus is discarded, and fewer can result if persona or scenario generation returns fewer than requested. --limit is therefore the spend control, and --max-summaries is not. --lookback-hours 168 with --limit 100 is a week capped at 100 traces. Which traces fill the limit is decided by the traces API: the command sends no sort, so "the first N in fetch order" means the order the API returned them in.
Exit codes and the output file. The command exits 0 when at least one datapoint was built, even if some traces were dropped (a failed --extend stage is the exception, see below): dropped traces are logged as warnings, with one aggregate line such as 3 of 20 trace conversation(s) produced no datapoint. It exits 1 on a runtime error, which covers a missing ORQ_API_KEY, no usable traces in the window, a run that built no datapoint at all (every trace dropped), and any failed --extend stage: the traffic-profile call failing, persona or scenario generation returning nothing, or every first-message call failing. Per-trace and extension work share one run, so an --extend failure also discards the per-trace datapoints already built and no file is written. It exits 2 on a usage error such as --limit 0, and 130 on Ctrl-C. The file is written once, at the end, so none of these leaves a partial file; an existing file at --output is overwritten in place (not through a temporary file) and missing parent directories are created. The command ends with a stderr line Built N datapoint(s) from M trace(s) -> <output>; the suffix + K distribution-matched datapoint(s) (--extend) is added before the arrow only when --extend added rows, so the line then reads Built N datapoint(s) from M trace(s) + K distribution-matched datapoint(s) (--extend) -> <output>. M counts only conversations that survived the fetch: traces dropped there (no usable conversation, or a span fetch failure) are not in M and show up only in a fetch-time warning such as 2 of 20 fetched trace(s) had no usable conversation and were dropped. Compare N with M if a CI step must fail on a run that dropped many traces, since the exit code will not. --quiet raises the log level to ERROR, so it hides those warnings, but the final Built ... line still prints.
Three of the flags set a field on TraceAnalysisConfig, which datapoints_from_traces() and extend_from_traces() take as config=. One of the three is spelled differently there:
from evaluatorq.simulation import TraceAnalysisConfig
# The Python equivalent of --replay-first-message --no-redact-pii --max-summaries 200.
config = TraceAnalysisConfig(generate_first_message=False, redact_pii=False, max_reduce_summaries=200)
print(config.generate_first_message, config.redact_pii, config.max_reduce_summaries)
# The field is max_reduce_summaries; the flag's own spelling is not a field.
try:
TraceAnalysisConfig(max_summaries=200)
except Exception as exc:
print(type(exc).__name__)
TraceAnalysisConfig forbids unknown fields, so a misspelled one raises instead of being dropped in silence. Its remaining fields have no CLI flag; the guide tabulates them under What happens between a trace and a datapoint.
eq sim export¶
Export simulation results: OpenResponses payload JSON, or an HTML/Markdown report.
eq sim export --input results.jsonl --output payload.json
eq sim export --input sim-report.json --output report.html --format html --recommendations
HTML exports use the dashboard's report layout with each available tab expanded into a section, embedded fonts and styles, and complete conversation evidence. Individual conversation transcripts are foldable and closed by default; the report sections stay expanded. They work offline. When the input is a full SimulationRun JSON file, the HTML preserves its saved run name, target configuration, agent snapshot, recommendations and whole-run usage. A results JSONL file contains conversation rows only, so those run-level details are unavailable.
Markdown/HTML exports include remediation suggestions if the input run JSON already carries them (runs generate them by default — see --no-recommendations), or if --recommendations is passed here to generate them at export time for a run that has none stored.
| Flag | Type / Default | Description |
|---|---|---|
--input / -i | Path (required) | Path to a results JSONL file or a SimulationRun report JSON (--report / --report-output). |
--output / -o | Path (required) | Path to write the exported file. |
--format | str / openresponses | Export format: openresponses (payload JSON), md (Markdown report), html (HTML report). |
--recommendations | bool / False | For md/html: generate LLM remediation suggestions at export time if none are stored. Extra LLM cost; uses --sim-model. |
--sim-model | str \| None / None | Model for --recommendations generation. Unset resolves the fast role (openai/gpt-6-luna by default). |
--target-label | str / agent | Target name shown in md/html report headers. An explicit value overrides the saved target label; HTML otherwise keeps the saved target. |
eq sim validate¶
Validate a simulation datapoints JSONL file.
| Flag / Argument | Type / Default | Description |
|---|---|---|
PATH | Path \| None | Path to datapoints JSONL file (legacy positional form). |
--input / -i | Path \| None | Path to datapoints JSONL file to validate. (validate-dataset is retained as a compatibility alias.) |
eq sim validate-dataset (compatibility alias)¶
Deprecated alias for eq sim validate --input PATH. Retained for compatibility.
| Argument | Type / Default | Description |
|---|---|---|
PATH | Path (required) | Path to datapoints JSONL file to validate. |
eq sim runs¶
List recent simulation runs.
| Flag / Argument | Type / Default | Description |
|---|---|---|
DIRECTORY | Path \| None / None | Directory to scan. Defaults to .evaluatorq/sim-runs/. |
--limit / -n | int / 20 | Maximum number of runs to show. |
--full / -f | bool / False | Render at full content width; do not truncate columns. |
--json | bool / False | Emit runs as a JSON array on stdout. |