Red Teaming (eq redteam)¶

Red teaming subcommand group. Always registered; the redteam extra (evaluatorq[redteam]) supplies the optional dependencies some of its commands need at runtime.

eq redteam run¶

Run adversarial red teaming against one or more targets.

eq redteam run --target agent:<key> [OPTIONS]

Delivery methods (--delivery-method): DAN, role-play, skeleton-key, base64, leetspeak, multilingual, character-spacing, crescendo, many-shot, authority-impersonation, refusal-suppression, direct-request, code-elicitation, code-assistance, tool-response, word-substitution.

Saving results. Persistence is controlled by two flags. --save accepts none (no files), final (summary JSON only), or detail (all per-stage artifacts). --artifacts-dir DIR sets where JSON is written and is required when --save detail (--output-dir was removed; use --artifacts-dir). What each file contains, and the fields of the report --report writes, are in the output reference.

Exit codes. eq redteam run exits 1 — after writing any requested report artifacts — in two cases, both read off report.summary:

• Zero verdicts (summary.no_verdict): attacks ran but not one could be evaluated. Always fails; there is no setting that disables this.
• Coverage below the floor (summary.coverage_below_minimum): fewer than --min-evaluation-coverage (default 0.8) of attacks got a verdict. A run that finishes at 79% coverage now exits 1, not 0 with a warning — the same run used to pass. Pass --min-evaluation-coverage 0 to warn instead of failing, or a higher value to be stricter. The Python equivalent is EvaluatorConfig.min_evaluation_coverage (None there also means warn-only) — see Red Teaming › In CI.

Both cases print the dominant failure cause with a sample message before exiting: an evaluation/<code> (timeout / parse / api_connection / api_status / scorer_exception) when the judge failed, or an execution/<code> when the target failed and there was nothing to judge. Either way a systematically blocked run is diagnosable from the CLI output alone.

Driving the CLI from a config file¶

--config takes the keyword arguments of the Python red_team() function as one JSON object, so every data-shaped parameter is reachable from the CLI, including the ones without a flag (datapoints, attack_techniques, description, the full llm_config, a tuned recommendations object). Pass a path, or - to read it from stdin. YAML is not accepted.

{
  "target": "agent:my-agent",
  "mode": "dynamic",
  "vulnerabilities": ["goal_hijacking", "prompt_injection"],
  "max_turns": 3,
  "llm_config": {
    "attacker": {"model": "openai/gpt-5.6-luna", "temperature": 0.9},
    "evaluator": {"model": "openai/gpt-5.6-luna"}
  },
  "recommendations": {"max_areas": 2},
  "save": "none"
}

eq redteam run --config run.json --json > report.json
cat run.json | eq redteam run --config - --max-turns 5 --json

A flag passed on the command line beats the file, and the file beats the default. The command checks where a value came from, not what it is, so --max-turns 5 wins over "max_turns": 3 even if 5 were the default. A null in the file means "not set" and falls back to the default. --llm-config and --system-prompt merge into the file's llm_config and target_config rather than replacing them.

Unknown keys fail the run before anything executes, at every depth: "max_turn" at the top level and "temprature" inside llm_config are both rejected with the field path. "target" takes only the string forms; an AgentTarget object, hooks and llm_client stay Python-only.

With --json, stdout carries the RedTeamReport and nothing else, so it pipes straight into jq or a file. The coverage gates above still apply: a run below the floor prints its report and then exits 1.

eq redteam schema¶

Print a JSON schema: what eq redteam run --config accepts, or what eq redteam run --json prints.

eq redteam schema [--input | --output]

eq redteam validate-dataset¶

Validate the shape of a red team dataset.

eq redteam validate-dataset [DATASET]

eq redteam runs¶

List previously saved red team runs.

eq redteam runs [PATH] [--limit N] [--json]