Get started
When to use A8
The judgments A8 is built for, the tools that fit the other jobs and when A8 pays off.
This page as Markdown: https://u22a8.ai/docs/when-to-use-a8.md. The index of every docs page: https://u22a8.ai/llms.txt.
A8 is built for one job: deciding whether a subject meets a criterion, at the minimum accuracy you set. It returns a verdict: a label, a number or a JSON object. Where its evidence falls short of the minimum accuracy, it abstains. For writing text, use a generative model, and use A8 to check what it wrote.
§1What A8 is for
A8 judges text against a criterion stated in plain words. Each request asks one question about one subject, and the answer is a verdict in the shape you give:
- A label: true or false, or one of several categories, such as the queue a ticket belongs in.
- A score: a number on a declared scale, such as how polite a reply is from 1 to 5.
- A JSON object: several fields at once, each judged as its own criterion.
Every verdict carries the level it is promised at, and the same request returns the same verdict.
§2Match the job to the tool
| You want to… | Do this |
|---|---|
| Write, rewrite, translate or summarize text | Use a generative model |
| Decide whether a reply meets a criterion | Call A8 with the criterion as the system message |
| Route a ticket or a document to one of several categories | Call A8 with an enum in the schema |
| Score a reply on a scale | Call A8 with a number or integer property |
| Hold an answer against a reference answer or a policy | Call A8 with reference in the subject |
| Grade the outputs of an LLM app in promptfoo or Inspect | Point the tool at A8; see promptfoo and Inspect |
| Gate a release on the same verdicts every run | Pin an A8 version; see Horizons |
| Show the reasons behind a verdict | Ask A8 for precedents: the taught examples that decided it |
| Judge an image or audio | Use a model that reads images or audio. A8 reads text |
| Move a prompt written for an LLM judge | Send it to A8 as it is; see the page for your tool: OpenAI SDK, Anthropic SDK, promptfoo, Inspect or DeepEval |
Each job beside the tool that does it; A8 covers the rows that end in a verdict.
§3When A8 pays off
- The same criterion runs over many subjects. A criterion you state once judges every ticket, reply or document that follows, and teaching it improves every later verdict.
- A wrong verdict has a cost. You set the minimum accuracy each call must meet. A8 answers where its evidence meets it, and a person takes the cases it abstains on. See Send abstentions to a person.
- The verdict has to repeat. Audits, regression suites and release gates need the same answer for the same request. A pinned version returns the same verdict while its
system_fingerprintholds. - The criterion is yours. House conventions and policies the reference model has yet to see are taught one example at a time, from inside a conversation. See Teaching.
A one-off question, or a judgment that needs a written argument as its answer, suits a generative model.