Using A8

Write a criterion

State a criterion A8 judges well, with one judgment per criterion, the situation that decides it and the practices that keep verdicts steady.

A criterion is what is being judged, stated in plain words. A8 reads the whole request: the conversation, the schema and the parameters. It finds the criterion, the subject and any standard wherever the request puts them. This page gives the practices that get a criterion judged well, and the clearest layout.

§1Principles

One judgment per criterion. A criterion asks one thing. Two judgments are two criteria, each in a schema property of its own.
Name the situation that decides the answer. “Does the reply promise a refund?” names what anyone reading the reply would see. A8 judges a criterion best when people reading the subject agree on its answer.
Give the verdict a shape. A schema says what comes back: a boolean, one of an enum, or a number on a declared scale.
Send the standard with the subject. A reference answer, a source or the question asked gives A8 something to judge the subject against.
Set the minimum accuracy in the model’s tag. a8:95 asks for 95. A8 reads words about certainty in the prompt as part of the criterion. See Accuracy and abstention.
Keep the wording fixed once you teach it. A8 identifies a teach by its exact input and format, so the same words reach the same examples. See Teaching.

§2Weak and strong, side by side

Weak	Strong	What changed
Is this reply good?	Does the reply answer the question the customer asked?	Names the situation that decides it
Rate the reply’s accuracy and tone.	Two criteria: “Does the reply answer the question?” and a `tone` property with its own description	States one judgment per criterion
Is the reply correct? The refund went out Monday.	“Does the output match the reference?”, with `reference` beside the subject	Sends the standard with the subject
Answer only if you are 95% sure. Is this spam?	“Is the message spam?”, with the model set to `a8:95`	Sets the minimum accuracy in the tag
Score the reply from 1 to 5 for politeness.	“How polite is the reply?”, with an integer property from `1` to `5`	Gives the verdict a shape

Table 1.

Criteria before and after: each strong version states one judgment, gives the verdict a shape and sets the minimum accuracy in the model’s tag.

§3A clear layout

A8 reads a prompt in any layout. The layout below keeps each part easy to find for the people who read your requests too, and it is the layout A8 reads as written with reasoning off:

The criterion in the system message. One question, in plain words.
The subject in the user message. Follow-ups go in later user messages, after A8’s earlier responses. See API overview.
The shape in the response format. A JSON Schema whose properties are the verdict’s fields.
The minimum accuracy in the model’s tag. model="a8:95" asks for 95, in every interface.

§3.1The subject and its standards

Where a judgment relates the subject to something else, send the subject as a JSON object: output, plus any of reference, context and question. The keys are on API overview:

Python

import json

subject = {
    "output": "Your refund went out on Monday.",
    "context": ["Refund issued on Monday to the original card."],
}
messages = [
    {"role": "system", "content": "Is the output supported by the context?"},
    {"role": "user", "content": json.dumps(subject)},
]

A criterion taught against a standard is judged against that standard. A request that leaves it out abstains, and the abstention names the standard in missing.

§3.2The verdict’s shape

Each schema property is one field of the verdict. A boolean or an enum is a label, and a number or integer is a score on its declared scale. See API overview for how A8 reads each property type.

§3.3Several criteria in one request

A property with a description carries its own criterion, and A8 judges it separately. The request’s criterion applies to the other properties. Each field answers or abstains on its own. See Check one subject against several criteria.

§4Prompts written for other judges

A prompt that puts the criterion and the subject in one message, the way many eval frameworks write their graders, works as it is. A8 finds the criterion, the subject and the format the prompt asks for, such as GRADE: C or a trailing Y or N, and answers in that format. A8 reads the same request the same way every time, so it gets the same verdict.

A request that is small talk, a greeting, a statement or several questions at once comes back as an abstention of kind no_question. Send each question as a request of its own. Each tool’s page shows how to point an existing grader at A8: OpenAI SDK, Anthropic SDK, promptfoo, Inspect and DeepEval.

§5Common fixes

Every request abstains. The criterion is untaught, or its evidence backs a lower level than the request asks for. Read answerable_at and earned on the abstention, then teach examples, or send the reference the criterion was taught against.
Two phrasings of one criterion behave differently. Each phrasing is its own request with its own examples. Pick one wording and keep it.
A threshold in the prompt changes the verdict. The words are part of the criterion. Move the threshold to the model’s tag.

Next steps

TeachingHow to teach and retract through MCP, when a teach lands in your organization’s model and what it changes.Check one subject against several criteriaJudge one subject on several criteria in one request, with each criterion in a schema property and each field answering on its own.Accuracy and abstentionHow a request sets the minimum accuracy a verdict must meet, what that level promises and the five kinds of abstention.