A8 with the Anthropic SDK

The Anthropic SDK calls A8 through the Messages API at https://api.u22a8.ai/eval. The SDK adds /v1/messages itself. This page builds a judge on it, then moves a judge prompt that runs on another model.

§1Before you start

Create an API key with the models:infer permission, as API keys shows, and set it as U22A8_API_KEY.

pip install anthropic
export U22A8_API_KEY="<your key>"

npm install @anthropic-ai/sdk
export U22A8_API_KEY="<your key>"

§2Judge one subject

Point the client at https://api.u22a8.ai/eval. system states the criterion, the user message holds the subject, and output_config.format gives the verdict its shape:

import json
import os

from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.u22a8.ai/eval",
    api_key=os.environ["U22A8_API_KEY"],
)

verdict = {
    "type": "json_schema",
    "schema": {
        "type": "object",
        "properties": {"refund": {"type": "boolean"}},
    },
}

message = client.messages.create(
    model="a8-1-1:95@20260715t140322z",  # pinned, at a minimum accuracy of 95
    max_tokens=1024,  # the SDK requires it
    system="Does the reply promise a refund?",
    messages=[
        {"role": "user", "content": "We will refund you in full by Friday."},
    ],
    output_config={"format": verdict},
)
for block in message.content:  # empty on an abstention
    print(json.loads(block.text))  # {'refund': True}

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL: "https://api.u22a8.ai/eval",
  apiKey: process.env.U22A8_API_KEY,
});

const message = await client.messages.create({
  model: "a8-1-1:95@20260715t140322z", // pinned, at a minimum accuracy of 95
  max_tokens: 1024, // the SDK requires it
  system: "Does the reply promise a refund?",
  messages: [
    { role: "user", content: "We will refund you in full by Friday." },
  ],
  output_config: {
    format: {
      type: "json_schema",
      schema: {
        type: "object",
        properties: { refund: { type: "boolean" } },
      },
    },
  },
});
const block = message.content[0];
if (block?.type === "text") console.log(JSON.parse(block.text));

{"refund": true}

Recorded from a8-1-1@20260715t140322z on 2026-10-26.

A pinned identifier keeps a test on one version, and its tag holds the minimum accuracy. a8 follows your organization’s latest version. See Model identifiers.

§3Set the minimum accuracy

The model's tag sets the minimum accuracy: model="a8:95" asks for 95, and a pinned identifier takes the same tag, as in a8-1-1:95@20260715t140322z. A whole number from 0 to 99 passes. The bare a8 asks for 90. What the minimum accuracy promises is on Accuracy and abstention.

thinking is the reasoning switch. thinking={"type": "disabled"} reads the request as it is laid out: system is the criterion and the latest user message the subject. adaptive, enabled and the default read the request however it is written. See Anthropic Messages.

§4Read an abstention

An abstention returns 200. stop_reason is "refusal" and content is an empty list. The SDK keeps A8’s abstention object in message.model_extra:

if message.stop_reason == "refusal":
    abstention = message.model_extra["abstention"]
    print(abstention["kind"])  # "too_close"
    print(abstention["answerable_at"])  # 80: the level that answers it
else:
    print(json.loads(message.content[0].text)["refund"])

In a test run, count an abstention as unanswered: it lowers coverage and leaves accuracy as it is. See Testing.

§5Move a judge prompt

A judge prompt that runs on Claude through the Anthropic SDK moves in four changes:

1. Set base_url to https://api.u22a8.ai/eval and api_key to your A8 key.
2. Set model to a pinned identifier, with the minimum accuracy as its tag, such as a8-1-1:95@20260715t140322z.
3. Remove thinking={"type": "disabled"} if the old configuration sends it. With reasoning off, A8 reads the layout as written, and a prompt that holds the criterion and the subject in one message needs reasoning on.
4. Check stop_reason before you read the verdict.

The prompt itself stays. A8 reads a prompt that holds the criterion and the subject in one message as it is, and answers in the format the prompt asks for:

GRADER = """You are grading a support reply.
Question: Does the reply promise a refund?
Reply: {reply}
Answer with Y or N."""

message = client.messages.create(
    model="a8-1-1:95@20260715t140322z",
    max_tokens=1024,
    messages=[{"role": "user", "content": GRADER.format(reply=reply)}],
)
print(message.content[0].text)  # "Y"

A8 accepts max_tokens, temperature and top_p as they are, and ignores them.

§6Errors on a moved judge