Try a fitted judgment

Eight invented tickets, two teams. The first run downloads one small encoder and fits on CPU. Eight rows show the command. They are not a benchmark.

uv sync --extra hf
uv run jjm --profiles examples/try/profile.json fit \
    --profile try-embed --criterion examples/try/team.criterion.json \
    --examples examples/try/examples.jsonl --promote
uv run jjm --profiles examples/try/profile.json evaluate --request examples/try/request.json

The first run downloads Qwen/Qwen3-Embedding-0.6B. It requires Python 3.10+ and uv, and writes the head and feature cache under runs/try/. The envelope remains fitted_unvalidated until scope acceptance; the tutorial request deliberately does not require that acceptance. Full walkthrough and validation rules.

What you might hand it

• A support note, to sort into the teams you already use.
• A receipt, a log, or a short passage, to see whether it settles the question.
• An application or a review, to place into the ratings you defined, once you have labelled a few.
• A tool result or a document snippet, to return as one of the answers you allowed.

What you send

A JSON request with a snapshot and explicit criteria. A choice has two to 26 options, or buckets, you define. An assertion has three outcomes: the supplied evidence supports the claim, contradicts it, or does not settle it.

{
  "state": {
    "content": {
      "ticket": "I was charged twice for the same subscription."
    }
  },
  "criteria": [{
    "id": "route-ticket",
    "kind": "choice",
    "question": "Which team should own this ticket?",
    "options": [
      { "id": "billing" },
      { "id": "technical" },
      { "id": "other" }
    ]
  }],
  "requirements": { "min_top_mass": 0.5 }
}

Shortened from examples/route-ticket.json. The file also names the scope, the profile, and the evidence list.

What you get back

• A weight over every option you allowed.
• Provenance. The method, the model, and digests of the inputs.
• Your check. Pass, hold, or reject against requirements you declared. A pass means those requirements were met. The next action stays with your application.

Weights are a normalized readout. Accepting a held-out report records that scope. It does not change the weights, and it is not a check that a weight of 0.95 occurs about 95% of the time.

For a download-free view of the envelope, use the separate contract demonstration. Its synthetic weights do not classify tickets.

New routing benchmark

On our eight-team Banking77 task, Qwen embeddings with a fitted head through JJM reached 96.69%, pretrained Laya 55.32%, and TF-IDF 95.65%. JJM and TF-IDF used local labels; Laya did not. The bare TF-IDF classifier took 0.40 ms median on CPU.

Read the full comparison, methods, limits, and reproducible results. You can also try TF-IDF as a JJM backend, with the same judgment contract.

What has been tried

Banking77, 77 intents mapped to eight teams, scored against the human labels, on one laptop GPU.

The head had seen the human training labels. Nimble and the chat model had seen none. About 64 labels per team, 512 in all, is where the head matches Nimble. The 97.1% is the earlier fit on the full test set. A refit scored 96.8%. In bulk that run was a few milliseconds per note, about 5 ms. Through the full contract, a batch of 400 took about 11 ms each, and one request at a time with nothing cached took 97 ms median on 100 requests. The other times in the table are medians.

On the same labels, a logistic regression on the same embeddings scored 97.2% of the 3,080, and a word-and-bigram TF-IDF classifier scored 94.9%. The tenth of a point is not a ranking. TF-IDF never calls the encoder.

A cutoff was chosen on 616 of the test messages and scored on the 2,464 held aside from that choice. At a top weight of 0.91 the head handled 2,259 of those 2,464. 19 of the 2,259 were wrong, 0.84%. A 95% Wilson interval for that rate is about 0.54% to 1.31%. That is an observation on this slice, not a guarantee of staying under 1%. A cutoff chosen for 2% error did not hold there: 2.8% of the handled messages were wrong.

Fit rank was one candidate. Agreement with the teacher moved with how the job was written, and agreement is not correctness.

The figures and the hardware are in the experiment log.

Limits

• The eight-ticket walkthrough has been run on the author's machine. It downloads one small encoder and fits those eight rows.
• Routing was measured on Banking77, eight teams, human labels, on one laptop GPU. The fitted head used the training labels. On those same labels it matches a plain classifier on the same embeddings, and it is about 2 points ahead of TF-IDF. About 64 labels per team is where it lines up with Nimble, which had seen no labels.
• Fit rank was measured on one candidate. Agreement with the teacher is not correctness, and it moved with how the job was written.

Check the contract, with no model

"requirements": {
  "min_top_mass": 0.5
}

"prediction": {
  "weights": {
    "billing": 0.892,
    "technical": 0.005,
    "other": 0.103
  },
  "selected": "billing"
},
"acceptance": { "status": "pass" },
"provenance": {
  "method": "mock:normalized",
  "model": { "id": "jjm/synthetic-no-model" }
}

The synthetic path needs no download. Its weights are a hash of the input, so they carry no meaning. They show the envelope.

uv sync
uv run jjm evaluate --request examples/route-ticket.json

JJM depends on AnyJev, Apache-2.0. JJM is not affiliated with TypeSafe AI or Jev.

Built by Catalyst Forge LLC.