Open-model verification report
Each row is one open model served from one source, checked on the date shown. The checks are the input token count of a fixed prompt against the model maker’s published tokenizer, a tool call, the cache, time to first token and availability. Every row is generated from a conformance report; none is typed by hand. Run date: 2026-11-08.
Results by model and source
Each source appears under an anonymous label. A model maker's name appears only where that maker's own API is the source and its terms allow it; no source in this report is named. Every source in this report is an aggregator or reseller. No model maker's own endpoint is measured, so the reference for the fingerprint is the maker's published tokenizer, not a response from the maker.
| Model | Source | Type | Run date | Fingerprint | Tool call | Cache | TTFT | Availability |
|---|---|---|---|---|---|---|---|---|
| glm-4.7 | aggregator-a | Aggregator or reseller | 2026-11-08 | 86 input tokens against the tokenizer reference 86. Within tolerance. | Pass. 7 calls, 100% with arguments that fit the schema. | Pass. 99.3% of the prompt read from cache. | p50 1.80 s, p90 3.13 s. | 100.0% of 56 calls; no server errors. |
| glm-4.7 | aggregator-b | Aggregator or reseller | 2026-11-08 | 83 input tokens against the tokenizer reference 86. Differs; not shown as verified. | Pass. 7 calls, 100% with arguments that fit the schema. | Pass. 98.8% of the prompt read from cache. | Not measured. | 98.0% of 57 calls; 1 server error. |
A count that differs from the reference means the source counted the same prompt differently. It does not say which model answered. In this run, streamed responses from every source carried two usage chunks where one is expected. The cause is in the Apertis relay, not in the source, so it is not counted against any row.
How each check works
The same requests go to every source, one model at a time. Each source is reached through Apertis with a key pinned to that source for the run.
| Check | How it is run | How to read the row |
|---|---|---|
| Input token count | One fixed prompt, printed in full below, mixes scripts, emoji, code and numbers, which tokenizers split differently. The input token count in usage.prompt_tokens is read from each source. The reference is the count of the model maker's published tokenizer on Hugging Face, made locally at a pinned revision, with the prompt as one user message wrapped by the repository's chat template and the generation prompt included. | The count is within tolerance when it is within 2 tokens or 3% of the reference, whichever is larger. A count outside it means the source wraps or splits the prompt differently from the published tokenizer, by another chat template or another tokenizer; it does not say which. A model with no pinned reference shows its token count and no verdict. |
| Tool call | A request declares one function and asks about the weather in Paris with tool_choice set to auto. The check needs a tool call with an id, the declared name and arguments that parse as JSON; a second turn returns the tool result. | The row shows pass or fail, the number of tool calls made across the whole run, and the share whose arguments fit the declared JSON schema. |
| Cache | The same prompt of roughly 7,000 tokens is sent three times with two seconds between calls. Calls two and three must report cached tokens, and a prompt with another first line must report none. | The row shows pass or fail and the share of the third call's prompt that was read from cache. |
| Time to first token | For each streamed request that returned 200, the time from sending the request to the first streamed chunk. | The row shows the median (p50) and the 90th percentile (p90) over the run. A source whose streamed calls gave no first-chunk time shows not measured. |
| Availability | Over every call in the run: calls that returned 200, divided by calls that returned 200, a 5xx or a 408. A 429 is counted separately and is not in this ratio. | The row shows the ratio, the number of calls in the run and the number of server errors. |
How to read the receipts
An open-model response carries three headers that name the model and the source that answered. The source label is the one used in the Source column above.
| Header | Values | Meaning |
|---|---|---|
| X-Apertis-Served-Model | A public model id, for example glm-4.7. | The model id that answered the request, after any fallback. |
| X-Apertis-Source | A source label from the Source column of the results table, or undisclosed, or apertis-cache. | The source that answered. A label matches the rows with that label in this report. undisclosed means the source is not in the report; apertis-cache means the answer came from the Apertis cache. |
| X-Apertis-Source-Type | maker, aggregator, cache or unregistered. | The kind of source. It matches the Type column of the results table for a label, and is cache or unregistered when there is no row to match. |
Match a receipt to a row
- Send a request for an open model and keep the response headers (curl -i, or the Network panel).
- Read X-Apertis-Served-Model and X-Apertis-Source.
- Find the row in the results table with that model and that label.
- Read the row's fingerprint, tool call, cache, TTFT and availability results, and its run date.
Rerun the fingerprint
The fingerprint prompt is public, and so is the reference: the maker’s tokenizer repository, the revision and the chat template it was counted with.
- Run the request below with your own Apertis key. The body is the fingerprint prompt with max_tokens 16, the same request the report used.
- Read usage.prompt_tokens in the JSON response and compare it with the tokenizer reference for the model.
- Read X-Apertis-Source in the response headers and find that label in the results table.
- To recount the reference, load the tokenizer from the repository at the revision listed below, apply its chat template to the prompt as one user message with the generation prompt, and count the tokens.
Tokenizer references
- glm-4.7: 86 input tokens. Repository zai-org/GLM-4.7, revision 602d01efcdd332c5238ca4bcede555defbe83eb7, chat template chat_template.jinja (sha256 d63ad536c3c81880043e22ec7fd08db42b4d8fb7c89c7138bc562bfa25281375), generation prompt included.
The prompt
Tokenizer probe: 東京タワー 🗼🚀 naïve façade — def ƒ(x): return x**2 # 0x7FFF 3.14159 Ελληνικά العربية हिन्दी 한국어 ∑∫√ ```json {"k": [1, 2, 3]}``` Reply with the word ok.
Request
curl -i https://api.apertis.ai/v1/chat/completions \
-H "Authorization: Bearer $APERTIS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "glm-4.7",
"messages": [
{
"role": "user",
"content": "Tokenizer probe: 東京タワー 🗼🚀 naïve façade — def ƒ(x): return x**2 # 0x7FFF 3.14159 Ελληνικά العربية हिन्दी 한국어 ∑∫√ ```json {\"k\": [1, 2, 3]}``` Reply with the word ok."
}
],
"max_tokens": 16
}'Dates and limits
- The reference is a count from a published tokenizer and chat template, not a response from the model maker. A source that applies another chat template counts differently without serving another model.
- Each row is a snapshot of one run on the date shown. A source can change after that date, and a new run adds dated rows.
- A row describes one model on one source. It is not a statement about any other model on that source.
- The report is regenerated when a model or a source is added, and at least once more within 14 days of launch. Each publication is a dated snapshot.
A markdown version of this page is at /verification.md.