Profile evidence and freshness¶
Framework and coding-harness profiles display a summary directly below the page title. Read it before using a feature statement to make a product decision.
Evidence labels¶
| Label | What Agentic Arena has established | What it does not establish |
|---|---|---|
| Source-reviewed | The linked source and documentation were inspected at the displayed revision or release. | Runtime behavior, task quality, latency, cost, or safety. |
| Diagnostic only | A bounded integration or protocol probe ran and exposed a specific incompatibility. | A fair score or complete capability assessment. |
| Protocol mismatch | The product cannot enter the shared experiment without changing a controlled boundary. | That the product is broken or unsuitable for its intended protocol. |
| Mock-tested | The real adapter ran against deterministic scripts, so wiring and controlled mechanics can be compared. | Native model quality, provider latency, token billing, or production reliability. |
| Native live | Repeated runs used the provider's native API with the recorded model and controls. | Performance outside the recorded task, model, configuration, and date. |
| Independently reproduced | A second environment reproduced the declared result and retained its artifacts. | Universal behavior outside the reproduced contract. |
An evidence label describes the strongest evidence supporting that profile. Read the page's Evidence here field and the linked result before interpreting a specific claim. The run-mode guide explains the separate mock, Codex, and native-live boundaries.
Freshness fields¶
- Reviewed version identifies the package version, release, or commit that the page describes.
- Verified is the date Agentic Arena last checked the declared evidence.
- Review due is the latest acceptable revalidation date.
- Review overdue means the profile must be checked again before its details are presented as current.
The profile metadata test fails when a required field is missing, a status is unknown, or the revalidation date has passed. Updating a date without checking the source or rerunning the stated evidence is not a valid revalidation.
Comparison rule¶
Compare claims only when their evidence contracts match. A source review and a mock result can inform a test plan, but neither belongs in a native provider ranking. Use the evidence workspace to inspect whether two proposed records support the same claim.