Release notes from AgentEval
Articles about Release notes from AgentEval
Release v0.43.0-beta · AgentEvalHQ/AgentEval
Warning This is a preview release. APIs may change without notice. Not for production use. See DISCLAIMER.md for full details. No verdict reads PASS when something failed or was not measured One release with three parts: no verdict reads PASS when something failed or was not measured (reported in #203,
V0.42.0-beta — MAF 1.23, Gatekeeper v1, over-refusal beside attack success, honest-scoring fixes
Warning This is a preview release. APIs may change without notice. Not for production use. See DISCLAIMER.md for full details. MAF 1.23, Gatekeeper v1, over-refusal beside the attack success rate, and honest-scoring fixes One release with seven parts, newest first below: Microsoft Agent Framework 1.23, with judge-prompt provenance and a block of corrections to earlier claims; Gatekeeper v1; over-refusal reported beside the attack success rate; a composite cannot pass on a minority of its components;
V0.41.0-beta — a decision model, measured on every lane that could use one
Warning This is a preview release. APIs may change without notice. Not for production use. See DISCLAIMER.md for full details. A decision model, measured on every lane that could use one ADR-033 moves to Accepted. What is accepted is the shape — a decision model as its own transport, its own leaf and its own provenance kind. It is not a licence to grade with one: no criterion switches to a decision model in this release, no gate is built, and the escalation primitive the proposal sketched is deliberately not built.
V0.40.0-beta — a third evaluator kind, and the samples stop assuming Azure
Warning This is a preview release. APIs may change without notice. Not for production use. See DISCLAIMER.md for full details. A third evaluator kind, and the samples stop assuming Azure Decision models join code checks and LLM judges as a third evaluator kind (ADR-033, Proposed), and the samples gain a provider selector so they run on Bitdeer, OpenAI, Foundry, Azure or any OpenAI-compatible endpoint. Twelve packaged files changed, nine of them new; the edits to
V0.39.0-beta — the report stops printing only the score
One packaged file changed: TypedMemEvalEvalResultAdapter.cs. No corpus byte moved — all ten corpus_sha256, every question id and every count are identical to 0.36.0-beta. Nothing a consumer pins resets.
V0.38.0-beta — a second dense retriever, published
A second dense retriever is now PUBLISHED, not described 0.37.0-beta named the dense retriever and disclosed, in prose, that the per-shape flag moves under a different embedding model. The consumin
V0.37.0-beta — the dense retriever is named
The dense retriever is named, and no corpus byte moved No corpus content changed. All ten corpus_sha256 values, every question id and every question count are identical to 0.36.0-beta. 587 question
V0.36.0-beta — the calibration echo was a tell, and headroom turned out to be conditional
Two arcs, one release 1 · The calibration echo was a tell — six corpora moved equalise_echo exists so gold and its distractors carry the same number of woven filler terms; otherwise gold is identifiable by having fewer commas. On the seven verticals calibrating per shape it was handed a literal 0.0, so gold's clause was sized at echo 0 while its distractors were woven at the real knob — gold averaged 1.00 terms against filler's 3.56 (episodic), 3.17 (prospective), 2.75 (semantic), 1.93 (arithmetic).