WTK Engineering Journal
Articles about WTK Engineering Journal
AI Vulnerability Reports and Patch Review | WTK
Receiving an AI security report does not establish that the issue is valid or that its proposed fix is safe. Reproducers and project-specific requirements make those decisions more inspectable.
AI Agent Team Review and Known Failure Modes | WTK
A project can list a known problem and still release an output containing it. Review needs evidence that the current result was checked for that problem, not just another approval.
AI Agent Workflow Evaluation and Approval Rules | WTK
An agent can create a purchasing workflow that looks complete but sends requests through the wrong approval path. Check how the finished process behaves around its required boundaries.
Adaptive AI Model Inference and Executed Compute | WTK
ALoDLM distinguishes token commitment from executed computation. WTK examines what runtime evidence can support resource and deployment claims.
AI Agent Evaluator Execution Errors and Invalid Results | WTK
A scoped WTK diagnostic change separates evaluator execution failures from rejected results without weakening validation or readiness gates.
AI Agent Plugin and Harness Composition Evaluation | WTK
PluginRSI screens reusable components before evaluating assembled harnesses. WTK examines what plugin evidence can support, and what needs workflow-specific qualification.
AI Agent Evaluation of Blocked and Unfinished Tasks | WTK
OSWorld-Pro reports stronger agreement on completion than feasibility. WTK asks what evidence supports retrying, stopping or escalating unfinished agent work.
Source Labels and AI Agent Selection Bias | WTK
Source Preference in the Wild tests domain labels in agent search. WTK examines the distinction between source preference, verified evidence and goal success.
AI Agent Reliability in Repeated Business Workflows | WTK
Thinkingbox checks persistent business records across repeated attempts. WTK examines the gap between finding a success and completing the workflow reliably.
AI Agent Team Selection and Permission Boundaries | WTK
Dynamic Harness Search predicts which agent setup suits a query. WTK examines task-specific catalog composition, selection costs and fixed permission boundaries.
RN-081: Matching AI Agent Checks to the Required Result
VeriHarness reports checks that confirm intermediate quantities while downstream answers remain wrong. WTK asks what evidence settles the actual acceptance requirement.
RN-080: Protecting Secrets Across AI Agent Handoffs
Covert-assistance research asks whether screened agent messages preserve confidentiality through rewriting, recipient recovery, and downstream use.
Interpreting and Revising Intent in AI Agent Packages | WTK
How WTK distinguishes package purpose, goals, and permissions, and what intent-drift research suggests about retaining authorized revisions.
AI Agent Evaluator Stability and Discrimination | WTK
A reviewer study shows why stable scores across rhetorical rewrites are insufficient when a gate cannot distinguish evidence that should lead to different decisions.
AI Agent Action Gates Need State Evidence | WTK
A tool-agent study separates an action request from the resource state that gives it meaning. WTK asks what bounded evidence a target qualification claim must retain.
AI Agent Prompt Logs, Tokenization, and Authority | WTK
A prompt-injection study shows that identical visible text can produce different model-facing token representations. WTK asks what target qualification must bind when untrusted input reaches an agent.
AI Agent Workflow Revisions and Repair Reliability | WTK
ControlScope separates allowed revisions, chosen repairs and execution stability. WTK asks how agents can repair workflows within delegated authority.
AI Review Gates: Errors Prevented and Corrections Lost | WTK
A study of LLM debate counts harmful answer changes and useful corrections separately. WTK considers how to evaluate oversight without equating intervention with value.
RN-073: Defining Success Before AI Improves the System
WTK examines Anthropic’s account of automated AI development and distinguishes changing a method from changing its goals, evaluation or deployment authority.
Evaluating AI Agent Team Contributions and Handoffs | WTK
AgentWorld studies collaboration in a game sandbox. WTK asks how role and handoff evidence can explain team outcomes without treating model judgments as proof.
RN-071: Choosing Signals and Triggers for Agent Improvement
Two preprints sharpen WTK’s questions about when telemetry or failures should trigger improvement work, and whether the response beats a simpler baseline.
Governing AI Agent Workflows and Subagent Controls | WTK
WTK examines JAZ’s recursive agent framework and asks how permissions, budgets and required checks should apply to workflows generated during execution.
AI Agent Evaluation: Quick Checks and Release Evidence | WTK
LangChain’s evaluation workflow raises a WTK question: when do quick development checks miss changes that a fuller release review would reject?
Independent AI Agent Assessments: Scope and Access | WTK
WTK interprets OpenAI’s proposed assessment principles through review scope, evidence access and explicit limits on conclusions about agent deployments.
AI Agent Activity Logs: Checking for Trace Tampering | WTK
WTK examines a trace-tampering study and proposes testing whether a target’s evidence collector is independent from the agent it assesses.
AI Agent Goal Understanding: Checking User Preferences | WTK
WTK examines Project Swap’s evidence that agent performance against an inferred preference model can diverge from outcomes scored against the principal’s own ranking.
Bounded Semantic Decisions for AI Agents and Teams | WTK
A WTK Field Note on exploring Jev for source screening, tool selection, context filtering and team handoffs, with explicit authority and evidence boundaries.
RN-065: Evaluating AI Agents Without Leaking Prior Task Outcomes
WTK examines how advice from earlier attempts changes an agent evaluation, and why assisted retries should be distinguished from performance on unseen tasks.
Testing AI Agent Evaluation Checkers for Fault Coverage | WTK
A GPU-kernel oracle study shows why WTK should measure checker fault coverage and false rejections separately from a passing evaluation receipt.
RN-064: Defining Authority for Automated Agent Approvals
WTK examines OpenAI’s auto-review report and asks which agent actions a reviewer may authorize, which controls remain enforced, and when people must decide.
RN-061: Checking Agent Summaries Before Work Resumes
OpenAI’s compaction incident reports raise a WTK review question: does resumed work preserve the original instructions, unresolved problems, and evidence?
Measuring AI Agent Research Outcomes and Human Rework | WTK
OpenAI’s internal research-acceleration report motivates separating agent activity, verified outcomes, and human correction when evaluating WTK workflows.
AI Agent Test-Time Budget Allocation | WTK
A preprint on agent test-time scaling suggests measuring serial and parallel budget allocations under matched conditions, not treating a larger budget as an improvement by itself.
Evaluating AI Agent Runtime Configurations | WTK
A coding-harness study motivates evaluating supported runtime settings for WTK deployments, while separating task performance from security enforcement.
RN-058: Checking Shared Services for Unauthorized Agent Communication
OpenAI’s Hugging Face incident report raises a WTK deployment-review question: can shared services connect agents that were approved to work separately?
AI Agent Safety When a Task Exceeds Its Permissions | WTK
Anthropic reports real internet access in supposedly offline evaluations. WTK draws lessons for deployment review, explicit permissions, and safely blocked tasks.
RN-056: A Warning Is Not Containment
A long-running multi-agent study separates threat recognition from restraint, memory cleanup, propagation control, and recovery. WTK proposes a bounded target-harness test, not a safety claim.
AI Agent Evaluation Errors and Acceptance Criteria Gaps | WTK
Research on rubric-based reward hacking distinguishes evaluator mistakes from incomplete criteria. WTK asks how to diagnose each without confusing a test pass with task success.
AI Agent Monitoring and Prevention: A Proposed WTK Test
MOLE measures retrospective AI-agent threat detection. WTK proposes testing whether an additional action-time control improves existing protections.