Signals
Production-oriented research signals and interpretation for AI systems builders.
Guides and explainers
Detailed guides and practical technical analysis.
Latest analysis
Recent research, benchmark reviews and technical updates.
Practical tools
Templates for budgets and project planning
BoredTools offers practical spreadsheets for budgets, freelance work and small projects.
Coding Agent Benchmarks Hit the Generalization Wall
Scale's SWE-Bench Pro public leaderboard reports that top models scoring above 70% on SWE-Bench Verified fall to 23.3% for OpenAI GPT-5 and 23.1% for...
Agent Test-Time Scaling Needs Reuse, Not More Rollouts
General AgentBench reports that running agents for more interaction steps or more sampled trajectories did not reliably improve ten leading agents,...
Multi-Agent Finance Workflows Need Cost Curves, Not More Agents
A March 2026 benchmark on financial-document processing makes the uncomfortable point: the most accurate multi-agent architecture was not the obvious...
Coding Agents Need Trajectory Reviews, Not Pass Bits
Most coding-agent benchmarks still compress a whole run into one bit: did the task pass? AgentLens argues that users experience the whole trajectory...
Tool-Use Agents Need Failure Labels, Not Pass Rates
Tool-use agents can fail in ways a final accuracy score hides, because the same wrong answer can come from skipped tools, ignored outputs, fabricated...
Agent Observability Needs Provenance, Not More Logs
Agent observability is drifting toward a familiar trap: capture every trace, then ask an engineer to work out why the agent did the wrong thing. A June...
Agent Leaderboards Can Be Cheaper Without Being Safer
A March 2026 paper on efficient agent benchmarking found that mid-difficulty task subsets can remove large parts of an agent benchmark while preserving...
Multi-Agent Systems Need Specs Before More Agents
Multi-agent systems are getting easier to assemble and harder to trust. A new June 2026 paper from Cisco researchers argues that the missing layer is not...
Power Grid Agents Need Constraint Tests, Not Chat Scores
A June 2026 power-systems benchmark argues that language-model agents can solve grid-engineering tasks, but the useful signal is narrower: the agent must...
Agent Tool Menus Are a Safety Surface
New agent benchmarks suggest the visible tool menu is not a neutral implementation detail. It changes success, cost, wrong-tool calls, and risk exposure.