LISTEN TO THIS ARTICLE

ToolPrivacyBench turns agent privacy into a tool-call audit: when a workflow succeeds, did each tool receive only the private facts it needed? The June 2026 paper matters because it separates task completion from purpose-bound disclosure, a failure mode that ordinary function-calling scores can miss ToolPrivacyBench.

Evidence base: ToolPrivacyBench, Anthropic's agent-autonomy and trustworthy-agent research, Human-on-the-Bridge evaluation work, and Swarm Signal coverage on trust boundaries and agent ownership ToolPrivacyBench.

Key takeaways

  • Main change: privacy evaluation is moving from final answers to full tool-call trajectories.
  • Practical implication: a successful workflow can still leak unnecessary private facts into tickets, handoffs or free-text fields.
  • Caveat or risk: ToolPrivacyBench uses mock business backends, so it diagnoses a control gap rather than proving the same rates in every production stack.
  • Recommendation: score purpose-bound disclosure alongside task success before giving tool agents sensitive workflows.

The same runs still showed multi-tool privacy over-disclosure.

The Privacy Boundary Is Purpose, Not Sensitivity

ToolPrivacyBench defines the problem as purpose-bound over-disclosure. A private fact is not automatically forbidden everywhere. It may be necessary for one tool and inappropriate for the next. A symptom can belong in a clinical record but not in a payment note; a phone number can be justified for notification but not for an internal handoff ToolPrivacyBench.

That distinction is useful because most tool-agent checks still focus on whether the model chose the right function and passed valid arguments. The June 2026 ToolPrivacyBench paper asks a different question: did the agent route each current-task private atom only to the tools and downstream sinks authorised for that purpose ToolPrivacyBench?

The June 2026 benchmark represents each case with a policy knowledge base covering private atoms, tool purposes, sink types, allowed and forbidden field-tool relations, free-text slots and backend audit evidence ToolPrivacyBench. After the agent acts, the evaluator compares recorded tool arguments and mock-backend logs against that policy. The privacy judgement is therefore attached to what the agent actually sent, not only to the answer the user saw ToolPrivacyBench.

That is the missing layer between agent data-injection trust boundaries and agent security ownership. Trust boundaries decide which inputs and tools deserve access. Purpose-bound disclosure decides which facts each allowed tool should receive during a specific workflow.

Task Success Hides The Leak

The headline result is uncomfortable for teams that use task completion as their release gate. ToolPrivacyBench contains 2,150 multi-tool cases, with 1,150 fully synthetic privacy-sensitive business workflows and 1,000 cases adapted from public multi-tool and function-calling benchmarks ToolPrivacyBench.

Across the synthetic private split, the tested agents completed tasks at high rates: TaskSuccess ranged from 92.23% to 97.70% ToolPrivacyBench. The same runs still showed multi-tool privacy over-disclosure. The benchmark's MT-POI privacy-risk metric remained between 19.19 and 28.04, which means workflow success did not imply appropriate disclosure ToolPrivacyBench.

The paper identifies tickets and handoffs as frequent leakage locations, with aggregated forbidden-opportunity rates of 51.43 and 34.79 respectively ToolPrivacyBench. Free-text business fields also repeat as channels for over-disclosure ToolPrivacyBench. That pattern should feel familiar to anyone who has reviewed agent traces: the agent writes a convenient narrative, and the narrative carries more private detail than the next recipient needed.

This is not prompt injection. The paper explicitly evaluates a benign workflow setting: no attacker changes the prompt, tools, backend or audit log ToolPrivacyBench. The failure is ordinary over-sharing during tool use. That makes it more operationally important, not less. A system can be well-intentioned, helpful and still too loose with private facts.

Builders should test whether the agent can write a ticket or handoff using role-appropriate facts only, not the easiest complete narrative.

Why Trajectory Audits Matter

Anthropic's agent-autonomy research defines an agent operationally as a system equipped with tools that let it take actions such as running code, calling external APIs and sending messages Anthropic. The same post notes a measurement limit: broad API traffic can show individual tool calls, while product-level traces can show whole workflows, and neither view alone answers every autonomy question Anthropic.

ToolPrivacyBench lands inside that gap. Privacy risk is not always visible in the final answer or in a single API-call success bit; the benchmark measures it through recorded tool arguments and backend audit logs ToolPrivacyBench. It appears in the sequence: which tool was called, which argument field carried which fact, whether the sink needed that fact, and whether a later handoff repeated it.

Human-on-the-Bridge makes a related evaluation argument. Its June 2026 paper says agent failures can include phantom tool calls, missing mandatory tool calls, policy drift, manipulation paths and safe but non-resolving refusals; those failures are difficult to catch through static benchmarks or single-response scoring Human-on-the-Bridge. ToolPrivacyBench adds a privacy-specific case: the agent may call the right tools and still fail the information-flow policy.

Inference from these sources: production evaluation should treat tool traces as evidence, not debugging residue. If a workflow handles health, finance, tax, recruiting, education, security or customer-support data, the release gate should be able to answer which private atoms moved through which tools and why; those domains appear in ToolPrivacyBench's benchmark construction and representative workflow set ToolPrivacyBench.

Controls To Test Before Deployment

The first control is a field-tool authorisation matrix. For each workflow, list the private facts the user may provide, the tools the agent can call, the sink attached to each tool, and whether each fact is necessary for that purpose. This does not need to be perfect on day one. It needs to be explicit enough that a trace reviewer can distinguish authorised disclosure from convenience copying.

The second control is free-text containment. ToolPrivacyBench's leakage pattern points at notes, descriptions, summaries and handoffs because free-text fields invite the model to restate the whole story ToolPrivacyBench. Builders should test whether the agent can write a ticket or handoff using role-appropriate facts only, not the easiest complete narrative.

The third control is sink-aware logging. A conventional trace may show that a create_ticket call succeeded. A privacy audit needs the ticket body, the receiving system, the allowed purpose, and the exact private atoms that appeared. Without that, the team can prove task completion but not data minimisation.

Anthropic's trustworthy-agent guidance reaches the same operating posture from the security side: agent reliability depends on choices about tools, data, permissions and environments, and shared benchmarks remain underdeveloped for agent resistance to attacks and uncertainty surfacing Anthropic. Purpose-bound privacy belongs in that same infrastructure layer.

The Procurement Question

If a vendor claims its agent is safe for sensitive workflows, ask for a purpose-bound trace report. The useful artefact is not a demo transcript. It is a sample workflow where the agent completed the task, plus a table showing each private fact, each tool call, each sink, and why the disclosure was authorised.

For internal builds, add a small red-team set where completion and privacy pull in different directions. Use healthcare, finance, tax, recruiting or software-security scenarios similar to ToolPrivacyBench's domains ToolPrivacyBench. Seed facts that are necessary for one stage and forbidden in another. Then score task success and over-disclosure separately, matching the benchmark's separate TaskSuccess and MT-POI metrics ToolPrivacyBench.

The decision changes when a system can complete the workflow while proving data minimisation. Until then, a high task-success score is incomplete evidence; ToolPrivacyBench reports high TaskSuccess alongside persistent over-disclosure on MT-POI ToolPrivacyBench. It says the agent can get the job done. It does not say the agent kept the right facts inside the right purpose boundary.

Source trail

Research:

Industry and evaluation context:

Related Swarm Signal analysis: