The independent daily for Salesforce admins, developers and architects

Curated by Abhinav (opens in a new tab)

Recommended

Salesforce Agent Fabric adds cross-platform evals, due January 2027 (opens in a new tab)

You will learn what Salesforce's planned Agent Evals and API Evals do, how the availability timeline looks, and how they compare with evaluation tools already on the market.

Why read this

You will also see where the promised features are still only roadmap.

  1. Define what the agent should and should not do
  2. Generate a test set from requirements
  3. Grade results with an LLM judge and data checks
  4. Quarantine the agent if quality thresholds fail
Agent Evals workflow as described for the January 2027 general availability

Vernon Keenan, writing for SalesforceDevops.net, reviews Salesforce’s Agent Fabric update, which adds Agent Evals and API Evals to its agent control plane. Agent Evals is pitched as working across any platform: it generates tests before deployment, scores production agents against quality thresholds, and can trigger a kill switch. API Evals generates functional and performance tests from API specifications so breaking changes are caught early. Salesforce’s earlier Agentforce Testing Center, generally available in February 2025, covered only Agentforce agents.

The availability timeline matters. Agent Evals and API Evals target beta in November 2026 and general availability in January 2027, and the triggered kill switch also targets January 2027. What is generally available today is the defensive layer: a manual kill switch, a FinOps dashboard, Model Wallet budget limits, cost optimization policies, and federated gateway policies. Salesforce says the policies cut token usage by up to 75% in test scenarios, which Keenan treats as a best case. Cross-platform LLM and MCP server evals are still labeled “coming soon,” and Salesforce has not confirmed live-traffic scoring. Keenan argues that enterprises are buying the control plane mainly to manage sprawl, while orchestration features wait on customers’ workflows.

Keenan compares the offering with tools already on the market, including Arize Phoenix, LangSmith, Langfuse and Braintrust, which offer evals and tracing, some with online scoring. He also cites AWS AgentCore Evaluations, Google Cloud agent evaluation, IBM watsonx Orchestrate and ServiceNow AI Control Tower, which has a kill switch that revokes credentials. Developer tools instrument the agent’s own build, while Agent Fabric governs agents its customers did not build and can pause them. A key unanswered question is whether Agent Evals will show step-level traces for third-party agents, since a failing score without a trace explains little.

In his closing section, Keenan urges Salesforce to make evals the centerpiece. His recommendations include shipping a versioned test suite with each context pack, making eval libraries exportable in open formats, grading live traffic with linked traces, and bringing Testing Center customers into Agent Evals. He also asks Salesforce to publish a reproducible benchmark before the January general availability, comparing a frontier model alone, the same model with equivalent data access, and the model using the Enterprise AI Harness.

Read the original at SalesforceDevOps.net (opens in a new tab)

Worth reading?