The independent daily for Salesforce admins, developers and architects

Curated by Abhinav (opens in a new tab)

Koa Decisions, a small model for fast routing Decisions in Agentforce (opens in a new tab)

You will learn what Koa Decisions is, how it differs from a general-purpose LLM for categorical decisions, and the latency figures Salesforce reports.

Why read this

You will also see the planned extension to flows, Apex and Prompt Templates.

General-purpose LLM

  • Generates text token by token
  • Roughly 1.4 seconds for routing
  • Built for writing and reasoning

Koa Decisions

  • Reads the whole request at once
  • Roughly 26 milliseconds for routing
  • Returns a choice, yes/no or score
A decision model returns a short answer far faster than a general-purpose LLM

Salesforce is releasing a new version of Koa Decisions, an AI decision model built on its HyperClassifier model and trained on Google’s open-weight Gemma. It takes a request and a list of categories you define, and returns a choice, a yes or no, or a score within milliseconds. The latest version adds multimodal support for images and audio.

The article argues that a general-purpose LLM is slower and more costly for quick, targeted decisions such as routing a request to a team. A decision model reads the whole request at once and can return an answer in as little as one token, while an LLM generates text token by token. Salesforce reports that, when routing requests in Agentforce, the decision model took roughly 26 milliseconds against roughly 1.4 seconds for a general-purpose LLM.

Salesforce plans to extend the model beyond routing into flows, Apex and Prompt Templates, with an API for developers. The company frames it as a way to keep larger models for reasoning and generation while smaller models handle routine decisions.

Read the original at Salesforce Ben (opens in a new tab)

Worth reading?

Koa Decisions brings fast single-step decision models to Salesforce builders (opens in a new tab)

You will learn what a decision model is, how it differs from a general-purpose LLM for choosing among options, and how Koa Decisions is positioned to reach flows, Apex and Prompt Templates.

Why read this

The article also points to a free Trailhead demo for testing it.

Decision model

  • Reads the whole request at once
  • Returns one choice in one step
  • About 26 ms in Agentforce routing

General-purpose LLM

  • Writes its answer token by token
  • Generates text before the choice
  • About 1.4 s in Agentforce routing
A decision model returns one choice directly, while an LLM writes its answer token by token

Salesforce has introduced Koa Decisions, the next version of its HyperClassifier model. It reads a request and a list of categories the user defines, then returns a decision in a single step, in milliseconds. It was built by applying training techniques to Google’s open-weight Gemma model, and it adds multimodal support for images and audio. Salesforce plans to extend it beyond Agentforce routing into flows, Apex and Prompt Templates, plus an API for developers. It joins a separate reasoning model built with NVIDIA in the Koa family.

The article argues that much of an agent’s work is choosing an option, a yes or no, or a score rather than generating text. A decision model returns that choice directly, while an LLM writes its answer one token at a time. In Agentforce routing measurements, the decision model took roughly 26 milliseconds against about 1.4 seconds for a general-purpose LLM. New categories can be added without retraining, which the article contrasts with building a custom predictive model over months.

Readers can test the model through the Koa Decisions demo on a free Trailhead account by changing the options and request text. The article also notes pricing and availability are subject to change and vary by region.

Read the original at Salesforce News (opens in a new tab)

Worth reading?