The independent daily for Salesforce admins, developers and architects

Curated by Abhinav (opens in a new tab)

Set up Data Cloud web crawlers to ingest and vectorize external pages for AI agents (opens in a new tab)

Readers will learn how to configure a Data Cloud Web Content Connector, filter target elements with CSS selectors, and generate searchable vector chunks from live websites.

Why read this

The guide also shows how to verify the ingested data using Data Explorer and SOQL.

The article introduces Retrieval-Augmented Generation and explains why keeping AI agents grounded in real-time data matters. It outlines the prerequisites of enabling Einstein, Agentforce, and Data Cloud in a development org.

It details the configuration of a Web Content Connector in Data Cloud Setup. The steps explain how to set crawl depth to zero to avoid following irrelevant links and how to use browser developer tools to copy CSS selectors for targeting specific page elements.

The guide describes creating a Data Lake Object and its corresponding Data Model Object. It notes that semantic search must be enabled while skipping image embedding and knowledge harmonization for this text-only use case. It concludes by showing how to validate the chunked output through Data Explorer and standard SOQL queries against the unstructured data model object suffix.

Read the original at Salesforce Break (opens in a new tab)

Worth reading?

Recommended

Use custom Python chunking to improve Agentforce retrieval accuracy in Data 360 (opens in a new tab)

You will learn how to deploy Python-based Code Extension functions to override default chunking in Data 360 search indexes.

Why read this

The guide demonstrates how structure-aware splitting preserves table headers and speaker labels, resulting in more accurate Agentforce responses.

The article argues that poor chunking, not the language model, is often the real cause when an Agentforce agent misses details from a PDF or misreads a conversation, because retrieved snippets lose their context before the model sees them. Native Data 360 chunking (section-aware and semantic passage extraction) works well for standard content, but it struggles with complex multipage PDFs, dense tables and multispeaker transcripts when character or token-based splitters are used. Code Extension functions let developers deploy sandboxed, stateless Python chunking logic inside the Salesforce trust boundary, authored with the Data Custom Code SDK, deployed via the Salesforce CLI or UI, and wired into search index creation. The authors caution that their regex and pattern-based examples are illustrative happy-path code, not production-grade, and can produce false positives and negatives.

The piece tests three scenarios, comparing native chunks with custom ones. In a quarterly financial report, native splitting left numbers stranded without their column headers, making a growth-rate question unanswerable; a structure-aware function that detects table boundaries and repeats headers in sub-chunks lets the agent give a precise segment comparison. In a recorded escalation call with five speakers, native chunking produced unattributed walls of text, while a sliding-window function over speaker turns (window of 6 turns, stride of 3, capped at 1,500 characters) kept questions with their answers and preserved speaker names and timestamps, so the agent could identify a contractual downtime commitment. In a regulated-sector document, native chunking exposed SSNs, card numbers, emails, phone numbers, medical record numbers and other identifiers in the vector index; a regex pass before chunking masked the standard formats, though proprietary identifiers such as medical record, payer member, case and token numbers passed through until organization-specific patterns were added. The conclusion is that Code Extension is an escalation path for complex data rather than a replacement for native chunking, and it points developers to the documentation, the Code Extension setup in Data 360 and a video playlist to get started.

Read the original at Salesforce Developers Blog (opens in a new tab)

Worth reading?