The independent daily for Salesforce admins, developers and architects

Curated by Abhinav (opens in a new tab)

Must-read

Advanced PySpark patterns for batch data transforms in Data 360 (opens in a new tab)

Readers will learn how to structure, test, and deploy PySpark Code Extension scripts for complex batch transformations in Data 360.

Why read this

The piece demonstrates three production-ready patterns for hierarchical rollups, JSON event parsing, and machine learning scoring.

  1. Author PySpark script locally using Python SDK
  2. Validate script locally via SF CLI command
  3. Deploy packaged extension to Data 360 org
  4. Execute managed Spark job on demand or schedule
The end-to-end workflow for developing custom batch transforms in Data 360

The article introduces Data 360 Code Extensions as a mechanism to run custom PySpark logic against Data Lake Objects and Data Model Objects within an isolated compute environment. It outlines the full development lifecycle, detailing how engineers use the Salesforce Command Line Interface to validate scripts locally against a sampled dataset before packaging and deploying them to the platform.

The first example implements an iterative loop to calculate total tree revenue across variable-depth account hierarchies by continuously joining frontier records until convergence. The second example provides a reusable helper to parse and explode JSON array columns containing batched session events into individual rows for downstream analytics.

The third example demonstrates how to bundle a serialized scikit-learn classifier and its categorical feature grid to score leads efficiently via a broadcast join. The post concludes by reviewing managed Spark constraints around memory and timeouts, while noting that the transformed outputs become queryable objects for Agentforce agents.

Read the original at Salesforce Developers Blog (opens in a new tab)

Worth reading?

Recommended

Deploy Python Code Extensions for complex Data 360 batch transformations (opens in a new tab)

You will learn how to structure, validate, and deploy Python scripts as managed Code Extensions within Data 360.

Why read this

The guide also covers connecting these scripts to batch data transforms and monitoring execution through the platform.

Northstar Outfitters faced inconsistent product data across commerce systems and needed a trusted catalog. Native visual transforms could not handle the required standardization, enrichment, and scoring logic, prompting the use of Data 360 Code Extension to run modular Python and PySpark scripts directly on Salesforce infrastructure.

The solution relies on isolated runtime environments where scripts read from and write to designated Data Lake Objects using explicit permissions defined in a configuration file. Developers validate logic locally against sampled data before packaging the script, dependencies, and tests into a deployable archive.

The guide details both Setup interface and Salesforce CLI workflows for uploading packages, selecting compute sizes, and linking the extension to a batch data transform. Execution, scheduling, and logging are handled by the platform, while operators can trigger runs and review history through REST API calls.

Read the original at Salesforce Developers Blog (opens in a new tab)

Worth reading?