Release Notes for 1.25 Release - July 2026

Calibo Accelerate now supports masking and scrambling of Personally Identifiable Information (PII) during data ingestion for the following integration workflows:

• Databricks data integration with Amazon S3, Azure Data Lake Storage (ADLS), and Snowflake data lakes.

• Snowpark data integration with Amazon S3 and Snowflake data lake tables.

Masking hides or partially masks PII while retaining sufficient information for business use.

Scrambling randomizes PII values while preserving the original data format. These capabilities help protect sensitive data and support secure data processing across supported data integration workflows.

The Snowflake DQ Processor stage consolidates the Data Analyzer, Data Validator, and Issue Resolver stages into a single processing stage. The DQ Processor node in DPS now supports automatic generation of data quality rules, making it easier to create and support data quality checks. The DQ Processor provides the following rule creation options to support users with different levels of expertise:

• Automatic rule generation – Automatically generates data quality rules based on the selected dataset, ideal for users with limited knowledge of data quality rules but a clear understanding of the desired outcome.

• UI-based rule creation – Enables users with a working knowledge of data quality rules to create analyzer and validator rules by selecting predefined rules, columns, or reference tables.

• Code Editor – Allows expert users to write data quality rules from scratch for advanced and customized use cases.

These enhancements significantly reduce the manual effort required for data quality rule generation. Regardless of their level of expertise, users can implement robust data quality checks efficiently.

The Data Assets feature provides a centralized workspace for creating, managing, validating, approving, and publishing reusable data assets across the platform. It enables teams to organize datasets and govern the lifecycle of data assets before they are consumed by downstream processes.

This streamlines data asset management, improves collaboration between data producers and consumers, and provides a governed foundation for building analytics, machine learning, and AI workflows.

Calibo Accelerate introduces a comprehensive set of enhancements to Databricks-based Retrieval-Augmented Generation (RAG) pipelines:

• Improved ingestion and indexing reliability with enhanced handling of Vector Search provisioning, large and empty files, hashing failures, and long-running operations, resulting in more robust pipeline executions.

• Automatic rollback for failed RAG pipeline executions, enabling the platform to restore previously deployed Databricks and RAG assets, remove resources created during failed deployments, restore serving endpoint configurations where applicable, and log rollback activities independently for easier troubleshooting.

• Optimized vector index management that intelligently reuses existing vector indexes, synchronizes only modified documents, and skips unnecessary indexing when no data changes are detected, reducing deployment time, compute overhead, and operational costs.

• Automated cleanup of RAG resources when a RAG Builder technology, feature, or product is deleted. Associated Databricks resources, including Serving Endpoints, LLM Registry entries, and Vector Search indexes, are automatically removed. If cleanup is only partially successful, failed resources are reported and can be retried.

• Advanced serving configuration options for RAG deployments, allowing users to configure compute resources and scaling parameters to optimize endpoint performance, availability, and infrastructure costs.

• Pre-deployment testing of RAG endpoints, enabling users to validate responses, verify retrieved knowledge sources, and confirm retrieval accuracy before promoting endpoints to production.

These enhancements improve the overall reliability, scalability, performance, and manageability of Databricks-based RAG deployments while reducing operational effort and infrastructure costs.

Calibo Accelerate enhances RAG pipelines with context-aware retrieval to deliver more accurate and conversational responses. The platform now supports agentic retriever workflows with short-term conversational memory, enabling responses to leverage recent interactions before querying the vector database. When relevant context is not available in memory, the platform automatically retrieves information from the vector database to ensure reliable and contextually relevant responses.

This enhancement improves retrieval accuracy, maintains conversational continuity, and delivers a more reliable enterprise-grade RAG experience.

Calibo Accelerate enhances RAG pipelines with expanded embedding model support to improve retrieval quality and pipeline efficiency. The platform now supports all compatible Databricks embedding models through automatic embedding dimension validation and includes support for the Qwen-3 English embedding model with multiple embedding dimensions.

This enhancement improves retrieval accuracy, increases embedding flexibility, and enables more scalable and efficient RAG pipeline execution.