Release 5.7.3 | Enhanced DataOps & GIT integration, Databricks DLT support

VaultSpeed Agent: Key updates to improve CI/CD for the Data Product SDLC

The VaultSpeed Agent is essential for gathering metadata and executing deployments within VaultSpeed's architecture. This release introduces significant improvements to the agent, enhancing the CI/CD pipelines. These enhancements considerably reduce the workload for Data Engineers and DataOps teams, ensuring a consistent and reliable flow of data products to the business.

1. Support for Multiple Custom Deployment Scripts

Previously, custom deployment scripts were limited to a single static script. Now, multiple deployment scripts can be defined in the connections.properties file.

Key benefits:

• Users can dynamically choose scripts on the UI during the deployment process.
• Greater flexibility over deployment execution.
• The old AGENT_CUSTOM_DEPLOY_SCRIPT system parameter is now deprecated.

Please refer to the documentation on configuring the custom deployment scripts for detailed setup instructions.

2. Dynamic branch configuration for GIT deployments

GIT deployment just got more flexible. Users can now specify target branches dynamically from the UI.

What’s new?

• No more hardcoded branch names in agent configs.
• If no branch is specified, the currently checked-out branch is used.
• Improved error handling: clear notifications, e.g., if a branch doesn’t exist.

One of the big benefits is the ability to control the deployment better from within VaultSpeed, making it easier to run GIT compares and code diffs against the entire generated code base.

For details, see the updated GIT Deployment documentation.

3. Enhanced Agent Overview & Version Management

Monitoring and managing VaultSpeed agents is now more transparent than ever with new data points added to the Agent Overview page.

Newly added fields:

• Last Response Time – Shows when the agent last responded.
• Host Name – Identifies where the agent is running.
• Software Version – Displays the agent’s current version.
• Process Identifier (PID) – Helps with debugging.
• Remote Kill Action – Allows users to terminate agent processes remotely.

Improved version management:

• Agents running older versions will now get grace period warnings.
• After the grace period, typically one month following the release, outdated agents will be marked in red and will no longer be able to connect to VaultSpeed. This will help prevent accidental errors caused by version mismatches.

Materialized Views in VaultSpeed Studio for Databricks Delta Live Tables (DLT)

VaultSpeed now supports Materialized Views as a persistent target object in Databricks SQL. This new feature enables teams to accelerate the delivery of data products in the Gold layer by utilizing Delta Live Tables alongside VaultSpeed's automation capabilities.

What this means for you:

• Materialized views are an effective option for optimizing query performance.
• Effectively utilizing the DLT framework in a metadata-driven approach for automated data product delivery.
• Incremental & full refresh schedules supported.
• Partitioning & Scheduling via SCHEDULE and TBLPROPERTIES.

For full setup details, check out the Materialized Views in VaultSpeed Studio: Setup for DLT documentation.

What’s Next?

As always, our focus remains on seamless integration, reduced manual effort, and full-scale automation of your Data Vault pipelines.

Got questions or feedback? Join the conversation in the VaultSpeed Community or check out our latest documentation for setup and best practices.