Our latest update, Version 5.5, is now available! This release includes enhanced version management for the Data Vault and automatically generated change documentation for delta releases. Additionally, our Python SDK will set up users for yet another productivity boost, and we have added support for dbt on Databricks and improved our integration with Google BigQuery.
Improved version management and documentation, dbt on Databricks, improved BigQuery support, and Python SDK (Release 5.5).
September 24th, 2023
Better version management and documentation
Managing versions in the realm of data warehousing can be a challenging task. However, VaultSpeed offers two types of versioning to help simplify the process: metadata versioning and code versioning. Metadata versioning is used to keep track of the various metadata versions of source systems, data vaults, business vaults, and information layers, and how they are related to each other. For instance, it is vital to identify which sources are included in which data vault release. Code versioning automatically versions the generated code into your Git repository, allowing it to be properly delivered to the CI/CD pipeline. Two new features have been added to significantly enhance these capabilities.
Revert back to a previous release
Our version management options for data vault releases have been enhanced. Users can create a new release from any past data vault version after the latest production install. This feature enables users to easily revert to a previous data vault version in case of any modeling issues or if changes in business priorities make the current release irrelevant.
Auto-generated documentation for delta releases
Another exciting new feature of VaultSpeed is the ability to generate a markdown report detailing every change made in a delta release. This report allows you to pinpoint exactly what caused the code generation between two versions. The document can be versioned alongside the code artifacts in your git repository, acting as a helpful readme for that specific code increment.
Python SDK
We are excited to announce a major enhancement to our product - a brand new Python SDK. This development offers a simple and intuitive interface for users to interact with the VaultSpeed application programmatically. The SDK enables users to perform almost any action that can be executed via the UI.
With the SDK, you can build a wide range of integrations. For instance, you can now generate and deploy code in a single step, create orchestration workflows programmatically, and import or export metadata with ease.
Installation instructions, documentation and examples can be found in the documentation portal: https://docs.vaultspeed.com/space/VPD/3496575098/VaultSpeed+SDK
More information and examples using the SDK can be found on Github:
https://github.com/Vaultspeed/sdk
dbtTM for databricks
VaultSpeed offers support for using dbtTM as a target to deploy Data Vault transformation code. While we strongly believe that data automation should be metadata-driven, it's also important to build your data model correctly, and aligned with business definitions. Our platform allows users to easily view and enrich source and target model metadata using a user-friendly GUI. From there, the platform automatically generates all the necessary code based on that metadata, saving you time and effort.
Using dbt as a runtime platform, you can store, version, and run these generated modular SQL scripts. Additionally, dbt features such as key definitions, data lineage, and dependency alerts make it an ideal solution for a Data Vault context. We added support for dbt on Snowflake release 5.3, and in this release, we're excited to bring dbt support to Databricks.
Improved support for Google BigQuery
Our team has made improvements to the integration of Google BigQuery. We have included added features such as supporting stored procedures, automatic deployment, and the creation of schemas, also known as datasets. With these enhancements, by bringing the integration of Google BigQuery to a higher level, users can now automate their data runtime with VaultSpeed more efficiently.
Other changes
Release 5.5 comes with several additional features:
- The agent supports Java OpenJDK 11, kindly get in touch with your customer success manager for upgrade recommendations.
- You can now utilize the business names you included in your business views to develop customized automation templates in VaultSpeed Studio. This means, for example, that you can now have your business attribute names displayed instead of the raw source attribute names in a dimension object.
- The sidebar is always visible and can be toggled between icon-only or full-width with menu item names, providing a smoother navigation experience.
- We have included a new Project selector in the top navigation bar. This feature enables filtering of all screens based on the selected Project, making it easier to manage multiple projects with similar sources and Data Vaults.
- Furthermore, there are more advanced setup options available for CDC attributes regarding data type, format mask, and so on.
- Enterprise subscriptions can now enable Single Sign-On (SSO) via a support desk request. SAML and OIDC are currently supported for federation.
To view the complete list of features, please check out the change log.
