Data Vault preview & metadata export (Release 4.2.1)

VaultSpeed stepped up the development pace in 2020. We delivered more, and more substantial releases than ever before. Our 10th and final release of 2020 brings great new features.

We added a data vault preview in our source editor. This enables users to see what their source would look like in a data vault model even before adding it to a data vault release. Second, we introduce full metadata export. Users can configure what metadata they would like to export from our repository. Third, we added support for multi-active many to many links. Finally, we enabled automatic deployment to Git. All generated code can automatically get committed in your Git repository.

Data Vault Preview

Our source editor has a great new feature. Our new Data Vault preview will show you what your source will look like in a data vault model, based on the metadata and specific options you’ve set for your source. All you need to do is save the current build of your source version and hit the SDVV button on the top right corner of the screen. This feature will help developers and analysts in making the right design decisions. You can keep altering your source version up until the point that your data vault model looks perfect, and then lock it in before adding it to a data vault release.

Metadata Export

We made significant changes to our metadata export functionality. It has become a separate form of delivery besides generated DDL, data flows and workflows.

We have built the export framework around the concept export configurations. You can build your desired configuration based on specific needs (e.g. data lineage, input for a metadata management tool, testing frameworks,…). Select metadata from a list of around 45 metadata attributes. These attributes include source, raw vault and business vault metadata.

Once your export configuration is saved, you can run it against a certain project and your metadata will be delivered to you in a csv-file. Various other export formats are coming soon.

Multi-active many to many links

VaultSpeed already supported multi-active satellites for HUB-SAT configurations. We now also support it for many-to-many links. Just like before, it also comes with the option to add a subsequence attribute to the unique key.
Multi-active satellites are important to enable data modellers to set the right business keys across your data vault when having multiple active records for a certain business key.

A typical example of a multi-active setup is a source table with multiple active records, one for each different language code. In that case the language code will become your subsequence attribute.

Auto Deploy Git

Another important new feature is the possibility to auto-deploy code to your designated Git repository. Simply add your git connection in the agent’s config file and you’re good to go. It is possible to deploy to different branches, as long as you properly target them in the configuration file.

Users can still auto-deploy code directly to the target database, ELT repo or Airflow instance, but this typically happens on a DEV environment. Automatically adding your code to Git makes it easier for users to add a specific version of VaultSpeeds generated code to a release. After you have verified the code on your development environment, you can use your Git repo in combination with a continuous integration tool like Jenkins, Travis or GitLab to deploy code throughout your DTAP cycle.

Other important changes

• We enabled Auto-deploy to Spark SQL targets.
• We added support for remote CDC with a DB link on SQL server. However, linked servers are not yet supported by Azure Synapse, so this can not be used for now.
• Delta generations can now also be used to generate FMC code. For initial loads, the new flow will only include the mappings for new objects.

That’s it for this blogpost… more features are coming soon!