Azure Data Factory meets VaultSpeed FMC (Release 4.2.2)
January 6th, 2021
Some of our developers just don’t know how to stop. They came up with something new during Christmas Holidays: Support for Azure Data Factory (ADF) in our Flow Management Control (FMC) solution.
ADF FMC
In a previous blogpost we introduced our FMC solution on top of Apache Airflow. From now on, we also offer our workflow solution on top of Azure Data Factory.
This solution fits ideally for Azure DB or Synapse customers. They can use VaultSpeed to generate DDL and ELT to integrate their sources into the data warehouse. VaultSpeed can now also generate the orchestration in the form of json, which you can automatically deploy to ADF.
The VaultSpeed FMC for ADF uses Azure PAAS components exclusively. Azure Data Factory and your Data warehouse Database (SQL server or Synapse).
The database contains procedures and load metadata tables, meanwhile
ADF FMC will use stored procedure activities to execute those procedures.
ADF has visual presentation of the workflows and built-in monitoring. ADF provides seamless pipeline restart-ability and failure management. You can also create Azure Dashboards based on ADF metrics. It is also possible to export the metrics into an external reporting tool like Power BI, Grafana or other candidates.
ADF FMC allows you to optimize parallelism and Azure cloud costs. Code generation is fully metadata driven while it still allows for integration with existing ADF pipelines like pre-staging or post processing.
Other changes
Despite this being a smaller release, some other quality of life changes are included. Every page in VaultSpeed now contains a link to the relevant VaultSpeed docs (book icon). We also added all the subscription info to the dashboard, such as extra modules and support tiers.
