December 10, 2018 at 7:48 am
Hello
I work for a non-profit organization. Our organization has a lot of different data sources / databases. In order to make all this data excisable for users we want to build a central DWH where the data is uniformed.
Our databases are all local and not in the cloud. Currently because of the different databases it is impossible for users to easily combine the data.
On advise, we have choses for a variant of the Kimball method. This is, we extract all the data from the different individual data sources to a central ‘data-store’, this is build with the use of SQL server. The ‘data-store’ also maintains the history of all the data from the individual databases. This means if a record is added or changed the old line will be closed with an end date and a new line will be added with the new data. This way we can track our historic data.
The idea is to transfer all this data to a central DWH where the data is combined through business rules to uniform data. That make it accessible for users.
The question is do you know if there is tooling for automating the writhing of the coding to fill the DWH on given metadata?
Because we are a non profit organization we cannot buy really expensive tooling, due to our limited budget we have to be smart.
Can anybody point us into the right direction?
Thanks a lot!
December 17, 2018 at 8:24 pm
I'd recommend looking into Biml. There is a great Stairway To Biml series on this website to help you get started.
Tim Mitchell, Microsoft Data Platform MVP
Data Warehouse and ETL Consultant
TimMitchell.net | @Tim_Mitchell | Tyleris.com
ETL Best Practices
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply