June 15, 2017 at 1:57 pm
I am looking for feedback back on staging data. I have two options,
1) Creating a staging database for each source system, we have approximately 10-12 sources
2) Create one staging database and within that one database create a schema for each source system, there is over 100 tables
I am not concerned with backups because the staging is just temporary anyhow, it is updated each day. The database is placed there for downstream processes to access it so the OLTP system is not touched.
I am looking for pro's and con's on both options, thank you in advance for your feedback!
June 15, 2017 at 2:23 pm
What are the downstream sources? Are there multiple downstreams that would be treating the staging data as a copy of the source data? Is this staging for a data warehouse? Are these intermediate staging areas where you'll be doing some ETL before loading them depending on the downstreams?
June 15, 2017 at 2:34 pm
The staging area would be used for the data warehouse and other processes as needed, at this time most of the ETL is handled in the SSIS packages pulling from the staged data.
It makes me nervous to put all of the incoming data into one single staging database even if they are in different schemas. There is no telling how many tables over time we could end up with in one single database.
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply