This would be a primary load location (or a series of scripts deployed to the database that you've just built via Model) that would take care of all standard cleanup components.
Firstly let me say thank you for your time reading my long post. I really appreciate your opinion.
You are right and I really like your suggestion.
Also, you might think about giving your users DDLAdmin on the databases so they can deploy their scripts as procs so you can more easily keep track of what belongs to whom. My guess is this is the way it's easiest for business because they've never been shown an alternative.
They already have and they create many staging tables and do all sorts of inserts and deletions. I've seen this in a lot of scripts.
However, really, that's a big deal you're dealing with over there and it sounds like that's their primary source of revenue, at least for the department you're supporting. I'd tread lightly and first learn their process before you go and start making major alterations.
You're spot on once again. I had a meeting with the Director of Software development and she made a point about the data services team being afraid of change. This explains why the the Director of Data Services is so keen to know what it is that I'm doing! She's asked for two updates this week already!
So I have some decisions to make.....
a) Do I start with small changes like enhance existing functions/sprocs?
For example yesterday I rewrote two data cleansing functions. The original version took 27 seconds to run on a data set of 250k record. My new version took 5 seconds! But this is a small cog in a large data hygiene/merge machine!
b) Do as you suggest and start by creating a generic data cleanse script?
I think I bit off more than I could chew taking on this role! :doze:
---------------------------------------------------------
It takes a minimal capacity for rational thought to see that the corporate 'free press' is a structurally irrational and biased, and extremely violent, system of elite propaganda.
David Edwards - Media lens[/url]
Society has varying and conflicting interests; what is called objectivity is the disguise of one of these interests - that of neutrality. But neutrality is a fiction in an unneutral world. There are victims, there are executioners, and there are bystanders... and the 'objectivity' of the bystander calls for inaction while other heads fall.
Howard Zinn