Zero down time database migration into production

Question

Zero down time database migration into production

getalok_akg

Ten Centuries

Points: 1006
More actions
December 7, 2016 at 12:13 am

#332409

Hello All,
We are working into database migration project. There is requirement to migration sub-set of database into production without any down time .Also we have to migrate data and merge into existing tables in production.
Kindly guide me what strategy we should follow to acheive this migration .
Thanks
Alok

Viewing 6 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic. Login to reply

sgmunson SSC Guru Points: 110639 More actions · Answer 1

getalok_akg (12/7/2016)
Hello All,
We are working into database migration project. There is requirement to migration sub-set of database into production without any down time .Also we have to migrate data and merge into existing tables in production.
Kindly guide me what strategy we should follow to acheive this migration .
Thanks
Alok

That's not much to go on. You haven't provided ANY indication of what kind of data volume you're dealing with, nor any indication of what kind of environment this is beyond the labeling of it as "production", nor any information on where the data is coming from, and yet you want a strategy for dealing with it? Seriously? We may have expertise, but we're not genie's capable of magically understanding what you are up against. You'll need to get into a LOT more detail before anyone can give you much more than the equivalent of BE CAREFUL and TEST your migration process on a non-production system to validate that it works.

Steve (aka sgmunson) 🙂 🙂 🙂
Rent Servers for Income (picks and shovels strategy)

getalok_akg Ten Centuries Points: 1006 More actions · Answer 2

Thanks Steve for reply .

Allow me elaborate more on this we are migrating data from one database to another by using SSIS ETL package and doing all the Business login and transformation there into intermediate layer .

Here the key challenge is we have to move data from old legacy database into production and merge into existing production tables under different schema . And during transformation we are generating data from legacy system similier to target/production database schema .

Business do want to provide any down time in production during this data migration process.

We are looking into suitable strategy in this scenario that we can implement .

Data Volume is not that challenge we are not dealing with huge volume.

Thanks

Alok

sgmunson SSC Guru Points: 110639 More actions · Answer 3

getalok_akg (12/8/2016)
Thanks Steve for reply .
Allow me elaborate more on this we are migrating data from one database to another by using SSIS ETL package and doing all the Business login and transformation there into intermediate layer .
Here the key challenge is we have to move data from old legacy database into production and merge into existing production tables under different schema . And during transformation we are generating data from legacy system similier to target/production database schema .
Business do want to provide any down time in production during this data migration process.
We are looking into suitable strategy in this scenario that we can implement .
Data Volume is not that challenge we are not dealing with huge volume.
Thanks
Alok

Well, that doesn't really help very much. Even if it's not "a lot", different people define "a lot" rather differently, so again, we still have no idea what constraints exist in your environment, nor even the slightest clue as to how much data we're talking about. Without concrete details, anything anyone provides is, at best, guesswork. All we know so far is that you have some legacy data that needs to make it into production tables without any "down time". We don't even have any definition as to exactly what "down time" means. In some places, that phrase could even apply to degradation of production response times due to an unforeseen additional load, however slight. For others, down time is only when the entire application is unavailable. Given that the term "production" could refer to anything from just one application to an entire fleet of apps, that too starts to lose meaning. There are way too many variables here to be able to offer much in the way of detailed advice. Do you even have any ideas on how you plan to do this without down time? You'll need to provide a heck of a lot more detail before useful advice is likely to be forthcoming.

Steve (aka sgmunson) 🙂 🙂 🙂
Rent Servers for Income (picks and shovels strategy)

doug.brown SSCertifiable Points: 5607 More actions · Answer 4

I suspect the "zero down time" requirement is unrealistic. There has to be a time where all transactions on the old system are finalized and the data is frozen for migration. Then it has to be translated and moved to the new system, and validated there before you let the world in. At the extreme least there will need to be several minutes where the data isn't available, assuming you move the majority of the historical data in advance and then migrate the final day/hour at Go Live time. I've been involved in two migrations, and both had over four hours of downtime (and we were delighted to get it down to that). Whatever process you decide on, practice multiple run-throughs before go-live day.

Eric M Russell SSC Guru Points: 125523 More actions · Answer 5

On a medium powered OLTP database server, I would consider 100,000 rows or less to be a small amount of data. However, to understand the potential for blocking and duration of the process, tell us how many concurrent users, the type usage, and also describe the ETL step that loads the destination database: bulk insert, batch insert, merge, slowly changing dimention, etc. ?

"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho