An easy way to synchronize data

  • We are about to begin User Acceptance Testing. The business users want to use a copy of "live" production data in the test environment. Basically, they want to compare their test reports to the reports they currently use in the production system. To support them, I would like to have the data from the production back end SQL Server 2008 R2 database copied over to the test environment. These will only be incremental things, since syncing will happen daily.

    What is an easy way of doing this? I would like to avoid using replication if possible since this is only a temporary situation. Are there any recommended third party tools?

  • Do you need to retain changes made on the test system? Or just have a copy of live data to work on?

  • I just need a copy of the live data in the test environment. I hope to get a fresh set of data for the test environment once a day.

  • Take full backup once.

    Daily take differential backups and restore them to test after a full backup restore.

    Turn "instant file initialization" and backup compression on to speed-up the process.

    _____________________________________________________
    Microsoft Certified Master: SQL Server 2008
    XDetails Addin - for SQL Developers
    blog.sqlxdetails.com - Transaction log myths
  • imani_technology (1/25/2013)


    What is an easy way of doing this? I would like to avoid using replication if possible since this is only a temporary situation. Are there any recommended third party tools?

    you can consider database mirroring too , easy to handle

    -------Bhuvnesh----------
    I work only to learn Sql Server...though my company pays me for getting their stuff done;-)

  • Taking a backup of live and restoring to the test system will be the easiest way to do this.

    Cheers

  • Is there an automated way to backup and restore? Also, the test environment is on a totally different server from the production environment.

  • From what I understand about database mirroring, you cannot use the mirrored database directly. Is that correct?

  • That is correct, you cannot use the mirrored database *directly*.

    However, you can create a snapshot of your mirrored database and have the report pointed to that Snapshot database (different database name).

    Whenever you wish to have an updated snapshot (to reflect the updates/changes in your production DB), you can just drop and recreate your snapshot.

    Hope that helps.

  • I'm considering mirroring a "staging" database that's in the production environment. Can I use SSIS to extract data from the mirrored database directly or will I need to have the SSIS package access a snapshot?

  • You existing backup process should already be automated.

    Assuming this is the case: determine where those backups are going; determine whether the test server has access to the backup files; determine the frequency and naming convention of the backup files.

    Then you can create a SQL Agent job which will contain a RESTORE command referencing the backup file you want to restore to the test server.

  • Of course you can automate that.

    Make a sqlcmd script that

    1) finds last full backup name and diff backup name.

    2) restore them to test.

    1) To find a backup name you can by querying msdb.dbo.backupset table on the production server,

    or using dir command like this:

    !!dir *.bak /o-d /b

    or xp_cmdshell with the same command.

    2) Once you have filenames, restore is almost trivial:

    :CONNECT testserver\instancename

    alter database XY set single_user with rollback after 2

    GO

    RESTORE DATABASE XY FROM DISK='$(bak_filename)'

    WITH

    MOVE('logicaldatafilename' TO 'new data path and file name'),

    MOVE('logicallogfilename' TO 'new log path and file name'),

    REPLACE,

    NORECOVERY

    GO

    RESTORE DATABASE XY FROM DISK='$(dif_filename)'

    WITH RECOVERY

    GO

    Verify that script work fine in sqlcmd mode in SQL Management Studio (Query->SQLCMD Mode).

    After that, create a sql agent job with step type "CmdExec" and put sqlcmd.exe command to execute the script you just created.

    Of course, you have to take care about the rights of sql agent account (or proxy account if you set one for that step) to be able to connect/read/write where it needs.

    If you need to copy files, use "!!ROBOCOPY sourcedir destinationdir filename1 filename2 filename3 /MT:2"

    _____________________________________________________
    Microsoft Certified Master: SQL Server 2008
    XDetails Addin - for SQL Developers
    blog.sqlxdetails.com - Transaction log myths
  • So which is the better approach, scripted restore or mirrored database? Which has better performance, when is easier to maintain?

  • Mirror affects primary (production) server in many aspects. One of them is - your production transaction log will grow if restoring stream to mirror slows down. You do not want to pair a production with a test in such tight way as mirror is, jeopardizing availability. Mirror should be specially monitored. Log shipping would be better. But simple, automated restore will do.

    _____________________________________________________
    Microsoft Certified Master: SQL Server 2008
    XDetails Addin - for SQL Developers
    blog.sqlxdetails.com - Transaction log myths
  • Here's a problem. The prod server backs up the file to a location that the test server can't access. I don't have authority to change that location.

Viewing 15 posts - 1 through 15 (of 24 total)

You must be logged in to reply to this topic. Login to reply