SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


clear/reuse transaction log in Azure SQL db


clear/reuse transaction log in Azure SQL db

Author
Message
rob.lewis 86087
rob.lewis 86087
SSChasing Mays
SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)

Group: General Forum Members
Points: 600 Visits: 143
Ok so my understanding is that all Azure SQL databases are full recovery and cannot be changed.

I am using one for developing something and I am trying to move some data to an empty database. The problem is because of how it is running I got an error saying the transaction log for database... is full due to 'active_transaction'

Usually in a simple model I would batch load the data with a transaction and checkpoint and not get his issue.

I'm going to try some things now, but whilst doing so does anyone know of a sure way of dealing with this on Azure SQL Db? loading 300k at a time for example and re-using the transaction log space?
FridayNightGiant
FridayNightGiant
SSCertifiable
SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)SSCertifiable (6.2K reputation)

Group: General Forum Members
Points: 6242 Visits: 3416
What service tier are you using and what region are you in? How are you loading the data? There used to be a limit of 2gb on the transaction log file in azure but this was removed a few years ago - in the North America region at least. Now there is no official limit but transactions which cause excessive growth will be rolled back. it is not know what counts as excessive.
The transaction log is backed up every 5 to 10 minutes in azure.
Depending on the number of tables you are loading into and the number of records you may need to break your load down into separate transactions.
rob.lewis 86087
rob.lewis 86087
SSChasing Mays
SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)SSChasing Mays (600 reputation)

Group: General Forum Members
Points: 600 Visits: 143
Hi, It was in PRS4 i think, north american server,

I was loading from a insert into select from external table.

I was loading 5 tables with about 30-60 million records in each.

They all had full text indexes on them also.
Grant Fritchey
Grant Fritchey
SSC Guru
SSC Guru (354K reputation)SSC Guru (354K reputation)SSC Guru (354K reputation)SSC Guru (354K reputation)SSC Guru (354K reputation)SSC Guru (354K reputation)SSC Guru (354K reputation)SSC Guru (354K reputation)

Group: General Forum Members
Points: 354768 Visits: 34363
For the load process, bump it up to a premium service tier so you have more memory and faster processing, then bump it back down when the load is done.

I agree with @FridayNightGiant, break up the transactions. For large loads that's the way to do it anyway.

Also, get rid of the indexes for now and replace them after the load. That'll speed up the transactions.

----------------------------------------------------
The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood...
Theodore Roosevelt

The Scary DBA
Author of: SQL Server 2017 Query Performance Tuning, 5th Edition and SQL Server Execution Plans, 3rd Edition
Product Evangelist for Red Gate Software
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum








































































































































































SQLServerCentral


Search