May 17, 2017 at 2:31 am
Ok so my understanding is that all Azure SQL databases are full recovery and cannot be changed.
I am using one for developing something and I am trying to move some data to an empty database. The problem is because of how it is running I got an error saying the transaction log for database... is full due to 'active_transaction'
Usually in a simple model I would batch load the data with a transaction and checkpoint and not get his issue.
I'm going to try some things now, but whilst doing so does anyone know of a sure way of dealing with this on Azure SQL Db? loading 300k at a time for example and re-using the transaction log space?
May 17, 2017 at 8:51 am
What service tier are you using and what region are you in? How are you loading the data? There used to be a limit of 2gb on the transaction log file in azure but this was removed a few years ago - in the North America region at least. Now there is no official limit but transactions which cause excessive growth will be rolled back. it is not know what counts as excessive.
The transaction log is backed up every 5 to 10 minutes in azure.
Depending on the number of tables you are loading into and the number of records you may need to break your load down into separate transactions.
May 17, 2017 at 9:22 am
Hi, It was in PRS4 i think, north american server,
I was loading from a insert into select from external table.
I was loading 5 tables with about 30-60 million records in each.
They all had full text indexes on them also.
May 17, 2017 at 11:06 am
For the load process, bump it up to a premium service tier so you have more memory and faster processing, then bump it back down when the load is done.
I agree with @FridayNightGiant, break up the transactions. For large loads that's the way to do it anyway.
Also, get rid of the indexes for now and replace them after the load. That'll speed up the transactions.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply