Technical Article

Quick and dirty way to make a very large table

,

Here's a quick and dirty way to generate massive amounts of data. Basically the columns are in the form of: Identity(1,1), datetime, uniqueidentifier1...uniqueidentifierN

and the uniqueidentifier columns are set to populate automatically. I generated about 1 Gig (260000 rows of max rowsize about 8000) in 3.5 minutes on my simple basic desktop. I've limited it to 8000 so it would work with SQL 2000 but not sure how massive you could make it on 2005/2008. The reason for doing it this way is all the other ways that use checksum for example, take a couple hours to generate 1 Gig of Data.

I did notice when I had the columns be defaulted as part of the definition, the generation was quicker than inserting a newid() for all applicable columns.

One more thing, to speed up the generation, set the database this table is located in to grow in magnitude of 500MB chunks at a time rather than a percentage, as it will greatly speed up the generation.

if exists (select * from sysobjects where name like '%bigtable%')
  drop table bigtable
go
set nocount on
declare @ctr int,
        @ctr2 int,
        @cmd varchar(8000),
        @NEWLINE char,
        @maxcols int,
        @numrows int,
        @rowctr int

set @maxcols = 220
set @NEWLINE = char(13)
-- First create a basic table of rowid and entrydate
set @cmd = 'create table bigtable ' + @NEWLINE + '(rowid int identity(1,1), entrydate datetime)' + @NEWLINE
exec (@cmd)
set @ctr = 0
while @ctr < @maxcols
begin
-- The reason for this alter loop is because my @cmd is limited to 8000 characters
-- so having a massive create table overflowed (may not be an issue with declare @cmd(MAX)
-- SQL 2005.  Even if looped 220 times, this alter is trivial in terms of time
-- so no need to worry much about it.
    set @cmd = 'alter table bigtable add VAL' + cast(@ctr as varchar(4)) + ' uniqueidentifier default newid()' + @NEWLINE
    exec(@cmd)
    set @ctr = @ctr + 1
end

set @ctr = 0
set @rowctr = 1
-- 265000 should give approximately 1 Gig file
set @numrows = 265000
while @ctr < @numrows
begin
    set @cmd = 'insert bigtable (entrydate) values (getdate())'
    exec(@cmd)
    set @ctr = @ctr + 1
end

-- This part is optional if you want even more data generated
create nonclustered index IDX_bigtable on bigtable(rowid, entrydate)
go

Rate

4 (1)

You rated this post out of 5. Change rating

Share

Share

Rate

4 (1)

You rated this post out of 5. Change rating