February 20, 2013 at 8:54 am
I was trying to dump data from a Temp table to 4 different tables (Computers, ComputerInfo, Vulnerabilities, and AuditInfo) in C# with different methods I created in classes.
if (_NetBIOSName != string.Empty)
{
//Only add the computers that are not existed in Computers table
AddCompSucc = cc.AddComputer(_NetBIOSName, _IP);
//add compputer's details
CompId = cc.GetComputer(_NetBIOSName).ID;
AddCompInfoSucc = cci.AddComputerInfo(CompId, _DNSName, _MAC, _OS, _NetBIOSDomain);
//add vulneribilities
AddIavSucc = ciav.AddIAVulnerability(CompId, _IAV, _Name, _AuditDate, 0);
//add audit details
IavId = ciav.GetIAVulnerability(CompId, _IAV, _Name, _AuditDate).ID;
AddAuditInfoSucc = cai.AddAuditInfo(IavId, _AuditID, _AuditDate, _SevCode, _Risk, _PCILevel, _Exploit, _Context);
}
In each method to insert data to tables, I used “if not exists…” statement to prevent duplicated data to be inserted.
IF NOT EXISTS (
SELECT DISTINCT NetBIOSName
FROM Computers
WHERE NetBIOSName = @NetBIOSName)
BEGIN
INSERT
INTO [dbo].[Computers]([NetBIOSName], 127.0.0.1)
VALUES (@NetBIOSName,@IP)
END
Issue: it takes forever to insert data to 4 tables when the Temp database is big, like having 10000 rows.
Any suggestion on how to do this differently to cut down the processing time.
Very Respectfully,
February 20, 2013 at 5:37 pm
huy1002 (2/20/2013)
I was trying to dump data from a Temp table to 4 different tables (Computers, ComputerInfo, Vulnerabilities, and AuditInfo) in C# with different methods I created in classes.if (_NetBIOSName != string.Empty)
{
//Only add the computers that are not existed in Computers table
AddCompSucc = cc.AddComputer(_NetBIOSName, _IP);
//add compputer's details
CompId = cc.GetComputer(_NetBIOSName).ID;
AddCompInfoSucc = cci.AddComputerInfo(CompId, _DNSName, _MAC, _OS, _NetBIOSDomain);
//add vulneribilities
AddIavSucc = ciav.AddIAVulnerability(CompId, _IAV, _Name, _AuditDate, 0);
//add audit details
IavId = ciav.GetIAVulnerability(CompId, _IAV, _Name, _AuditDate).ID;
AddAuditInfoSucc = cai.AddAuditInfo(IavId, _AuditID, _AuditDate, _SevCode, _Risk, _PCILevel, _Exploit, _Context);
}
In each method to insert data to tables, I used “if not exists…” statement to prevent duplicated data to be inserted.
IF NOT EXISTS (
SELECT DISTINCT NetBIOSName
FROM Computers
WHERE NetBIOSName = @NetBIOSName)
BEGIN
INSERT
INTO [dbo].[Computers]([NetBIOSName], 127.0.0.1)
VALUES (@NetBIOSName,@IP)
END
Issue: it takes forever to insert data to 4 tables when the Temp database is big, like having 10000 rows.
Any suggestion on how to do this differently to cut down the processing time.
Very Respectfully,
Define a PK constraint - or at least a unique index on NetBIOSName column, get rid of the "IF NOT EXIST" block, handle the duplicate key exception.
Hope this helps.
_____________________________________
Pablo (Paul) Berzukov
Author of Understanding Database Administration available at Amazon and other bookstores.
Disclaimer: Advice is provided to the best of my knowledge but no implicit or explicit warranties are provided. Since the advisor explicitly encourages testing any and all suggestions on a test non-production environment advisor should not held liable or responsible for any actions taken based on the given advice.February 21, 2013 at 6:44 am
Agreed. Make sure you have a PK on the table. If it were me, I would make an integer field as the primary key and add a unique nonclustered index to the computer name field so SQL won't worry about sort order based on a string field.
One more tip - use an appropriate data type for the computer name before you index it. If you're storing computer names, you probably don't need an nvarchar(max) or anything like that. More companies have strict policies on computer names, so get yours and go 2 bytes beyond it for when they exceed their own limit.
February 21, 2013 at 8:33 am
thank you for tips. Very respectful.
February 21, 2013 at 1:05 pm
Why do 1 insert call per temp table row on a 10,000 row temp table?
just
INSERT
INTO [dbo].[Computers]([NetBIOSName], 127.0.0.1)
select netbiosname,ip from temptable
where temptable.netbiosname not in ( select netbiosname from computers))
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply