Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Loading Data With Powershell


Loading Data With Powershell

Author
Message
cmille19
cmille19
SSC Veteran
SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)

Group: General Forum Members
Points: 275 Visits: 724
Comments posted to this topic are about the item Loading Data With Powershell



corey lawson
corey lawson
SSC Veteran
SSC Veteran (204 reputation)SSC Veteran (204 reputation)SSC Veteran (204 reputation)SSC Veteran (204 reputation)SSC Veteran (204 reputation)SSC Veteran (204 reputation)SSC Veteran (204 reputation)SSC Veteran (204 reputation)

Group: General Forum Members
Points: 204 Visits: 544
That's a good article, except for a couple of things.

Why do people still literally use commas as field separators (unless they have to, because the tool/function/utility doesn't support anything else)? Gaaaahhh! The example even uses the field separator parameter...

And, well, it's PowerShell. I'd rather do this stuff in INTERCAL, but to each their own...

Of course, BCP does this stuff as well...
jcrawf02
jcrawf02
Ten Centuries
Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)

Group: General Forum Members
Points: 1424 Visits: 19324
corey lawson (12/15/2008)
That's a good article, except for a couple of things.

Why do people still literally use commas as field separators (unless they have to, because the tool/function/utility doesn't support anything else)? Gaaaahhh! The example even uses the field separator parameter...

And, well, it's PowerShell. I'd rather do this stuff in INTERCAL, but to each their own...

Of course, BCP does this stuff as well...
In search of enlightenment, what should we be using as field separators?

---------------------------------------------------------
How best to post your question
How to post performance problems
Tally Table:What it is and how it replaces a loop

"stewsterl 80804 (10/16/2009)I guess when you stop and try to understand the solution provided you not only learn, but save yourself some headaches when you need to make any slight changes."
Anipaul
Anipaul
SSCertifiable
SSCertifiable (6.3K reputation)SSCertifiable (6.3K reputation)SSCertifiable (6.3K reputation)SSCertifiable (6.3K reputation)SSCertifiable (6.3K reputation)SSCertifiable (6.3K reputation)SSCertifiable (6.3K reputation)SSCertifiable (6.3K reputation)

Group: General Forum Members
Points: 6275 Visits: 1407
Good Article ...



thermanson
thermanson
SSC Rookie
SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)SSC Rookie (45 reputation)

Group: General Forum Members
Points: 45 Visits: 287
I have found that TAB separated files seem to cause the least number of problems. Unfortunately TABs aren't always possible, if you have to work with a file created by a mainframe or a file that you receive by FTP then TABs probably aren't an option.

It is possible to Bulk Insert a file starting at the "nth" row without creating a format file. Simply use the firstrow option.

Bulk Insert #temp
From 'c:\data.txt'
With (firstrow = 2)


cmille19
cmille19
SSC Veteran
SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)

Group: General Forum Members
Points: 275 Visits: 724
thermanson (12/16/2008)
I have found that TAB seperated files seem to cause the least number of problems. Unfortunately TABs aren't always possible, if you have to work with a file created by a mainframe or a file that you receive by FTP then TABs probably aren't an option.

It is possible to Bulk Insert a file starting at the "nth" row with out creating a format file. Simply use the firstrow option.

Bulk Insert #temp
From 'c:\data.txt'
With (firstrow = 2)


Thanks for the tip about about firstrow. Unfortunatley the built in Powershell cmdlet, Export-CSV only supports a comma as a separator.



David Ly-381060
David Ly-381060
Grasshopper
Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)

Group: General Forum Members
Points: 10 Visits: 464
when i ran:

./diskusage.ps1 sydw1255|export-CSV -path ./diskusage.csv -noTypeInformat

I get this error:

An empty pipe element is not permitted.
At F:\Working Folders\PowerShell\diskusage.ps1:4 char:2
cmille19
cmille19
SSC Veteran
SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)

Group: General Forum Members
Points: 275 Visits: 724
David Ly (12/16/2008)
when i ran:

./diskusage.ps1 sydw1255|export-CSV -path ./diskusage.csv -noTypeInformat

I get this error:

An empty pipe element is not permitted.
At F:\Working Folders\PowerShell\diskusage.ps1:4 char:2


David,

Try executing just ./diskusage.ps1 by itself and see if you get any output then add space pipe space export-csv -path ./diskusage.csv -noTypeInformat as shown in the article.

Verify your execution policy in Powershell also by running get-executionpolicy. Set your policy to remotesigned by running set-executionpolicy remotesigned if it is not already.



NevJam
NevJam
Grasshopper
Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)Grasshopper (10 reputation)

Group: General Forum Members
Points: 10 Visits: 22
David Ly (12/16/2008)
when i ran:

./diskusage.ps1 sydw1255|export-CSV -path ./diskusage.csv -noTypeInformat

I get this error:

An empty pipe element is not permitted.
At F:\Working Folders\PowerShell\diskusage.ps1:4 char:2


I get the same error - I believe it's due to the the following pipe (bold)?

Get-WmiObject -computername "$ComputerName" Win32_LogicalDisk -filter "DriveType=3"
|
foreach { add-member -in $_ -membertype noteproperty UsageDT $((Get-Date).ToString("yyyy-MM-dd"))

I can't use Get-DiskUsage.ps1 if I run it on it's own. Any help appreciated. I would really like to get this working. I'm hoping to use it for all manner of wmi queries so I can store them in a db. However I haven't managed to get any of the examples listed working at all.

Cheers
cmille19
cmille19
SSC Veteran
SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)SSC Veteran (275 reputation)

Group: General Forum Members
Points: 275 Visits: 724
There are some extra returns in the code in the article. Try this instead (notice the line breaks are different):

param ( [string]$ComputerName = "." )

Get-WmiObject -computername "$ComputerName" Win32_LogicalDisk -filter "DriveType=3" |
foreach { add-member -in $_ -membertype noteproperty UsageDT $((Get-Date).ToString("yyyy-MM-dd"))
add-member -in $_ -membertype noteproperty SizeGB $([math]::round(($_.Size/1GB),2))
add-member -in $_ -membertype noteproperty FreeGB $([math]::round(($_.FreeSpace/1GB),2))
add-member -in $_ -membertype noteproperty PercentFree $([math]::round((([float]$_.FreeSpace/[float]$_.Size) * 100),2)) -passThru } |
Select UsageDT, SystemName, DeviceID, VolumeName, SizeGB, FreeGB, PercentFree



Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search