SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Loading Data With Powershell


Loading Data With Powershell

Author
Message
NevJam
NevJam
Grasshopper
Grasshopper (14 reputation)Grasshopper (14 reputation)Grasshopper (14 reputation)Grasshopper (14 reputation)Grasshopper (14 reputation)Grasshopper (14 reputation)Grasshopper (14 reputation)Grasshopper (14 reputation)

Group: General Forum Members
Points: 14 Visits: 22
Great thanks that works now - I'm having trouble getting sqlcmd to work correctly so I've started looking at BCP.exe

I've managed to rewrite your script so that my wmi results are output to a .xls file
Does anyone know the BCP command to get a local xls file loaded into a remote sql table?

I can manage to get the command working if I copy the data out of the .xls and into a .txt:

db = TEST
table = LogMemory
server = lonwintel01

This bcp command works for .txt file - does anyone know the correct switches for a .xls file:

bcp TEST..LogMemory in "c:\ps\scripts\import.txt" -S lonwintel01 -T
cmille19
cmille19
Mr or Mrs. 500
Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)

Group: General Forum Members
Points: 537 Visits: 724
NevJam (1/14/2009)
Great thanks that works now - I'm having trouble getting sqlcmd to work correctly so I've started looking at BCP.exe

I've managed to rewrite your script so that my wmi results are output to a .xls file
Does anyone know the BCP command to get a local xls file loaded into a remote sql table?

I can manage to get the command working if I copy the data out of the .xls and into a .txt:

db = TEST
table = LogMemory
server = lonwintel01

This bcp command works for .txt file - does anyone know the correct switches for a .xls file:

bcp TEST..LogMemory in "c:\ps\scripts\import.txt" -S lonwintel01 -T




BCP does not work with Excel files. Keep in mind the T-SQL shown in the article for sqlcmd could have been executed with Query Analyzer, SSMS or osql instead of sqlcmd

BULK INSERT dbautility.dbo.DiskUsage FROM 'C:\Users\u00\bin\diskusage.csv'
WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = ''

You can use DTS or SSIS with Excel files.



cmille19
cmille19
Mr or Mrs. 500
Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)

Group: General Forum Members
Points: 537 Visits: 724
Blogged about Executing Powershell in SQL Server . Using the technique described in the blog post you have a fourth method to load Powershell data into SQL Server. The approach uses xp_cmdshell and XQUERY to return the result set.



cmille19
cmille19
Mr or Mrs. 500
Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)

Group: General Forum Members
Points: 537 Visits: 724
Created a Powershell script to import Excel 2007 or 2003 to a SQL table, Import-ExcelToSQL



Prakash Heda
Prakash Heda
Mr or Mrs. 500
Mr or Mrs. 500 (590 reputation)Mr or Mrs. 500 (590 reputation)Mr or Mrs. 500 (590 reputation)Mr or Mrs. 500 (590 reputation)Mr or Mrs. 500 (590 reputation)Mr or Mrs. 500 (590 reputation)Mr or Mrs. 500 (590 reputation)Mr or Mrs. 500 (590 reputation)

Group: General Forum Members
Points: 590 Visits: 416
cmille19 (1/8/2009)
There are some extra returns in the code in the article. Try this instead (notice the line breaks are different):

param ( [string]$ComputerName = "." )

Get-WmiObject -computername "$ComputerName" Win32_LogicalDisk -filter "DriveType=3" |
foreach { add-member -in $_ -membertype noteproperty UsageDT $((Get-Date).ToString("yyyy-MM-dd"))
add-member -in $_ -membertype noteproperty SizeGB $([math]::round(($_.Size/1GB),2))
add-member -in $_ -membertype noteproperty FreeGB $([math]::round(($_.FreeSpace/1GB),2))
add-member -in $_ -membertype noteproperty PercentFree $([math]::round((([float]$_.FreeSpace/[float]$_.Size) * 100),2)) -passThru } |
Select UsageDT, SystemName, DeviceID, VolumeName, SizeGB, FreeGB, PercentFree



Thanks...though its so silly this article is still not updated, to get a 5 start rating known issue should have been fixed and a full set of files should have been attached for anyone to try quickly...

So i am able to export, lets see how does import do...

Prakash

Prakash Heda
Lead DBA Team - www.sqlfeatures.com
Video sessions on Performance Tuning and SQL 2012 HA
jshurak
jshurak
SSC Veteran
SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)

Group: General Forum Members
Points: 207 Visits: 649
Is there anyway to append the csv file instead of overwriting its contents?
cmille19
cmille19
Mr or Mrs. 500
Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)Mr or Mrs. 500 (537 reputation)

Group: General Forum Members
Points: 537 Visits: 724
If you mean keep appending output to a CSV file, the export-csv cmdlet doesn't directly support this, however you could use the convertto-csv cmdlt and remove the header row as follows:


get-psdrive | export-csv ./psdrives.csv -NoTypeInformation
get-psdrive | ConvertTo-Csv -NoTypeInformation |
foreach {$start=$true} {if ($start) {$start=$false} else {$_}} |
Out-File .\psdrives.csv -Append -Encoding ASCII



The code snippet above is based on a more complete solution described by Dmitry Sotnikov:
[url=http://dmitrysotnikov.wordpress.com/2010/01/19/export-csv-append/][/url]



jshurak
jshurak
SSC Veteran
SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)SSC Veteran (207 reputation)

Group: General Forum Members
Points: 207 Visits: 649
thanks for the reply. I'll take a look at that broader solution for that snippet. I actually got it working by saving my outgoing data into a variable, importing the contents of the csv file into another variable and appending the two together. My powershell scripting is quite novice so if you see any issues with this let me know Smile

Here is the code I used:

 
$Temp = Get-SqlData $srcServer 'master' $qry
$Last = import-csv "C:\filename.csv"
$out = $Temp + $Last | export-csv -noTypeInformation -path "C:\filename.csv"


jerry-jacob
jerry-jacob
SSC Rookie
SSC Rookie (36 reputation)SSC Rookie (36 reputation)SSC Rookie (36 reputation)SSC Rookie (36 reputation)SSC Rookie (36 reputation)SSC Rookie (36 reputation)SSC Rookie (36 reputation)SSC Rookie (36 reputation)

Group: General Forum Members
Points: 36 Visits: 177
@SSC Veteran : Thanks . Its working now by using the above commands
Jeff Moden
Jeff Moden
SSC Guru
SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)

Group: General Forum Members
Points: 88948 Visits: 41136
jcrawf02 (12/16/2008)
corey lawson (12/15/2008)
That's a good article, except for a couple of things.

Why do people still literally use commas as field separators (unless they have to, because the tool/function/utility doesn't support anything else)? Gaaaahhh! The example even uses the field separator parameter...

And, well, it's PowerShell. I'd rather do this stuff in INTERCAL, but to each their own...

Of course, BCP does this stuff as well...
In search of enlightenment, what should we be using as field separators?


How about it, Corey? What do you people should be using?

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
If you think its expensive to hire a professional to do the job, wait until you hire an amateur. -- Red Adair

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search