Write PowerShell Output to SQL Server Table

  • Heh... First time I've had to admit being a "newbie" in a long time so be gentle with me. I might not even know enough to ask the right questions.

    I managed to pick up the following PS script from the internet. Lot's of you know what it is. It gets some disk space information from every disk device for the given computer. I've removed all field formatting just to keep it super simple.

    Get-WmiObject Win32_LogicalDisk -computer 'SomeComputerName' | Select SystemName,DeviceID,VolumeName,Size,FreeSpace | Format-Table

    What I'd like to do is write the output to a table in a given SQL Server. I've Googled for answers and have found some (IMHO) very over complicated methods. Some are generic (which is very cool) but not what I need. I just want to write this one output to a table.

    The table for this example is simple as well (I've removed all but the necessary columns including a Date/Time column)...

    CREATE TABLE dbo.DiskSpace

    (

    SystemName VARCHAR(128),

    DeviceID CHAR(2),

    VolumeName VARCHAR(128),

    Size BIGINT,

    FreeSpace BIGINT

    )

    ;

    I realize that we're going to need a connection to the database. I'd like it to be a trusted connection so we don't have to hardcode a user name or password. Assume that the server name is "TestServer" and that the database name is "TestDB". From what I've been able to read up on, the business of the connection would look something like this (please correct it if I'm wrong)...

    $conn=new-object System.Data.SqlClient.SQLConnection

    $ConnectionString = "Server=TestServer;Database=TestDB;Integrated Security=True;Connect Timeout=0"

    $conn.ConnectionString=$ConnectionString

    $conn.Open()

    ...something goes here but I don't know what...

    $conn.Close()

    Like I said, I'm brand-spanking-new to PowerShell. I sure could use some help on how to get the data from the first script above into the table that I posted the CREATE TABLE code for without having to build a bunch of functions like in the following link.

    http://blogs.technet.com/b/heyscriptingguy/archive/2010/11/01/use-powershell-to-collect-server-data-and-write-to-sql.aspx

    Thanks for the help, folks. And, yeah... if you have a good book recommendation for the syntax and how to "start thinking in PowerShell" like I do in T-SQL, it would also be much appreciated.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • The simplest way is to build up a string to execute:

    $commandText = "INSERT DiskSpace VALUES ('" + $wmiObject.GetValue(0)["SystemName"] + "')"

    $command = $conn.CreateCommand()

    $command.CommandText = $commandText

    $command.ExecuteNonQuery()

    This way is not necessarily ideal as it is a little bit verbose, however, I would recommend building up the text as a separate string because you can always output it to the console before using it in anger:

    Write-Output $commandText

    Just say if you need more or if this is not what you were looking for.

    Gaz

    -- Stop your grinnin' and drop your linen...they're everywhere!!!

  • Jeff,

    I have the very script you're looking for that does exactly what you want.

    I pieced it together using a couple functions I found online and it works very well.

    -It pulls a list of servers from the text file

    -creates temporary data table

    -creates function to retrieve data

    -executes function which stores data in data table

    -imports data table into sql server table

    replace values in <> with your variables.

    #define servers to be monitored

    $server = get-content "<path>.txt"

    #data table to hold results

    Function out-DataTable

    {

    $dt = new-object Data.datatable

    $First = $true

    foreach ($item in $input){

    $DR = $DT.NewRow()

    $Item.PsObject.get_properties() | foreach {

    if ($first) {

    $Col = new-object Data.DataColumn

    $Col.ColumnName = $_.Name.ToString()

    $DT.Columns.Add($Col) }

    if ($_.value -eq $null) {

    $DR.Item($_.Name) = "[empty]"

    }

    elseif ($_.IsArray) {

    $DR.Item($_.Name) =[string]::Join($_.value ,";")

    }

    else {

    $DR.Item($_.Name) = $_.value

    }

    }

    $DT.Rows.Add($DR)

    $First = $false

    }

    return @(,($dt))

    }

    #function to retrieve disk information

    Function Get-DisksSpace ([string]$Servername, $unit= "GB")

    {

    $measure = "1$unit"

    Get-WmiObject -computername $serverName -query "

    select SystemName, Name, DriveType, FileSystem, FreeSpace, Capacity, Label

    from Win32_Volume

    where DriveType = 2 or DriveType = 3" `

    | select @{Label="SystemName";Expression={$serverName.ToUpper()}} `

    , Name `

    , @{Label="SizeIn$unit";Expression={"{0:n2}" -f($_.Capacity/$measure)}} `

    , @{Label="FreeIn$unit";Expression={"{0:n2}" -f($_.freespace/$measure)}} `

    , Label

    }

    #execute the functions

    foreach ($s in $server)

    {

    Get-DisksSpace $s

    $dataTable = Get-DisksSpace $s | where {$_.name -like "E:\*" -or $_.name -like "C:\*" -or $_.name -like "F:\*"} | out-DataTable

    $connectionString = "Data Source=<server\instance>; Integrated Security=True;Initial Catalog=<databaseName>;"

    $bulkCopy = new-object ("Data.SqlClient.SqlBulkCopy") $connectionString

    $bulkCopy.DestinationTableName = "<schema.table>"

    $bulkCopy.WriteToServer($dataTable)

    }

    As a side note, I use Win32_Volume instead of LogicalDisk because the volume class returns mount points whereas the logicaldisk class returns only drive letters.

    ______________________________________________________________________________________________
    Forum posting etiquette.[/url] Get your answers faster.

  • Gary Varga (6/17/2013)


    The simplest way is to build up a string to execute:

    $commandText = "INSERT DiskSpace VALUES ('" + $wmiObject.GetValue(0)["SystemName"] + "')"

    $command = $conn.CreateCommand()

    $command.CommandText = $commandText

    $command.ExecuteNonQuery()

    This way is not necessarily ideal as it is a little bit verbose, however, I would recommend building up the text as a separate string because you can always output it to the console before using it in anger:

    Write-Output $commandText

    Just say if you need more or if this is not what you were looking for.

    Thanks, Gary.

    So would I build the insert for the first two columns like this?

    $commandText = "INSERT DiskSpace (SystemName,DeviceID) VALUES ('" + $wmiObject.GetValue(0)["SystemName"] + "," + $wmiObject.GetValue(0)["DeviceID"]+ "')"

    Also, there seems to be a bit of hidden magic there. Will 1 insert do it or am I going to have to iterate over each row from the $wmiObject with incrementing values for the operand of .GetValue?

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Almost, unless I am mistaken you have missed single quotes either side of the comma i.e.

    $wmiObject.GetValue(0)["SystemName"] + "','" + $wmiObject.GetValue(0)["DeviceID"]+ "')"

    I am sure that you will have come across the need for quotes within quotes before. Lovely. πŸ˜‰

    Gaz

    -- Stop your grinnin' and drop your linen...they're everywhere!!!

  • calvo (6/17/2013)


    Jeff,

    I have the very script you're looking for that does exactly what you want.

    I pieced it together using a couple functions I found online and it works very well.

    -It pulls a list of servers from the text file

    -creates temporary data table

    -creates function to retrieve data

    -executes function which stores data in data table

    -imports data table into sql server table

    replace values in <> with your variables.

    #define servers to be monitored

    $server = get-content "<path>.txt"

    #data table to hold results

    Function out-DataTable

    {

    $dt = new-object Data.datatable

    $First = $true

    foreach ($item in $input){

    $DR = $DT.NewRow()

    $Item.PsObject.get_properties() | foreach {

    if ($first) {

    $Col = new-object Data.DataColumn

    $Col.ColumnName = $_.Name.ToString()

    $DT.Columns.Add($Col) }

    if ($_.value -eq $null) {

    $DR.Item($_.Name) = "[empty]"

    }

    elseif ($_.IsArray) {

    $DR.Item($_.Name) =[string]::Join($_.value ,";")

    }

    else {

    $DR.Item($_.Name) = $_.value

    }

    }

    $DT.Rows.Add($DR)

    $First = $false

    }

    return @(,($dt))

    }

    #function to retrieve disk information

    Function Get-DisksSpace ([string]$Servername, $unit= "GB")

    {

    $measure = "1$unit"

    Get-WmiObject -computername $serverName -query "

    select SystemName, Name, DriveType, FileSystem, FreeSpace, Capacity, Label

    from Win32_Volume

    where DriveType = 2 or DriveType = 3" `

    | select @{Label="SystemName";Expression={$serverName.ToUpper()}} `

    , Name `

    , @{Label="SizeIn$unit";Expression={"{0:n2}" -f($_.Capacity/$measure)}} `

    , @{Label="FreeIn$unit";Expression={"{0:n2}" -f($_.freespace/$measure)}} `

    , Label

    }

    #execute the functions

    foreach ($s in $server)

    {

    Get-DisksSpace $s

    $dataTable = Get-DisksSpace $s | where {$_.name -like "E:\*" -or $_.name -like "C:\*" -or $_.name -like "F:\*"} | out-DataTable

    $connectionString = "Data Source=<server\instance>; Integrated Security=True;Initial Catalog=<databaseName>;"

    $bulkCopy = new-object ("Data.SqlClient.SqlBulkCopy") $connectionString

    $bulkCopy.DestinationTableName = "<schema.table>"

    $bulkCopy.WriteToServer($dataTable)

    }

    As a side note, I use Win32_Volume instead of LogicalDisk because the volume class returns mount points whereas the logicaldisk class returns only drive letters.

    Thank you, Sir. I was hoping to avoid the out-datable (God bless the "Scripting Guy"!) function because I may have several columns in the table that won't be populated by PowerShell. I just might have to resort to that (can always use a staging table), though, because it seems like the $bulkCopy would probably write to the server more quickly than individual inserts (If that's what Gary's code ends up doing).

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Gary Varga (6/17/2013)


    Almost, unless I am mistaken you have missed single quotes either side of the comma i.e.

    $wmiObject.GetValue(0)["SystemName"] + "','" + $wmiObject.GetValue(0)["DeviceID"]+ "')"

    I am sure that you will have come across the need for quotes within quotes before. Lovely. πŸ˜‰

    Heh... thanks, Gary. Lovely nested quotes indeed. πŸ™‚

    The outstanding question is will the code you posted do all of the rows or am I going to have to build 1 insert row for each row found in the $wmiObject?

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • @gary,

    As a continuation of my previous question, what does the "0" in the parenthesis of .GETVALUE(0) do?

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • As for the number of executions you have two choices:

    1) Loop through adding multiple inserts to a single command.

    $isFirst = $true

    $commandText = "INSERT DiskSpace"

    $wmiObject = Get-WmiObject Win32_LogicalDisk -computer 'SomeComputerName'

    Foreach ($logicalDisk in $wmiObject)

    {

    if($isFirst) { $isFirst = $false } else { $commandText += "," }

    $commandText += " VALUES ('" + $logicalDisk["DeviceID"] + "')"

    }

    $command = $conn.CreateCommand()

    $command.CommandText = $commandText

    $command.ExecuteNonQuery()

    2) Loop through executing multiple commands.

    $wmiObject = Get-WmiObject Win32_LogicalDisk -computer 'SomeComputerName'

    Foreach ($logicalDisk in $wmiObject)

    {

    $commandText = "INSERT DiskSpace VALUES ('" + $logicalDisk["DeviceID"] + "')"

    $command = $conn.CreateCommand()

    $command.CommandText = $commandText

    $command.ExecuteNonQuery()

    }

    Gaz

    -- Stop your grinnin' and drop your linen...they're everywhere!!!

  • Ah yes, good spot:

    .GetValue(0) returns the first logical disk requested. Please see the looping above to see that you can ignore this call now.

    Gaz

    -- Stop your grinnin' and drop your linen...they're everywhere!!!

  • Thank you Gary and calvo... this is a great start for me. It's been a thousand years since I've gone anywhere near procedural code outside of SQL Server. Compound that with being a real newbie to PowerShell and hopefully you'll understand why wasn't even sure what I was looking for in Google. Now that I have a couple of examples that I actually understand, I might be able to make some serious headway until I get a book on the subject.

    Thanks again to both of you for help me out.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Jeff,

    A while back I experimented with TVPs and Powershell.

    Basically, you create a table on a SQL server, and create a stored proc with a TVP to handle the inserts.

    In Powershell, you create a datatable object and populate it. Then create a sqlcommand object of type stored proc, add a parm that will be a TVP, then set the value of the parm = datatable, and call the sproc.

    It's not exactly elegant, and you still have to populate the PoSh datatable RBAR.

    But you can pass many rows to SQL Server in a single call.

    I adapted the following from code found on the net. I wish I could remember where for credit.

    #Datatable object

    $Datatable = New-Object system.Data.DataTable

    $null = $Datatable.Columns.Add("Server", "System.String")

    $null = $Datatable.Columns.Add("DBName", "System.String")

    $null = $Datatable.Columns.Add("TableName", "System.String")

    etc ....

    $Data = <some result set you're going to insert into your $Datatable>

    if ($Data)

    {

    foreach ($Row in $Data)

    {

    $Info = $Row.ItemArray

    if ($Info)

    {

    $null = $Datatable.Rows.Add($Info)

    }

    }

    }

    #Connection and Query Info

    $serverName = "<MyServer>"

    $databaseName = "<MyDB>"

    $query = "<MyInsertSproc>"

    #Connect

    $connString = "Server=$serverName;Database=$databaseName;Integrated Security=SSPI;"

    $conn = new-object System.Data.SqlClient.SqlConnection $connString

    $conn.Open()

    #Create Sqlcommand type and params

    $cmd = new-object System.Data.SqlClient.SqlCommand

    $cmd.Connection = $conn

    $cmd.CommandType = [System.Data.CommandType]"StoredProcedure"

    $cmd.CommandText = $query

    $null = $cmd.Parameters.Add("@TVP", [System.Data.SqlDbType]::Structured)

    $cmd.Parameters["@TVP"].Value = $Datatable

    #Create and fill dataset

    $ds = New-Object system.Data.DataSet

    $da = New-Object system.Data.SqlClient.SqlDataAdapter($cmd)

    $null = $da.fill($ds)

    $conn.Close()

  • Jeff

    Below is full routine for your specific needs. RBAR I'm afraid, but unless you have an impressive number of drives I don't see that as an issue here.

    With thanks for all of your articles that have helped and instructed me.

    ###############################################################################

    $drives = Get-WmiObject Win32_LogicalDisk -computer 'SomeComputerName' | Select SystemName DeviceID, VolumeName, Size, FreeSpace

    # Open Connection

    $conn = New-Object System.Data.SqlClient.SqlConnection

    $connectionString = "Server=TestServer;Database=TestDB;Integrated Security=True;Connect Timeout=0"

    $conn.ConnectionString = $connectionString

    $conn.Open()

    # Create the Command object to execute the queries

    $cmd = New-Object System.Data.SqlClient.SqlCommand

    $cmd.Connection = $conn

    $cmd.CommandType = [System.Data.CommandType]::Text

    # Write Data

    foreach ($drive in $drives)

    {

    # Test for Null Values - otherwise Dynamic SQL goes wrong

    if ($drive.Size -eq $null) { $drive.Size = 0 }

    if ($drive.FreeSpace -eq $null) { $drive.FreeSpace = 0 }

    # Compose Query for this $drive - watch the single quotes for string values!!

    $query = "INSERT INTO dbo.DiskSpace (SystemName, DeviceID, VolumeName, Size, FreeSpace)

    VALUES ('" + $drive.SystemName + "', '" + $drive.DeviceID + "', '" + $drive.VolumeName + "', " + $drive.Size + ", " + $drive.FreeSpace + ")"

    # Uncomment next line to display query for checking

    #$query

    # Setup Command

    $cmd.CommandText = $query

    # Execute Command to insert data for this $drive

    $result = $cmd.ExecuteNonQuery()

    }

    # Tidy Up

    $conn.Close()

    ###############################################################################

    And if you want to read the data back:

    ############################################################################################

    # Check Data Stored via Reader

    # Open Connection

    $conn = New-Object System.Data.SqlClient.SqlConnection

    $connectionString = "Server=TestServer;Database=TestDB;Integrated Security=True;Connect Timeout=0"

    $conn.ConnectionString = $connectionString

    $conn.Open()

    # Create the Command object to execute the queries

    $cmd = New-Object System.Data.SqlClient.SqlCommand

    $cmd.Connection = $conn

    $cmd.CommandType = [System.Data.CommandType]::Text

    # Create Query

    $query = "SELECT SystemName, DeviceID, VolumeName, Size, FreeSpace FROM dbo.DiskSpace"

    # Get records

    $cmd.CommandText = $query

    $reader = $cmd.ExecuteReader()

    $values = @("","","",0,0) # empty array object

    # Read Rows

    while ($reader.read())

    {

    $reader.GetValues($values) | Out-Null # suppress the field count

    $values -join "`t" # join the row values with Tabs and display

    }

    $reader.Close()

    # Tidy Up

    $conn.Close()

    ############################################################################################

  • This is absolutely incredible.

    Gary, thanks for kicking off the thread with the bone of the code and some great examples to get me thinking in loops again. It's been a very long time.

    Calvo, Schleep, and Jonbes… thank all 3 of you for the wonderful examples and the embedded documentation. Using your examples and explanations, I'll be able to quickly get my arms around PowerShell and its nuances.

    There's just nothing like some practical examples and a bit of documentation. I really appreciate what all 4 of you have done for me. Thank all of you very much.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Hi Jeff,

    You are most welcome. I am sure I speak for all of us when I say it was a privilege to help you especially considering the huge input you provide to this community.

    I guess it once more shows that all of us have something to offer πŸ™‚

    Gaz

    -- Stop your grinnin' and drop your linen...they're everywhere!!!

Viewing 15 posts - 1 through 15 (of 31 total)

You must be logged in to reply to this topic. Login to reply