January 1, 2013 at 6:41 am
Hi all,
Need help in solving an issue.
i have a SSIS pkg deployed in INT & Dev env respectively. The data flow is like this
One oledb source and script component component (using it as Destination)
Inside oledb source component i'm using an stored procedure as Sql Command. Please see the attached screenshot of my DataFlow.
Now the issue is:
If i execute the pkg in Dev env i'm getting one output file generated in the shared path, however if i execute the same pkg in INT env i'm getting two o/p files getting generated in the shared path , with the header created in one of the o/p file and the data records divided among these two o/p files equally.
Now if i deploy the INT stored procedure version in dev env then multiple o/p files are again generated 🙁 and i have compared both the version of stored procedure and they are same... 🙁
I'm not sure why the script component is generating multiple o/p files.:unsure:
Please advice.
Thanks
sam
January 1, 2013 at 6:47 am
Can you share the code inside the Script Component?
If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.
January 1, 2013 at 6:52 am
Hi Phil,
Please find the script component code below....
Option Explicit Off
Option Strict Off
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.IO
Imports System.Text
<Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute> _
<CLSCompliant(False)> _
Public Class ScriptMain
Inherits UserComponent
Dim fPHTaxReport As FileStream
Dim swPHTaxWriter As StreamWriter
Dim UTF8Encoding As Encoding
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim FileFormat As String = Me.Variables.varfileformat.ToString()
Try
Dim strtarget As String = Me.Variables.varTargetFilepath.ToString + "Consolidated Report to PH Tax Department_" + DateTime.Now.ToString("yyyyMMddhhmm") + FileFormat
fPHTaxReport = New FileStream(strtarget, FileMode.Append, FileAccess.Write)
swPHTaxWriter = New StreamWriter(fPHTaxReport, UTF8Encoding.UTF8)
If Row.RecordType = "H" Then
swPHTaxWriter.WriteLine(Row.RecordType + "|" + _
Row.TransactionType + "|" + _
Row.TINNumber + "|" + _
Row.OwnersRegisteredName + "|" + _
Row.OwnersLastName + "|" + _
Row.OwnersFirstName + "|" + _
Row.OwnersMiddleName + "|" + _
Row.OwnersTradeName + "|" + _
Row.OwnersAddress1 + "|" + _
Row.OwnersAddress2 + "|" + _
CStr(Row.ExemptSales) + "|" + _
CStr(Row.ZeroRatedSales) + "|" + _
CStr(Row.TaxableSales) + "|" + _
CStr(Row.OutputTax) + "|" + _
Row.RDOCode + "|" + _
Row.TaxableMonth + "|" + _
Row.FiscalYearEnding)
ElseIf (Row.RecordType = "D") Then
swPHTaxWriter.WriteLine(Row.RecordType + "|" + _
Row.TransactionType + "|" + _
Row.TINNumber + "|" + _
Row.OwnersRegisteredName + "|" + _
Row.OwnersLastName + "|" + _
Row.OwnersFirstName + "|" + _
Row.OwnersMiddleName + "|" + _
Row.OwnersAddress1 + "|" + _
Row.OwnersAddress2 + "|" + _
CStr(Row.ExemptSales) + "|" + _
CStr(Row.ZeroRatedSales) + "|" + _
CStr(Row.TaxableSales) + "|" + _
CStr(Row.OutputTax) + "|" + _
Row.RDOCode + "|" + _
Row.TaxableMonth)
End If
swPHTaxWriter.Flush()
fPHTaxReport.Close()
Catch ex As Exception
swPHTaxWriter.Flush()
fPHTaxReport.Close()
Throw ex
End Try
End Sub
End Class
Thanks
January 1, 2013 at 7:49 am
I'm puzzled by this. The Script Component only seems to write to a single file.
Can you tell me the file names of the two files which are generated please? Are they created at the same time?
If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.
January 1, 2013 at 7:57 am
Hi Phil,
Can you tell me the file names of the two files which are generated please? Are they created at the same time?
The files that are generated are of different time stamp, they differ by few minutes/sec apart.
Thanks
Sam
January 1, 2013 at 8:04 am
Is there any additional complexity to the SSIS package which you have not mentioned?
Is the data flow in a Foreach container, perhaps?
No idea why replacing a proc with another version which is exactly the same should cause this behaviour - sounds very unlikely.
If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.
January 2, 2013 at 4:58 am
Hi Phil,
I had to make few changes in my script component code, so as to generate only one consolidated output file. 🙂
If you had seen my code earlier i had created both filestream & streamwriter object inside the sub module 'Input0_ProcessInputRow', now i have added two new submodule one is 'Pre execute and another is 'PostExecute'. Inside PreExecute i tried to create new object for filestream & streamwriter and inside postexecute sub module i tried to dispose the objects i have created.
here is my updated code in script component:
Option Explicit Off
Option Strict Off
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.IO
Imports System.Text
Public Class ScriptMain
Inherits UserComponent
Dim strtarget As String
Dim FileFormat As String
Dim fPHTaxReport As FileStream
Dim swPHTaxWriter As StreamWriter
Dim UTF8Encoding As Encoding
Public Overrides Sub PreExecute()
FileFormat = Me.Variables.Varfileformat.ToString()
strtarget = Me.Variables.VarTargetFilepath.ToString + "Consolidated Report to PH Tax Department_" + DateTime.Now.ToString("yyyyMMddhhmm") + FileFormat
fPHTaxReport = New FileStream(strtarget, FileMode.Append, FileAccess.Write)
swPHTaxWriter = New StreamWriter(fPHTaxReport, UTF8Encoding.UTF8)
MyBase.PreExecute()
End Sub
Public Overrides Sub PostExecute()
swPHTaxWriter.Flush()
fPHTaxReport.Close()
MyBase.PostExecute()
End Sub
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
If Row.RecordType = "H" Then
swPHTaxWriter.WriteLine(Row.RecordType + "|" + _
Row.TransactionType + "|" + _
Row.TINNumber + "|" + _
Row.OwnersRegisteredName + "|" + _
Row.OwnersLastName + "|" + _
Row.OwnersFirstName + "|" + _
Row.OwnersMiddleName + "|" + _
Row.OwnersTradeName + "|" + _
Row.OwnersAddress1 + "|" + _
Row.OwnersAddress2 + "|" + _
CStr(Row.ExemptSales) + "|" + _
CStr(Row.ZeroRatedSales) + "|" + _
CStr(Row.TaxableSales) + "|" + _
CStr(Row.OutputTax) + "|" + _
Row.RDOCode + "|" + _
Row.TaxableMonth + "|" + _
Row.FiscalYearEnding)
ElseIf (Row.RecordType = "D") Then
swPHTaxWriter.WriteLine(Row.RecordType + "|" + _
Row.TransactionType + "|" + _
Row.TINNumber + "|" + _
Row.OwnersRegisteredName + "|" + _
Row.OwnersLastName + "|" + _
Row.OwnersFirstName + "|" + _
Row.OwnersMiddleName + "|" + _
Row.OwnersAddress1 + "|" + _
Row.OwnersAddress2 + "|" + _
CStr(Row.ExemptSales) + "|" + _
CStr(Row.ZeroRatedSales) + "|" + _
CStr(Row.TaxableSales) + "|" + _
CStr(Row.OutputTax) + "|" + _
Row.RDOCode + "|" + _
Row.TaxableMonth)
End If
End Sub
End Class
One quick observation i found:
if i run the old code for record count less than 800 than i'm getting 1 o/p file, if its more than 1000 then i'm getting multiple output file.
But with the above updated code there don't seems to be any issue. Withn 'N' no. of records coming frm the source i'm getting only one o/p file created in the shared path.
Not sure what exactly is ithe issue with script component....
Is it something to do with no of records cached in the buffers???
Any idea
Thanks
Sam
January 2, 2013 at 5:52 am
sam 55243 (1/2/2013)
Hi Phil,I had to make few changes in my script component code, so as to generate only one consolidated output file. 🙂
If you had seen my code earlier i had created both filestream & streamwriter object inside the sub module 'Input0_ProcessInputRow', now i have added two new submodule one is 'Pre execute and another is 'PostExecute'. Inside PreExecute i tried to create new object for filestream & streamwriter and inside postexecute sub module i tried to dispose the objects i have created.
here is my updated code in script component:
Option Explicit Off
Option Strict Off
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.IO
Imports System.Text
Public Class ScriptMain
Inherits UserComponent
Dim strtarget As String
Dim FileFormat As String
Dim fPHTaxReport As FileStream
Dim swPHTaxWriter As StreamWriter
Dim UTF8Encoding As Encoding
Public Overrides Sub PreExecute()
FileFormat = Me.Variables.Varfileformat.ToString()
strtarget = Me.Variables.VarTargetFilepath.ToString + "Consolidated Report to PH Tax Department_" + DateTime.Now.ToString("yyyyMMddhhmm") + FileFormat
fPHTaxReport = New FileStream(strtarget, FileMode.Append, FileAccess.Write)
swPHTaxWriter = New StreamWriter(fPHTaxReport, UTF8Encoding.UTF8)
MyBase.PreExecute()
End Sub
Public Overrides Sub PostExecute()
swPHTaxWriter.Flush()
fPHTaxReport.Close()
MyBase.PostExecute()
End Sub
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
If Row.RecordType = "H" Then
swPHTaxWriter.WriteLine(Row.RecordType + "|" + _
Row.TransactionType + "|" + _
Row.TINNumber + "|" + _
Row.OwnersRegisteredName + "|" + _
Row.OwnersLastName + "|" + _
Row.OwnersFirstName + "|" + _
Row.OwnersMiddleName + "|" + _
Row.OwnersTradeName + "|" + _
Row.OwnersAddress1 + "|" + _
Row.OwnersAddress2 + "|" + _
CStr(Row.ExemptSales) + "|" + _
CStr(Row.ZeroRatedSales) + "|" + _
CStr(Row.TaxableSales) + "|" + _
CStr(Row.OutputTax) + "|" + _
Row.RDOCode + "|" + _
Row.TaxableMonth + "|" + _
Row.FiscalYearEnding)
ElseIf (Row.RecordType = "D") Then
swPHTaxWriter.WriteLine(Row.RecordType + "|" + _
Row.TransactionType + "|" + _
Row.TINNumber + "|" + _
Row.OwnersRegisteredName + "|" + _
Row.OwnersLastName + "|" + _
Row.OwnersFirstName + "|" + _
Row.OwnersMiddleName + "|" + _
Row.OwnersAddress1 + "|" + _
Row.OwnersAddress2 + "|" + _
CStr(Row.ExemptSales) + "|" + _
CStr(Row.ZeroRatedSales) + "|" + _
CStr(Row.TaxableSales) + "|" + _
CStr(Row.OutputTax) + "|" + _
Row.RDOCode + "|" + _
Row.TaxableMonth)
End If
End Sub
End Class
One quick observation i found:
if i run the old code for record count less than 800 than i'm getting 1 o/p file, if its more than 1000 then i'm getting multiple output file.
But with the above updated code there don't seems to be any issue. Withn 'N' no. of records coming frm the source i'm getting only one o/p file created in the shared path.
Not sure what exactly is ithe issue with script component....
Is it something to do with no of records cached in the buffers???
Any idea
Thanks
Sam
Hi Sam
I think you've cracked it. Your solution now is much nicer. Previously everything was running once for every row of data and, in particular, this expression was being recalculated every time:
DateTime.Now.ToString("yyyyMMddhhmm")
What this means, I think, is that if your process spanned more than a minute you would get multiple output files.
If you haven't even tried to resolve your issue, please don't expect the hard-working volunteers here to waste their time providing links to answers which you could easily have found yourself.
Viewing 8 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply