July 30, 2012 at 9:55 am
Hi all,
I have a table in my developing database that has URLs. Each URL is a link to a source and each link's data set is updated monthly but on different dates. How can I use SSIS to go through each link in the table, check it's last date modified, and then download the file if today is a newly modified date? I know that I need to use the HttpWebResponse.LastModified property but I am unsure about how to put it all together. Any help is truly appreciated.
Regards:
Mordred
Keep on Coding in the Free World
July 30, 2012 at 12:14 pm
Here is an algorithm for what I want:
For each link in tblTables_And_Links
variable DownloadURL = link
open httpconnection(DownloadURL)
TodaysDate = now
If TodaysDate > LastModified then
execute DownloadData
else goto next link
next link
Does this make sense?
Regards:
Mordred
Keep on Coding in the Free World
July 30, 2012 at 10:23 pm
Alright so I have an Execute SQL Task that connects to a Foreach Loop container. The SQL Task queries a view that shows all the links but I cannot get it to iterate through each row from the table in the database. I know that I am messing up with my properties somewhere but after hours of reading, tweaking, reading, and more tweaking, I am not much farther ahead. It's getting there though.
Regards:
Mordred
Keep on Coding in the Free World
July 31, 2012 at 10:14 pm
Hi all, please inform me as to why the following parameters for an Execute SQL Task and Foreach Loop Container are not right. This format is almost identical to a book I am reading but it is not working! The variables themselves are scoped to the package and everything else here seems correct but being new to SSIS I am not understanding the errors which are:
Error: 0xC002410D at Foreach Loop Container: The enumerator failed to retrieve element at index "1".
Error: 0xC001C012 at Foreach Loop Container: ForEach Variable Mapping number 1 to variable "User::sDbName" cannot be applied.
I know that the SQL statement outputs all of the rows that I want because I built it and the SQL Task completes successfully but the Foreach Loop stops. There are only 26 rows from this table for now so I know that the enumerator should have been able to retrieve the element at index 1 but for some reason it does not. Any ideas? Please?
Execute SQL Task
General:
ResultSet = Full result set
SQL Statement:
ConnectionType = OLE DB
Connection = MICHAEL-PC.EconAnalysis
SQLSourceType = Direct input
SQLStatement = SELECT Table_Links FROM dbo.tblLinks
BypassPrepare = True
No Parameter Mapping
Result Set:
Result Name = 0
Variable Name = flObj (of Object datatype)
No Expressions
Foreach Loop Container
Collection:
Enumerator = Foreach ADO Enumerator
ADO object source variable = User::flObj
Rows in first table selected
Variable Mappings:
Variable = User::sDbName
Index = 1
Regards:
Mordred
Keep on Coding in the Free World
Viewing 4 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply
This website stores cookies on your computer.
These cookies are used to improve your website experience and provide more personalized services to you, both on this website and through other media.
To find out more about the cookies we use, see our Privacy Policy