September 13, 2012 at 12:35 pm
Thanks for the effort Gus. At this point, I'm looking at WGet. Although I really don't like trying to get GNU utilities to work in production on Windows, it does have a sterling reputation and claims to be able to do this, including automatically checking the remote timestamps for updates (though not the Zip part, though I can do that separately).
If that falls through then I'll either write it myself in .Net (I can probably find a binary download example at StackOverflow) or else jump into Powershell.
[font="Times New Roman"]-- RBarryYoung[/font], [font="Times New Roman"] (302)375-0451[/font] blog: MovingSQL.com, Twitter: @RBarryYoung[font="Arial Black"]
Proactive Performance Solutions, Inc. [/font][font="Verdana"] "Performance is our middle name."[/font]
September 13, 2012 at 12:57 pm
RBarryYoung (9/13/2012)
EL Jerry (9/12/2012)
Barry,I noticed you can do HTTP downloads from this site, but can you do FTP downloads from this site? I was able to FTP connect to http://ftp.irs.gov, but I don't have a user/password to log in.
I think that it's HTTP only, but that is what I am trying to do.
One other question, is the name of the file to download always the same or is it dynamic?
The files have fixed names, but I also need to get the "Last Modified" date/time from the directory listing so that I can tell if the IRS has updated any of them in the previous 24 hours.
If this is HTTP-only, then I'm sorry this is well over my head. I have a batch/ftp script that works wonders for my requirements even with dynamic names (part of the file name is the current date in yyyymmdd format). I'm sorry I cannot be of further help here.
September 13, 2012 at 2:37 pm
Well, GNU's WGet works and turns out to be almost perfect for this usage. In fact I am blown away by how good it is.
One of the things that you can do is use its "--mirror" option to tell it to compare an HTTP sites' HTML link tree subfolders, their target files and last-modified dates against a local subfolder tree (created previously by the same command) and then to only DownLoad (and overwrite) the ones that have changed since. It even accepts a wildcard file-spec so that I can tell it which 6 of the 90 or so files I actually want. Seriously cool. :w00t:
[font="Times New Roman"]-- RBarryYoung[/font], [font="Times New Roman"] (302)375-0451[/font] blog: MovingSQL.com, Twitter: @RBarryYoung[font="Arial Black"]
Proactive Performance Solutions, Inc. [/font][font="Verdana"] "Performance is our middle name."[/font]
September 14, 2012 at 4:41 am
Wow. That does sound cool. Thanks for letting us know about it.
September 14, 2012 at 6:14 am
Awesome find. Sounds perfect for what you need.
GNU does some seriously good work. I've used a bunch of the products from that community. GNUPG for example.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
September 14, 2012 at 6:17 am
Another gnu program to look at is cURL. It is a little more complex than wget but can download every protocol under the sun
For performance Issues see how we like them posted here: How to Post Performance Problems - Gail Shaw[/url]
Need to Split some strings? Jeff Moden's DelimitedSplit8K[/url]
Jeff Moden's Cross tab and Pivots Part 1[/url]
Jeff Moden's Cross tab and Pivots Part 2[/url]
Viewing 6 posts - 31 through 35 (of 35 total)
You must be logged in to reply to this topic. Login to reply