SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Dynamically Download FTP Files Using SSDT


Dynamically Download FTP Files Using SSDT

Author
Message
Sifiso W. Ndlovu
Sifiso W. Ndlovu
Valued Member
Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)

Group: General Forum Members
Points: 65 Visits: 268
Comments posted to this topic are about the item Dynamically Download FTP Files Using SSDT
rajarao.m
rajarao.m
SSC Journeyman
SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)

Group: General Forum Members
Points: 83 Visits: 23
Hi,

Actually i am having one requirement i.e
i am running some scripts that are scheduled by sql server agent and generating some files and loading that files into some path of folder c:/../../output.
from there i am copying and separating by identifying the files with some location_id like file contain words AZC(andhra),IZC(Tamilnadu)... again that are zipping and doing ftp daily basis ...so

Now i want to copy those files from output folder results of scheduled scripts and separation by identifying location id AZC,IZC and ftp....

these all things i have implemented by using ssis taking file from output folder and ftp. but unable to automation of ftp. its working when the package running manually so please anyone let me know how to schedule package or give me new procedure ...for achieve this.
Sifiso W. Ndlovu
Sifiso W. Ndlovu
Valued Member
Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)

Group: General Forum Members
Points: 65 Visits: 268
Hi rajarao.m,

when you trying to automate your package what error do you receive?

Are you using SSDT or BIDS?

Is the ".... identifying the files with some location_id ...." done via one SSIS's Loop containers?
rajarao.m
rajarao.m
SSC Journeyman
SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)SSC Journeyman (83 reputation)

Group: General Forum Members
Points: 83 Visits: 23
i haven't schedule, even i couldn't come up with how to schedule the package.

i have a package but how to schedule the package.

at that time i am getting doubts in what tool we can schedule either SQL agent or is there any built-in schedulers in SSIS like informatica ETL TOOl.
Jeff Moden
Jeff Moden
SSC Guru
SSC Guru (381K reputation)SSC Guru (381K reputation)SSC Guru (381K reputation)SSC Guru (381K reputation)SSC Guru (381K reputation)SSC Guru (381K reputation)SSC Guru (381K reputation)SSC Guru (381K reputation)

Group: General Forum Members
Points: 381586 Visits: 42995
I think the article is fine. It looks like it's nicely laid out in a step by step fashion that most whom work with SSIS will easily understand. I didn't, however, read it at any depth because I don't actually use SSIS because of all of it's wonderful shortcomings. I think it odd (well, that's the politically correct term, anyway) that a very well advertised ETL product such as SSIS does't come with this functionality built in and that you have to learn yet another tool such as SSDT to do what should be a fairly simple job. It's not like SSIS is an inmature product agewise.

Shifting gears a bit, I haven't had to do it in a while but automatic sheduled FTP downloads aren't difficult to do through the Command Prompt which also means that if you know how to use xp_CmdShell correctly, it's also possible to automate FTP downloads in a fairly easy manner through stored procedures. That solution, of course, is for Developers, which brings us back 'round to the original problem of SSIS not being able to do this in its original state.

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
If you think its expensive to hire a professional to do the job, wait until you hire an amateur. -- Red Adair

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
Nadrek
Nadrek
SSCertifiable
SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)

Group: General Forum Members
Points: 7996 Visits: 2741
Thank you for the article!

In addition to the howto article, I would recommend making sure everyone is fully aware that nothing in FTP is encrypted - the data transferred is in cleartext, blatantly visible to everyone between the sender and the receiver, as is the username and the password.
sbamaty
sbamaty
Right there with Babe
Right there with Babe (784 reputation)Right there with Babe (784 reputation)Right there with Babe (784 reputation)Right there with Babe (784 reputation)Right there with Babe (784 reputation)Right there with Babe (784 reputation)Right there with Babe (784 reputation)Right there with Babe (784 reputation)

Group: General Forum Members
Points: 784 Visits: 198
That is correct, nothing in FTP is encrypted unless vendor/provider encrypts the file before dumping it in FTP site. I don't think any file transfer utility/software automatically encrypts the file during file transfer process. Vendor/provider needs to encrypt it. There are different software that can be used to encrypt the file as well as username and password that taps into FTP site to download it. Once encrypted file is downloaded, you can perform decryption of the file in the SSIS buffer and import it into database table(s). To perform encryption and decryption, you don't need to rely on Pragmatic Work's Task Factory but you do need some other software to encrypt/decrypt. You can use CozyRoc for this purpose.
Nadrek
Nadrek
SSCertifiable
SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)SSCertifiable (8K reputation)

Group: General Forum Members
Points: 7996 Visits: 2741
sbamaty (3/11/2013)
I don't think any file transfer utility/software automatically encrypts the file during file transfer process. Vendor/provider needs to encrypt it.


SFTP does that (see MIT licensed PuTTy's PSFTP client) (see also GPL licensed Filezilla client and server).

It's best to both encrypt the file either with a symmetric encryption (passphrase/key), or with asymmetric public key encryption (certificate), prior to sending, even if you're using SFTP. That way two compromises are required, not just one, to intercept in transit.
Misha_SQL
Misha_SQL
SSCrazy
SSCrazy (2.4K reputation)SSCrazy (2.4K reputation)SSCrazy (2.4K reputation)SSCrazy (2.4K reputation)SSCrazy (2.4K reputation)SSCrazy (2.4K reputation)SSCrazy (2.4K reputation)SSCrazy (2.4K reputation)

Group: General Forum Members
Points: 2436 Visits: 1025
Great article, thank you!



Sifiso W. Ndlovu
Sifiso W. Ndlovu
Valued Member
Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)Valued Member (65 reputation)

Group: General Forum Members
Points: 65 Visits: 268
pleasure
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum







































































































































































SQLServerCentral


Search