Dynamically Download FTP Files Using SSDT

  • Sifiso W. Ndlovu

    SSC Enthusiast

    Points: 151

    Comments posted to this topic are about the item Dynamically Download FTP Files Using SSDT

  • rajarao.m

    SSC Veteran

    Points: 241

    Hi,

    Actually i am having one requirement i.e

    i am running some scripts that are scheduled by sql server agent and generating some files and loading that files into some path of folder c:/../../output.

    from there i am copying and separating by identifying the files with some location_id like file contain words AZC(andhra),IZC(Tamilnadu)... again that are zipping and doing ftp daily basis ...so

    Now i want to copy those files from output folder results of scheduled scripts and separation by identifying location id AZC,IZC and http://ftp....

    these all things i have implemented by using ssis taking file from output folder and ftp. but unable to automation of ftp. its working when the package running manually so please anyone let me know how to schedule package or give me new procedure ...for achieve this.

  • Sifiso W. Ndlovu

    SSC Enthusiast

    Points: 151

    Hi rajarao.m,

    when you trying to automate your package what error do you receive?

    Are you using SSDT or BIDS?

    Is the ".... identifying the files with some location_id ...." done via one SSIS's Loop containers?

  • rajarao.m

    SSC Veteran

    Points: 241

    i haven't schedule, even i couldn't come up with how to schedule the package.

    i have a package but how to schedule the package.

    at that time i am getting doubts in what tool we can schedule either SQL agent or is there any built-in schedulers in SSIS like informatica ETL TOOl.

  • Jeff Moden

    SSC Guru

    Points: 997199

    I think the article is fine. It looks like it's nicely laid out in a step by step fashion that most whom work with SSIS will easily understand. I didn't, however, read it at any depth because I don't actually use SSIS because of all of it's wonderful shortcomings. I think it odd (well, that's the politically correct term, anyway) that a very well advertised ETL product such as SSIS does't come with this functionality built in and that you have to learn yet another tool such as SSDT to do what should be a fairly simple job. It's not like SSIS is an inmature product agewise.

    Shifting gears a bit, I haven't had to do it in a while but automatic sheduled FTP downloads aren't difficult to do through the Command Prompt which also means that if you know how to use xp_CmdShell correctly, it's also possible to automate FTP downloads in a fairly easy manner through stored procedures. That solution, of course, is for Developers, which brings us back 'round to the original problem of SSIS not being able to do this in its original state.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.
    "Change is inevitable... change for the better is not".
    "Dear Lord... I'm a DBA so please give me patience because, if you give me strength, I'm going to need bail money too!"

    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Nadrek

    SSC-Insane

    Points: 20039

    Thank you for the article!

    In addition to the howto article, I would recommend making sure everyone is fully aware that nothing in FTP is encrypted - the data transferred is in cleartext, blatantly visible to everyone between the sender and the receiver, as is the username and the password.

  • sbamaty

    SSC Eights!

    Points: 936

    That is correct, nothing in FTP is encrypted unless vendor/provider encrypts the file before dumping it in FTP site. I don't think any file transfer utility/software automatically encrypts the file during file transfer process. Vendor/provider needs to encrypt it. There are different software that can be used to encrypt the file as well as username and password that taps into FTP site to download it. Once encrypted file is downloaded, you can perform decryption of the file in the SSIS buffer and import it into database table(s). To perform encryption and decryption, you don't need to rely on Pragmatic Work's Task Factory but you do need some other software to encrypt/decrypt. You can use CozyRoc for this purpose.

  • Nadrek

    SSC-Insane

    Points: 20039

    sbamaty (3/11/2013)


    I don't think any file transfer utility/software automatically encrypts the file during file transfer process. Vendor/provider needs to encrypt it.

    SFTP does that (see MIT licensed PuTTy's PSFTP client) (see also GPL licensed Filezilla[/url] client and server).

    It's best to both encrypt the file either with a symmetric encryption (passphrase/key), or with asymmetric public key encryption (certificate), prior to sending, even if you're using SFTP. That way two compromises are required, not just one, to intercept in transit.

  • Misha_SQL

    SSCertifiable

    Points: 5401

    Great article, thank you!

  • Sifiso W. Ndlovu

    SSC Enthusiast

    Points: 151

    pleasure

Viewing 10 posts - 1 through 10 (of 10 total)

You must be logged in to reply to this topic. Login to reply