Azure file copy creating a directory to hold the renamed file

  • Irish-Toon

    Grasshopper

    Points: 12

    Hi All,

    I'm pretty new to azure and am hitting the following problem. I've got a notebook that creates an output csv file. The file gets named to the default Part-0000-tid and I want to rename it to something more sensible. The code below shows what I've done. The problem is it creates a subfolder in the destination folder with the new filename and then within that folder it has the renamed file ( see attached screenshot). It looks like I'm close but no quiet right. I need the new file to be up 1 level and appear in the outbound folder. Any help would be appreciated

    %python

    import os, sys, datetime

    readPath = "/mnt/publisheddatasmets1mig/metering/smets1mig/cs/system_data_build/notes/outbound/"

    writePath = "/mnt/publisheddatasmets1mig/metering/smets1mig/cs/system_data_build/notes/outbound/"

    file_list = dbutils.fs.ls(readPath)

    for i in file_list:

    file_path = i[0]

    file_name = i[1]

    file_name

    Current_Date = datetime.datetime.today().strftime ('%Y-%m-%d-%H%M%S')

    fname = "CS_Notes_" + str(Current_Date) + ".csv"

    for i in file_list:

    if i[1].startswith("part-00000"):

    dbutils.fs.cp(readPath+file_name,writePath+fname)

     

    Attachments:
    You must be logged in to view attached files.
  • Site Owners

    SSC Guru

    Points: 80379

    Thanks for posting your issue and hopefully someone will answer soon.

    This is an automated bump to increase visibility of your question.

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply