Think from the other direction - if there were no rows, delete the file.

Get the rowcount during your data flow and put it into a variable. In the control flow of the package use a precedence constraint to check if the rowcount was zero - if it was, use a file system task to delete the file.