I always use a script task to check for the file and set a file name variable. I then add a condition on the connection to the next task that checks to make sure it has a value. You could add a second connection to your email task that has an expression that checks for the variable not having a value.

If you don't like using script tasks, you could use a ForEach loop, and configure it to look for files in your drop folder location. Set up the ForEach loop to put the file name in a variable. Add the email step after the ForEach loop, and make it conditional on the file name variable not being set.

If you want to use the error handler though, I believe you can use a script task in your event handler to check the error number, then make your email conditional on a specific error number that indicates the file is missing.