I've put the file in G drive and ran the below script:

EXECUTE master.dbo.xp_delete_file 0,N'G:\',N'bak'

I don't get any error message. It says it completed successfully, but my file is still there.;-( So it can't be the $ sign.