What are the differences between your login and the SQL Server Agent account login? Default database? Language? Anything else?
Does your package do anything user-sensitive, for example INSERT INTO MyTable (mycol) VALUES (SUSER_NAME())? If you could find out what task it's failing on, that will help you to work out the reason for the failure.
John