It has been a long time since i worked in a mixed environments and my current one does not a sybase db so I can only mention few things which I can recall.

I can recall from my memory that we used to create DSNs in order to create linked server to Sybase. On your server, have you configured a system DSN for the above Sybase database? Please confirm the same.

Chandan