Sorted.

Needed to set the stored proc that serviced the queue to run as owner and made sure that owners was in the SQL Agent USer role in msdb and also had rights to run the stored procedure in msdb

Also had to digitally sign the stored procedure in my source database and use the certificate to create a user in msdb