After thinking more about this and retrying the same I managed to successfully upgrade from 2003 to 2008
I had to do some manual steps along the way but pretty much everything was smooth
Since I had the servers installed side by side on different machines
I stopped the sql 2003 and copied over the database and temp database to the new server
I attached the new database (renaming it... I had make and change a trigger set on the Schedules table to reflect this change)
I connected to the database from the RS Integrated instance
Again it showed that it is in integrated mode
I stopped and started the instance as explained in the article bellow and the Database got upgraded and switched to integrated mode
Now both the old server and the new server showed in the Scale-out Deployment section of teh RS Configuration.
I tried removing the old server here without luck...
I also could not backup the keys
I stoped the service and in SQL manager found teh table that stores this settings and simply removed it
delete from dbo.Keys where MachineName = 'OldServerName'
After this i started the instance and managed to backup the keys
I also had to change the links to the reports as I had to change the domain since I was doing this whole thing side by side and did not want downtime for the users
so i did thos for each link
set [Path] ='http://neurl/reports'
where [Path] ='http://oldurl/reports'
here I realized there was a trigger pointing to the temporary database which now had a different name
after updating the trigger with the new temp DB name I was ok.
after this I had to redeploy each report, as it could not find it... but that was not a big deal since I had to anyways...
teh following article helped clear out thingshttp://technet.microsoft.com/en-us/library/ms403392.aspx
especially the part
To upgrade the database manually
If Setup did not upgrade the report server database automatically, your report server installation is only partially upgraded. To complete the upgrade, you must upgrade the report server database manually by running the Reporting Services Configuration tool after Setup is finished.
1.Run the Reporting Services Configuration tool and connect to the report server that you just upgraded. For more information about how to start the tool and connect to a server, see How to: Start Reporting Services Configuration.
2.Open the Database Setup page, and then click Connect to connect to the SQL Server instance that hosts the report server database.
3.Select the report server database that you want to upgrade.
4.Click Apply to update the database schema.