A faster way to convert Hex to Dec is

select CAST(CONVERT(VARBINARY, 'FF00FF', 2) AS INT)

The string to convert must be at least 2 chars long :

select CAST(CONVERT(VARBINARY, 'F', 2) AS INT) : error

select CAST(CONVERT(VARBINARY, '0F', 2) AS INT) : 15