This is an example using the PatIndex. I believe it's just one of many options but will work. Just add this WHERE Clause to your statement to return only items that are truely simple numbers
where PatIndex('%[^0-9]%',(RTRIM(LTRIM(YourColName)))) = 0
Thanks,
Eddie H