linked server will always have that issue - what you need to do is to alleviate the problem.
when pulling data from Oracle to SQL one of the biggest issues is conversion of date and numbers data types - a trick that is commonly used and that normally results in significant performance increase is to do the following
use openquery to retrieve the data onto temp tables where feasible
within it and on the oracle side, convert ALL dates and numbers to strings - and then convert back to desired datatype within the sql side (use dynamic sql if required to pass parameters)
from openquery(linkedserver, 'select to_char(datetimecol, 'YYYY-MM-DD HH24:MI:SS') as datetimecol
, to_char(mydecimalcol, 'fm9999999.90') as mydecimalcol
from mytbl') t1
if you use your linked server as part of a join on with other local tables this is always going to perform badly