My first (very superficial) guess would be that when you query the SQL database directly, query folding happens (i.e. Power BI is able to simplify and filter the query which is executed against the source database) which ultimately could also reduce the amount of data being sent to the client.

When you use a tabular cube and direct query however, this may no longer be happening and all of the cube data may be sent to the client. I would run a trace to see what queries are executed against the tabular cube, and go from there.