One of the things to watch for with table-valued functions (especially the multi-statement table valued functions) is that the resultant table, just like a table variable) has no column statistics and no indexes. The optimiser doesn't know how many rows are in the function's result table. That's not an issue for small numbers of rows, but it becomes a very big issue for larger numbers of rows.

If the optimiser misjudges the number of rows, it can result in very poor query plans.

Note, this is a maybe, possibly, could be, not definitely and always. Test carefully.