You are mostly correct, but you do read the data far more often then you write it, correct? It is in this situation that the optimizer can be better with absolute certainty about the relationships. Hence it is often good to have referential integrity with foreign keys that are trusted.

Where the optimizer can do a better job is in eliminating joins that are not required when the foreign keys are trusted. Imagine a view that uses inner joins to combine attributes from several normalised tables and present it as one table. If a consuming query that uses this view and only requests attributes from one or two tables, then the query optimizer can only remove the unneeded joins if there are trusted foreign keys to these tables that are used in the view. It can do so, because it knows implicitly the records in the unneeded tables do exist, which would affect the outcome of the query if they did not.

Without trusted relationships, the optimiser must make a plan that joins the unused tables too, just to make sure a record can be part of the result set. This is the type of real situation you see performance benefits of having trusted relationships.