The less full index pages are, the more of them must be read into memory to do a given search. The fuller they are, the less pages need to be read - so there is a performance hit.

For maximum efficiency, you should use fill factor = 100% - but this only works for read-only tables, otherwise you get page splits.

So for updateable tables you need to find a balance, & rebuild the indexes from time to time.