The general guideline is to ignore tables under 1000 pages. Fragmentation only affects large range scans from disk, not general operations against pages in the buffer pool.

Make sure you're doing stats maintenance though.