Of course don't rebuild needlessly; check the stats and only rebuild those indexes that require it.
Also, with a reasonably well-tuned tempdb, you should strongly consider using the:
SORT_IN_TEMPDB = ON
option. This can help reduce fragmentation of the data file as well.
Even then, you may get a nasty suprise -- after you rebuild the indexes, the db file might be close to the same size it was originally! So be sure to check after rebuilding.
SQL DBA,SQL Server MVP(07, 08, 09) Prosecutor James Blackburn, in closing argument in the Fatal Vision murders trial:
If in the future, you should cry a tear, cry one for them [the murder victims]. If in the future, you should say a prayer, say one for them. And if in the future, you should light a candle, light one for them.