A differential backup contains all the changes made since the last full backup.
In your case, if you are taking a full backup and then rebuilding the indexes afterwards, all the changes involved in rebuilding the indexes will be reflected in differential backups going forward. If you're rebuilding all indexes on all tables, then I would expect for your differential backups to be about the same size as your full backups.
To prevent my differential backups from being excessively large, I tend to rebuild necessary indexes immediately before a full backup.
Does this help?
Microsoft Certified Master: SQL Server 2008
Blog: The Outer Join