Well first it is required by code. Secondly, you have the smell factor which can be umm pulled down into other areas of the house.
But most importantly, it is to remove the humid air from your bath and direct it outside.
Up front you probably won't realize any "savings" but if and when you start developing indoor air quality issues, have to deal with rotted sheathing or drywall.
For more information:
Read "Can I do something to prevent outside cool air being drawn into the house when I turn on the exhaust fan in the bathroom?" a Q&A answered by Michael Holcomb.
Also, read Sean Lintow's blog post "Getting the details right: Bathroom Exhaust Venting."