Electric baseboards are one of the least efficient heating options possible. Unfortunately, there isn't likely to be a cheap and easy solution.
However, the good news is that with bills like that, it may be very cost-effective to make some larger upgrades. I would recommend investing in a BPI Building Analyst energy audit, which would include someone inspecting your house in great detail for opportunities to reduce your bills.
Your ideal solution will most likely involve air sealing and insulating your home.
- Invest in upgrading your attic first, crawl/basement (if existing) second, and then walls third.
- Don't do anything with the windows unless you have a large budget and want to replace the windows for other reasons as well (comfort, sound management, aesthetic, etc.).
- Then I'd recommend installing a ductless heat pump, which is the most efficient form of space conditioning. It also has the nice bonus of being able to provide air conditioning.
For more information: