I would suggest that you look at:
- fiberglass windows such as Marvin's Integrity line or
- aluminum clad wood windows or
- wood windows.
None of these products should contain formaldehydes or petroleum.
Fiberglass does not require finishing of any kind. Aluminum clad wood requires the interior wood to be finished which can be done with a natural linseed oil, beeswax, or VOC free wood finish like Vermont Natural Coatings.
If you use all wood windows a VOC free paint like Mythic or Vermont Natural Coatings are what I'd recommend to finish them with.