Preserving your indoor air quality is always a good goal, so the short answer is yes, if the boxes are relatively new.
- Chances are, if you have had them for a while, the offgassing process is probably near complete.
- If they are new-ish, consider placing them outside for a day or two where the heat from the sun can accelerate the offgassing process.
- Do not raise the interior temperature of your house to try and achieve this as is sometimes done to accelerate the offgassing process for new paint or carpeting.
For the future, consider purchasing baskets or wooden boxes/crates from garage sales or reputable retailers that offer natural, chemical-free goods for storing items throughout your home.
These containers can be recycled or will biodegrade at the end of their useful lives, unlike nonrecyclable plastic tubs.