Well the answer is yes and no.
On the no side, any off gassing has already happened and tearing out / replacing the paneling would kick up a lot of dust, and add in new VOCs.
On the yes side, it could be worth it, especially on the exterior walls of the room, if you also do some air-sealing and insulating work. Also on the yes side is if there has ever been any water issues in the walls. This project would allow you to remediate it.