I think I'm with Pete on this one, but it depends on what you want. A "concours" car? Rip it all out and start over, spend lots of money and end up with something that is too expensive to drive hard and enjoy.
Want a driver? Make it work as it is today, drive the wheels off it, garage it when not in use and you'll get through many, many years without worrying about that fender. If it ever does rust again, you can always then spend the money to rectify it "properly".
As long as it is all metal, rust-free, sealed and not driven on wet&salty roads every day, it is going to be fine.