I own both an S3 and S4. My S3 is early (1983) and my S4 is a 1992. I bought my S4 for a specific reason. I like the look, but more important is that it's more of a GT car than a sports car. We use her for trips to the beach or drives up and down the coast. I also drive my S4 to work - I rarely take any of my S2's.
Please, don't take this wrong, but I worry about my S2's getting stolen - they're so hard to replace (except my 1980). If my S4 gets stolen, I'll be upset, but I can find another one in good condition. Today, I drove her to meet the trucking company, then to the office, then to meet my client, to lunch (drive thru window), back to my office, then to my favorite taco place and back home. Here's my oooops moment (I seem to do this with my S4, but not my other Spiders), I left the keys in the ignition again!!!! I'm not sure why this happens only in my S4, but it's happened too many times. Lucky for me, there weren't a lot of customers in the parking lot, so I dodged a bullet. The other times I've left the keys in the car has been safer (in front of my house or in the driveway).
The S4 is also longer than the S2 and S3. My S4 came with AC and 15" phone dial wheels (not crazy about the look, but it was the style in the day). The Motronic makes the car feel different. For instance, I was on the freeway today doing 75 or so. I decided to make her move and I was up to 95 pretty quickly. The VVT and Motronic (and maybe taller cam) make a difference. Here's the big BUT - the car feels more like a Toyota - starts easily, idles smoothly (no cammy idle), no smelly exhaust (good and bad), cat blocks some of the beautiful music from the tail pipe and a lot of the soul is gone. My early S2's are angry! Rough on the edges and you can tell there's a lot of camshaft in the motor. It doesn't matter if it's a 1750 or 2L, they both have a soul.
I asked some colleagues what they thought of the different versions of Spiders. Of course, the S1 was the favorite. The older crowd liked the metal bumper S2, while the younger crowd liked the S4. The rubber bumper S2 came next an the S3 was last. I find it very interesting, because prices follow in step.
My recommendation is to drive and S4 and see if you like it. Also, drive an early S2 (SPICA or carb).