Manual or auto?
On my manual-transmission 164 I managed to unbolt the starter from the gearbox without removing the rear exhaust manifold. However, I was taking the gearbox off, and so I didn't need to actually remove the starter from the engine bay. I think that since you do, you might be better to take the exhaust manifold off (though you could try without first and see if you can get the starter out). The manifold is a ***** to refit - needs three new copper gaskets and each is located on only one stud, so very difficult to keep them lined up while you get the bolts in (the heatshield is in the way).
The infamous starter top bolt can actually be dealt with by removing the left wheel and, from under the car, reach up with your left hand around the gearbox to guide the socket onto the bolt with your fingertips. Worked for me. The socket should be on several long extensions so the ratchet handle is able to swing somewhere beside the exhaust manifold. I used 1/4" extensions and socket, with a reducer from a 3/8" ratchet.
After I'd refitted the manifold three times it eventually stopped leaking - nearly - and I think that is a worse job than the starter removal itself.
'03 Stilo Abarth Selespeed, '02 Spider V6, '88 Uno Turbo