Another thing to check while in there looking at the retaining bolt is the condition of the rubber stuff in that end of the shifter. (there is a rubber bush between the shift post proper and the part that clamps the shifter to the trans tower)
Prolly the best way to check that is get the shifter out of the car and clamp the mounting part in a vice then lever on the shifter to see how much, if any, it moves.
Mine went all soft and gooey due to a combination of transmission heat and oil saturation, (PO was not what one would call a 'maintainence sensitive' person), and gave every impression that one was changing gears by shoving around a wet noodle.
A few fixes for it if that's what's occuring:
Replace the bushing if available
Replace the shift post assembly if not
Remove the shifter and either weld washers on either end of the bushing, or drill and tap for set screws in through the side so that the shifter, bushing and fixing section become all one solid piece. (vibration isn't really that bad with those fixes. besides, you shouldn't be driving with one hand resting on the shifter all the time anyway

)