Thanks for the replies, y'all. I checked the booster hose and there is some slight residue inside it, nothing dripping or anything, but it's there. Technically if I temporarily block the check valve at the intake, I should be able to isolate the issue to brake booster system overall, right?
Does the engine RPMs rising at idle when partially blocking barrel #4 just mean that that cylinder is running extremely lean due to air entering from elsewhere?
Also, any tips on checking out the brake booster itself? I've never removed it or messed with the braking system. Is it possible to service it without replacing it?