I usually check the output with a digital voltmeter. Heat it up (either by running the car to heat it up with exhaust gas or remove it and use a propane torch). A functioning O2 sensor will show between 0.2 and 0.9V.
That's right, it is an electro-chemical cell. The voltage generated is related to the difference in oxygen concentration (actually partial pressure) on the two sides of the cell. One side is air, so 20% oxygen on the other side gives zero output. Zero oxygen gives maximum output. The sensor application was an accidental find by researchers who were working on batteries for the space program. A retired former colleague was one of the original researchers at the Westinghouse R&D labs in Pittsburgh.