Although, few

I have a question about the sentence below:

Although the official work day finishes around five o'clock, few workers leave for home until much later.

This is supposed to mean" although the official work day finishes around 5 o'clock, but most people leave for home much later" (translation says that), but I don't know why the sentence uses few workes. if we use " few", it means similar to " no people". My question is, is there any grammar rule when we use "although" and " few". Or if I misunderstood the meanings, please advise..