Usually with a shorter dress and maybe a hat instead of a jeweled hair ornament.

My guess - and it's just that - is that it's a holdover from the Victorian/Edwardian upper class way of dressing, that you 'dressed' for all sorts of times of day or occasions and they got progressively more formal as the day went on. From a morning dress to a day dress to a tea dress to a dinner dress and then maybe a ballgown.