The usual answer is that pop-up headlights existed because of an American law. If that were all, they would not have existed before the law and would have gone when it changed. They were there before it and they stayed after it. The Cord 810 had them in 1935, five years before the American states prescribed a single type of lamp. And the rule on that type of lamp fell in 1983, after which Toyota launched the MR2 with pop-ups in 1984 and Chevrolet the Corvette C5 in 1997.
Three things get mixed up here: a design choice, a rule about the shape of the lamp and a rule about its height. Only the last still applies.
A design first
In the Cord, the driver wound the lamps out of the wings with cranks beside the dashboard. It was a nuisance. No rule asked for it. Buick followed in 1938 with electrically operated lamps on the Y-Job concept car. The 1962 Lotus Elan was the first production car with lamps raised by their own mechanism. Until then it was a way of getting a smooth nose, for anyone who wanted one and would pay.

A 1936 Cord 810 Westchester. The headlamps are wound away inside the wings.
Then the only legal way out
In 1940 the American states agreed that every car would have two round sealed-beam lamps of 7 inches. For seventeen years nothing else was allowed. Four round lamps arrived in 1957 and rectangular ones in 1974. A sealed beam comes in a fixed size, so the designer drew the nose around the lamp. Europe had no such requirement: a lamp there could be any shape, and European halogen units were effectively banned in America.
On top of that, the federal standard FMVSS 108 requires the centre of the headlamp to sit at least 22 inches above the road, or 55.9 cm. A Lotus Esprit S1 stood 1,111 mm tall. So the lamp had to sit halfway up the whole car, and at the front of a wedge there is no bodywork at that height.
Do the sum and one solution remains. The lamp has a fixed size, it must sit higher than the nose and it is not needed during the day. So it folds away.
The rest of the world got the same answer. A manufacturer selling in America would rather build one nose than two, because two noses means different pressings, different approvals and different wiring.
The Ferrari Daytona shows how far this reached. The first Daytonas, including all the European ones, had fixed lamps behind a continuous acrylic nose. Then a new American rule banned lamps behind covers. From around 1971 every Daytona got pop-ups. A buyer in Italy was paying for a mechanism fitted because of a rule on another continent. According to the dealer Girardo, collectors now pay more for the early acrylic nose.

A Ferrari 365 GTB/4 Daytona with the early acrylic nose.
The rule fell and the lamp stayed
In 1983 the American government granted a petition Ford had filed in 1981. Lamps could now take any shape, with replaceable bulbs and aerodynamic polycarbonate lenses. The 1984 Lincoln Mark VII was the first to use this. If the sealed beam were the whole explanation, the story would have ended there. Instead, pop-ups spread through the 1980s from expensive exotics to ordinary cars: the Celica, the Prelude and in 1989 the Miata. The MR2 hardly counts, since it was already drawn by 1983. The Corvette C5 does count: it was designed after the rule changed and got the lamps anyway.

A first-generation Toyota MR2, built from 1984, with its lamps down.
The height rule stayed, and lamps of the day were still big. A designer who wanted a low nose had few other options. They had also come to belong on a sports car, which is a matter of taste. What overtook them was the lamp itself. Projector lamps, as on the 1990 Nissan 300ZX Z32, get enough light out of a small lens. The lamp then fits in the wing at 56 cm. The nose can drop away below it. From then on a pop-up was a choice: motors, gears, hinges and wiring for something that could be done without.
Did pedestrian rules kill them?
You often read that, and as a matter of law it is wrong. The European pedestrian directive 2003/102 applied to new types from 1 October 2005 and to all new cars only from the end of 2012. The last Lotus Esprit left the line on 20 February 2004, the last Corvette C5 on 2 July 2004. As far as that directive goes, both could have carried on for years. And no simple American law against pop-ups exists.
As an expectation, it may be right. The Mazda MX-5's move to fixed lamps in 1998 is put down to pedestrian tests the old lamps no longer passed. That was seven years before the directive. Manufacturers were probably already counting the test, along with weight and cost. Each new generation lost another few. The rule is mostly why they will not return. A raised lamp is a hard edge that sticks out, and that is what such a test punishes.
The rule that made them necessary is still in American law: at least 55.9 cm. A Lamborghini Revuelto stands 1,160 mm tall, so in America the centre of its lamp sits at no less than 48% of the height of the car. Look at a low car today. You see narrow lamps high in the wing and a nose dropping away below them. People read that as style. It is the same sum as on the Esprit, with a lamp that has become small enough not to need to move.
Images: Lotus Esprit Classic-Days 2022 IMG 7239 by Alexander Migl (CC BY-SA 4.0); 1936 Cord 810 Westchester Sedan (21559608830) by Sicnag (CC BY 2.0); Ferrari 365 GTB 4 Daytona Plexiglass by Calreyn88 (CC BY 4.0); Toyota MR2 AW11 white by Jacob Frey 4A (CC BY 2.0). Images resized.
