The two things ESP-IDF was doing for you

Watchdogs. Before jumping into the image at offset 0, the ROM arms the RTC watchdog and timer-group 0's watchdog in "flash boot" mode — insurance against a second-stage bootloader that hangs. We are that second-stage bootloader now. Leave them armed and the board reboots every few hundred milliseconds. disable_watchdogs() turns both off and puts the super watchdog (which can't be disabled, only fed) into hardware auto-feed.

The CPU clock. The ROM leaves the CPU on the crystal divided by two — 20 MHz — and that is not fast enough to bit-bang an addressable LED. This is worth understanding, because it fails in a way that looks like something else entirely.

The cycle-count poll in led_write() costs a fixed ~9 cycles per pass, so every pulse overshoots by that much. The overshoot is a cycle count, so what it costs in nanoseconds depends on the clock — and the whole protocol comes down to telling a 350 ns pulse from an 800 ns one. Measured on the board:

A misread bit is a wrong colour, not a dark LED — so the symptom points at the palette when the cause is the clock.

set_cpu_160mhz() fixes it in three register writes. That works because the ROM has already started the PLL to clock the flash it read this image from, so there's no analog bring-up to do — which is the part that would have meant driving undocumented registers over the internal I2C bus. The order matches ESP-IDF's rtc_clk_cpu_freq_to_pll_mhz(): frequency, divider, source.

read_cpu_mhz() then reads the result back rather than trusting a constant, so the timing can't drift out of sync with the setting the way CPU_MHZ-versus-sdkconfig can in the IDF project. boot_cpu_mhz() keeps the speed the ROM handed over, so a program can print both and see the switch happen: cpu 20 -> 160 MHz.

If you ever hit a boot path where the PLL isn't already running, that switch hangs the CPU. The fallback is the crystal undivided — 40 MHz, marginal but alive:

uint32_t v = ESP32S3_REG(SYSTEM_SYSCLK_CONF_REG);
v &= ~0x3FFu;         ESP32S3_REG(SYSTEM_SYSCLK_CONF_REG) = v;  /* divide by 1  */
v &= ~(0x3u << 10);   ESP32S3_REG(SYSTEM_SYSCLK_CONF_REG) = v;  /* source XTAL  */