EngineeringJuly 22, 2025·2 min read

The device tells us why it crashed.

Companion writes a crash record to NVS flash on its way down, then uploads it on the next boot. A nurse at 3am should never be a debugger.

Companion lives at the bedside for weeks at a time. There is no keyboard, no serial cable, no engineer down the hall. When the ESP32-S3 or CoreS3 firmware panics and reboots, the most expensive thing in the world is asking a nurse at 3am what the screen said before it went black.

A post-mortem that survives the reboot

Just before the panic handler triggers a restart, the firmware writes a small diagnostic record to a dedicated NVS partition. The record is intentionally tiny — a fixed struct, well under a flash page — so the write completes inside the fault context. It carries the `esp_reset_reason`, free internal heap and largest free block, PSRAM allocation state, the depth of each FreeRTOS queue we care about, and the last WebSocket close code and reason string we saw before things went sideways.

Writing to flash from a fault handler is a narrow window. We avoid printf, avoid the heap, and use the low-level NVS API with a pre-opened handle held in a static slot. The panic hook commits the blob, then lets the watchdog or esp_restart finish the job. If the write fails, we still reboot — a missing record is better than a brick.

From silent crash to closed ticket

On the next boot, before the realtime session comes back up, the device checks NVS for a pending record, POSTs it to the CareOS API with the fleet JWT, and clears the slot. The backend groups records by firmware version and stack signature and opens a ticket automatically. The next OTA rolls the fix to every Companion in the field.

The product effect is simple. Facility staff stop being asked to describe what happened — the device already told us. The bedside experience stays a bedside experience, and the engineering work happens where engineers are.

firmwareobservabilityesp32

See it in a wing

30 days. One wing. Your numbers.

Ten Companion units, cellular preconfigured, ready in week one. Weekly outcome reports auto-emailed.

Schedule a 20-minute call →