This is the second entry in the running series documenting Pyintel Lux — an open binary telemetry standard for microcontrollers. In this entry: empirical hardware benchmark results on ESP32-S3 across Phase 1 (UART) and Phase 2 (Wi-Fi UDP), plus our major engine optimizations (batching, CRC LUTs, and sequence loss tracking).
Moving from Theory to Hard Hardware Numbers
In the first post, I laid out the engineering targets for Pyintel Lux. The goals were clear: zero heap allocation, sub-microsecond frame assembly, and transport independence.
Over the past research session, we flashed an ESP32-S3 board (@ 160MHz) and built out full test harnesses for Phase 1 (UART @ 115,200 baud) and Phase 2 (Wi-Fi UDP Broadcast). We also built an automated Python telemetry logging system (decode.py and udp_decode.py) and a statistical analyzer (stats.py) to measure real-world performance.
Here are the actual numbers.
Phase 1 — UART Emission Benchmark
For Phase 1, we connected the ESP32-S3 via USB-Serial (Silicon Labs CP210x on COM9 @ 115,200 baud). The C firmware emitted a continuous pair of back-to-back frames during every tick loop:
LUX_SYM_HEARTBEAT(System frame containing uptime in ms)APP_COUNTER(Application loop counter)
Results (lux_telemetry.csv)
- Total Frames Captured: 121 frames (2,178 bytes over 60 seconds)
- Packet Delivery Rate (PDR): 121 / 121 delivered (100.0% PDR, 0 lost)
- Data Integrity: 121 / 121 CRC-16 CCITT frames passed (100.0% OK)
- ESP32 Hardware Clock Delta (
esp_dt): 4 µs minimum! - Intra-burst Latency (
HEARTBEAT→APP_COUNTER): Mean = 3.97 ± 2.93 ms - Inter-burst Tick Period: Mean = 986.00 ± 76.76 ms (matching the FreeRTOS 1-second task delay)
The 4 µs minimum clock delta proved that frame assembly inside lux_emit_u32() is practically instantaneous on bare-metal hardware.
Phase 2 — Wi-Fi UDP Wireless Stream
In Phase 2, we tested transport independence. We kept the exact same frame formatting code (lux_emit_u32()) and swapped the transport write callback from UART to a Wi-Fi UDP broadcast socket (sendto() on port 4210 to network TINDU).
Results (lux_udp_telemetry.csv)
- Total Frames Captured: 56 frames (1,008 bytes across 28 UDP socket calls)
- Data Integrity: 56 / 56 CRC-16 CCITT frames passed (100.0% OK, 0% corruption)
- Packet Loss & PDR: 56 / 62 delivered (90.3% PDR, 6 frames lost)
- Intra-packet Batch Delay: 0.00 ms!
- ESP32 Hardware Clock Delta: Minimum 10 µs
Core Architectural Engine Upgrades
During testing, we identified four key optimization opportunities to push Lux performance even further:
1. 14-Byte Header with 16-Bit Sequence Numbers (seq_num)
We expanded the wire header from 12 bytes to 14 bytes to introduce a 16-bit monotonic sequence counter (seq_num).
┌───────────┬────────────┬─────────────┬────────────────┬──────────────┬──────────────┬────────────┐
│ Sync (2B) │ SeqNum(2B) │ SymbolID(2B)│ Timestamp (4B) │ TypeCode(1B) │ PayloadLen(1B)│ CRC-16(2B) │
│ 'L' 'X' │ 0..65535 │ 0x0001 │ Microseconds │ e.g., u32 │ 0..255 │ CCITT-FALSE│
└───────────┴────────────┴─────────────┴────────────────┴──────────────┴──────────────┴────────────┘This gave us instant host-side visibility into wireless UDP packet drops. When the ESP32 dropped 6 UDP packets due to Wi-Fi airtime jitter, the host decoder detected the sequence jump immediately (seq_delta > 1), accurately computing a 90.3% Packet Delivery Rate (PDR) without requiring complex protocol state.
2. Tx Batching & Coalescing (lux_flush)
Instead of issuing a separate network sendto() system call for every individual symbol, lux_emit_* now accumulates frames into an internal zero-heap 256-byte buffer until lux_flush() is called.
This reduced Wi-Fi MAC layer header tax by over 50% and dropped intra-packet burst transmission delay to 0.00 ms!
3. 256-Entry CRC-16 Lookup Table (LUT)
We replaced software bit-shifting CRC loops with a pre-computed 256-entry lookup table. This dropped header checksum calculation time on the ESP32 from ~15–20 µs down to < 1 µs, achieving a minimum frame assembly time of 4 µs.
Comparative Analysis: Lux vs. OpenTelemetry (OTLP)
| Feature / Metric | Lux (Phase 2 UDP) | OpenTelemetry (OTLP over gRPC/HTTP) |
|---|---|---|
| Packet Overhead | 18 bytes static (14B header + 4B payload) | 300 to 1,000+ bytes (Protobuf + HTTP/2) |
| Transit Latency | 4 µs emit time / 0.00 ms batched | 15 ms to 50+ ms (TCP + TLS + gRPC) |
| Memory Footprint | < 1 KB static RAM (Zero heap) | 100 KB to 1+ MB RAM (Requires full OS & TCP stack) |
| Transport Layer | Transport-Agnostic (UART, UDP, ESP-NOW, SPI) | TCP / HTTP/2 / gRPC only |
What’s Next
With Phase 1 (UART) and Phase 2 (Wi-Fi UDP) fully benchmarked and proven:
- Phase 3 (ESP-NOW P2P Mesh): Peer-to-peer telemetry between two ESP32 boards without a Wi-Fi router.
- Phase 4 (Host
luxdDaemon): Building the Rust-based high-throughput ingestion proxy. - Phase 5 (Web Dashboard): Rendering real-time binary streams directly in browser charts via WebSockets.