Spotflow

𝗬𝗼𝘂𝗿 𝘁𝗲𝗹𝗲𝗺𝗲𝘁𝗿𝘆 𝗽𝗮𝘆𝗹𝗼𝗮𝗱 𝘀𝗶𝘇𝗲 𝗯𝗮𝗿𝗲𝗹𝘆 𝗮𝗳𝗳𝗲𝗰𝘁𝘀 𝗯𝗮𝘁𝘁𝗲𝗿𝘆 𝗹𝗶𝗳𝗲. Your reporting interval dominates it, by an order of magnitude. Measured by a Siemens and Inria team, not by a vendor. 5 bytes versus 90 bytes came out about 10 microamps apart on a one hop node. Both fit in one 802.15.4 frame. Same wake, same transmission, same ACK. 𝗬𝗼𝘂 𝗮𝗹𝗿𝗲𝗮𝗱𝘆 𝗽𝗮𝗶𝗱 𝗳𝗼𝗿 𝘁𝗵𝗲 𝗳𝗿𝗮𝗺𝗲, 𝘀𝗼 𝘁𝗵𝗲 𝗯𝘆𝘁𝗲𝘀 𝗶𝗻𝘀𝗶𝗱𝗲 𝗮𝗿𝗲 𝗻𝗲𝗮𝗿𝗹𝘆 𝗳𝗿𝗲𝗲. Batch until the frame is full before you touch compression. Source: Graf, Watteyne and Villnow, ICT Express 2024. #embedded

“𝗪𝗲 𝗯𝘂𝗳𝗳𝗲𝗿 𝗲𝘃𝗲𝗿𝘆𝘁𝗵𝗶𝗻𝗴 𝗮𝗻𝗱 𝘀𝗲𝗻𝗱 𝗶𝘁 𝘄𝗵𝗲𝗻 𝘄𝗲 𝗿𝗲𝗰𝗼𝗻𝗻𝗲𝗰𝘁.” That works right up until the buffer fills. Then something gets dropped. And if you never decided what, the firmware decides for you. Usually the thing you needed. 𝗗𝗲𝗰𝗶𝗱𝗲 𝘄𝗵𝗮𝘁 𝘁𝗼 𝗱𝗿𝗼𝗽 𝗯𝗲𝗳𝗼𝗿𝗲 𝘁𝗵𝗲 𝗯𝘂𝗳𝗳𝗲𝗿 𝗱𝗲𝗰𝗶𝗱𝗲𝘀 𝗳𝗼𝗿 𝘆𝗼𝘂. Rank your signals before you ship: • Crash dumps first. Metrics second. Logs last. • A lost log line costs nothing. A lost dump chunk costs the whole dump. • Crash dumps persist to flash and survive the reboot. • Logs live in a small ring buffer, and the oldest go first. Deduplicate at the source: • The same crash reason, a thousand times, is not data. • Send it once and count the rest. Timestamp on the device, not on arrival: • Otherwise everything in the backlog looks like it happened at reconnect. This is a design decision, not a config option. You make it before the fleet ships. CTO reality check. Where does yours break first? a) buffer fills and drops silently b) crash dumps lost on reboot c) same crash sent a thousand times d) no idea, we have never tested a long outage #embedded #observability #iot

𝗛𝗼𝘄 𝗺𝗮𝗻𝘆 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 𝘀𝗵𝗼𝘂𝗹𝗱 𝗮 𝗱𝗲𝘃𝗶𝗰𝗲 𝗰𝗼𝗹𝗹𝗲𝗰𝘁? On a constrained part, this is not a philosophy question. It is math. The registry is statically allocated. You pick the size at compile time and pay for the allocation, used or not. We set ours to 12 on a board running at 99% of 36 KB RAM. The metrics: boot count, battery voltage, radio on percentage, sensor error streak, sensor errors by kind, shakes detected, peak g, link disconnects by reason, reset reason, gateway connected, heap free, heap allocated. If we raise the number, we would be taking RAM from the product. #embedded #observability

The first session of “From Crash to Fixed Firmware: Live Remote Debugging of a Zephyr Device” is behind us. Thank you to everyone who joined, and for the great questions in the chat and the Q&A. Mohammed Billoo and Jan Kucera took a Zephyr device from a live crash to its root cause. After the reboot, the core dump reached Spotflow, and the AI analysis pointed to the exact function and suggested a fix. Missed it? Session 2 is on Wed, Oct 21, 3PM CEST / 9AM ET / 6AM PT. Bring your questions. Join the event: https://lnkd.in/djTSRTxF #ZephyrRTOS #EmbeddedSystems #Firmware #Debugging MAB Labs Embedded Solutions

𝗧𝗵𝗿𝗲𝗲 𝘄𝗲𝗯𝗶𝗻𝗮𝗿𝘀. 𝗧𝗵𝗿𝗲𝗲 𝗶𝗻𝗰𝗶𝗱𝗲𝗻𝘁𝘀. 𝗕𝗮𝗱 𝘂𝗽𝘁𝗶𝗺𝗲 😀. 1 - Platform overloaded. Response times through the floor. Live, in front of everyone. 2 - A cleaning lady came in mid-session. I muted myself to ask her to leave and forgot to unmute for several minutes. 3 - Power went out in the building. Laptop fine, wifi gone. Switched to mobile hotspot. Mobile network went down too. #embedded #observability #downtime

"𝗜𝘁 𝘀𝘁𝗼𝗽𝗽𝗲𝗱 𝘄𝗼𝗿𝗸𝗶𝗻𝗴." 𝗧𝗵𝗮𝘁 𝘄𝗮𝘀 𝘁𝗵𝗲 𝗲𝗻𝘁𝗶𝗿𝗲 𝗯𝘂𝗴 𝗿𝗲𝗽𝗼𝗿𝘁. Three real fleets, all on Zephyr, each went dark in a different way: 200,000 BLE-mesh door locks, 1,500 cellular street-lighting controllers, 40,000+ gas detectors. Each team started from a single one-line report ("it stopped working," "lost power, reconnected," "sensor read timeout") with no way to see what was actually wrong. In the recording of our latest webinar, Spotflow co-founder Jan Kucera walks through what finally made each root cause visible: a metric that counted work instead of liveness, a five-month trend plus one log line fetched on demand, and a core dump with the stack still on it. Watch the recording: https://lnkd.in/eDHgduc5 #Firmware #EmbeddedSystems #IoT #Observability #ZephyrRTOS