Gist - Optimizing Segmentation for Decentralized Federated Learning on Tiny Devices

We introduce Gist, a decentralized federated learning framework for tiny microcontrollers. Rather than considering all model parameters as equally important, we let devices get the "gist"of the updates. Our contribution has three pillars: (1) it segments model parameters by their importance, identifying the most impactful updates; (2) it shares the segments probabilistically, ensuring rapid propagation of important knowledge while maintaining model diversity; and (3) it aggregates updates using a success-based scheme, giving more weight to information from better-performing peers. We implement and validate Gist through various simulation experiments, realistic large-scale emulation, and deployment on a physical cluster of ESP32-S3 microcontrollers. Across three models and two tasks, Gist outperforms existing baselines, achieving higher accuracy and faster convergence, especially in larger networks consisting of hundreds of devices.