ProjectMar – Jun 2025
BuddyBack: Federated Learning for Posture Classification
Training BuddyBack’s posture classifier across several devices without sharing anyone’s data. A FedAvg implementation in PyTorch Lightning over pose features small enough to train on a Raspberry Pi, tested with both even and skewed data splits.
BuddyBack is a desk device that watches your posture through a camera. A model that learns from many users would be better than one tuned on a single person, but sending posture data to a server defeats the privacy promise of keeping everything on the device. This project explores federated learning as the way out: each device trains locally, and only model weights travel.
Data
The starting point was a public Roboflow dataset of sitting postures labelled good or bad, about 600 images. Instead of training on pixels, a labelling script runs MediaPipe on each image and keeps four features: neck angle, torso angle, shoulder offset and relative neck angle. That keeps both training and inference light enough for a Raspberry Pi.
With so few samples, augmentation did a lot of the work: Gaussian noise to mimic sensor jitter, small angle variations, correlated noise between neck and torso, and SMOTE to balance the classes.
System
- Model. A small MLP (4 → 64 → 32 → 2) with dropout, written as a PyTorch Lightning module.
- Clients. Each simulated device receives the global weights, trains for a few local epochs on its own data and sends back only the updated weights and its sample count.
- Server. Aggregates with FedAvg, weighting each client by how much data it has, then evaluates the global model on a held-out test set.
- Experiments. Five clients, 30 rounds, run with an IID split and with a skewed (non-IID) split to mimic real users, all logged to TensorBoard.
In the non-IID run the global model reached 99.1% accuracy on the test set, while the clients averaged 91.5% on their own data. Those numbers come from a small, augmented dataset, so they show that the federated loop works rather than how the model would do on new people.
Team
Built with Lorenzo Spataro. The code, including the full walkthrough notebook, is on GitHub.