Federated Learning

TLDR: Federated learning trains a shared model across many devices without moving their data. The data stays local; only model updates are shared.

Federated learning trains a model across decentralized data. The data never leaves its source. Each device trains locally on its own data. Only model updates travel to a central server. The server combines those updates into a shared model. This protects privacy and cuts data transfer.

How Federated Learning Works

1. Distribute the Model: The server sends the current model to each device.
2. Train Locally: Each device trains on its own training data.
3. Send Updates: Devices return model updates, not raw data.
4. Aggregate: The server averages updates into a new global model.
5. Repeat: The cycle continues until the model converges.

Why Use Federated Learning

1. Privacy: Sensitive data stays on the device.
2. Compliance: It helps meet GDPR and HIPAA requirements.
3. Bandwidth: Model updates are far smaller than raw datasets.
4. Personalization: Models adapt to each device’s local data.

Challenges of Federated Learning

1. Non-IID Data: Device data distributions differ widely.
2. Communication Cost: Many update rounds add overhead.
3. Security: Updates can leak information without protection.
4. Device Variability: Phones differ in power and connectivity.

Federated vs Centralized Training

Centralized training pools all data in one place. Federated learning keeps data distributed across devices. Centralized is simpler and often more accurate. Federated wins on privacy and regulatory compliance. It often runs inference at the edge too.

Public Data Still Powers the Base Model

Federated learning protects private, on-device data. But models still need broad public data for pre-training. A strong foundation model starts on web-scale data. Bright Data’s datasets, AI data packages, and Web Scraper supply that public web data at scale.