Rate-Distortion in Efficient Multi-Agent Perception: A Unified Framework for Communication and Memory Optimization

Information-theoretic framework for communication-efficient multi-agent perception under bandwidth constraints

Overview

This project investigates efficient multi-agent perception from an information-theoretic perspective, focusing on the trade-off between communication bandwidth and perception performance.

The goal is to develop a unified framework that explains and guides the design of communication-efficient perception systems under resource constraints.

This work is ongoing and planned for submission to NeurIPS 2026.

Problem Motivation

In collaborative perception systems, agents need to exchange information to build a consistent global understanding of the environment.

However:

  • communication bandwidth is limited
  • redundant information is frequently transmitted
  • existing methods lack a principled way to balance efficiency and performance

This project studies how to formally model:

what information should be transmitted, and how much is sufficient

Core Perspective

The problem is formulated through a Rate–Distortion framework:

  • Rate → communication cost (bandwidth)
  • Distortion → perception quality degradation

The objective is to understand and optimize the trade-off between:

  • compact representations
  • information completeness
  • perception accuracy

Key Ideas

1. Information-Aware Representation

Instead of treating features as raw tensors, the system models representations as information carriers with varying importance.

2. Selective Information Transmission

Only a subset of information is transmitted across agents based on:

  • relevance to the task
  • redundancy across agents
  • temporal novelty

3. Communication–Memory Trade-off

The framework jointly considers:

  • communication (inter-agent information exchange)
  • memory (temporal information reuse)

to reduce redundant transmission and improve efficiency.

System-Level Implications

The theoretical framework provides guidance for:

  • designing communication-efficient token representations
  • selecting informative features under bandwidth constraints
  • balancing real-time communication and temporal memory

It also connects naturally with practical system designs for:

  • collaborative occupancy prediction
  • multi-agent perception pipelines

Experimental Observations

Preliminary results indicate that:

  • significant reduction in communication cost can be achieved
  • perception performance remains stable under constrained bandwidth
  • an effective rate–distortion trade-off emerges in practice

Research Significance

This work aims to:

  • provide a principled foundation for communication-efficient perception
  • bridge information theory and deep learning systems
  • support scalable multi-agent perception and world model learning

Note

Detailed formulations and implementations are omitted due to ongoing submission.