Distributed Radiance Fields for Edge Video Compression and Metaverse Integration in Autonomous Driving

Distributed Radiance Fields for Enhanced Video Compression in Autonomous Mobility

Introduction to Radiance Fields and Edge Computing

Connected autonomous vehicles (CAVs) have become increasingly common, generating substantial data from various sensors. This data, essential for real-time decision-making and immersive experiences in the metaverse, poses significant challenges regarding network congestion, costs, and latency. Multi-access edge computing (MEC) offers a solution by offloading data and computational tasks to edge servers. However, even with MEC, the demand for advanced data compression techniques is undeniable. The paper introduces a novel approach leveraging distributed radiance fields (RFs) for video compression and metaverse digital twins (DTs) updates, significantly reducing data transmission requirements while maintaining high-quality image reconstructions.

Advances in Video Compression

The traditional video compression schemes are dependent on optical flow methods, which have limitations, especially in dynamic environments seen by CAVs. In contrast, distributed RFs offer a structured understanding of the 3D scene, allowing for more efficient data compression. The paper suggests an RF-based encoder and decoder methodology that uses a sparse set of 2D images to reconstruct 3D scenes and subsequently compress video data, offering potential data savings of up to 80% compared to conventional methods like the H.264 codec, without compromising the quality metrics such as peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).

Neural Radiance Fields (NeRF)

NeRFs encode 3D scenes into a neural network, representing scenes in a compact form and allowing for the reconstruction of any camera view within the scene. By training on a set of 2D images, NeRFs can reproduce complex scenes with high fidelity. The paper utilizes Radiance Fields for encoding and decoding video frames, achieving significant compression by eliminating the need for transmitting complete frame information over the network.

Methodology and Results

The proposed methodology employs RF-based encoding and decoding, where the encoder uses camera poses to render and encode scene differences, and the decoder reconstructs the original image using the encoded differences and stored RF. This approach was validated using the CARLA simulator, showing significant compression savings across various scenarios without sacrificing image quality. The experimental results indicated that the RF-based method achieved substantial data savings and high-quality reconstructions compared to traditional methods.

Implications and Future Directions

The research presents a significant advancement in data compression techniques, with profound implications for the future of autonomous mobility and metaverse applications. By decreasing the data transmission requirements, the approach facilitates more efficient and scalable deployment of CAVs and immersive metaverse experiences. Looking ahead, integrating RF-based video compression with existing edge computing frameworks could further enhance the performance of autonomous systems and metaverse platforms, offering real-time updates with minimal latency and high fidelity.

Conclusion

This study introduces a pioneering approach for video compression and metaverse updates in the context of autonomous driving, using distributed RFs for efficient data encoding. The methodology shows promising results for reducing network congestion and enabling rapid, high-quality updates for DTs in the metaverse, setting the stage for future innovations in autonomous mobility and digital twin technologies.