OmniDS: Dual-Stream Context Fusion for
Omnidirectional Depth from Fisheye Cameras
ECCV 2026
- Chaesong Park* Seoul National University
- Jihyeon Hwang* Seoul National University
- Muyeol Sung Seoul National University
- Jongwoo Lim Seoul National University
* Equal contribution.
Abstract
Omnidirectional depth estimation from multi-fisheye camera rigs is complicated by visibility conflicts: wide baselines cause different cameras to observe different portions, or even different faces, of the same object, so aggregating their features into a unified equirectangular (ERP) representation under fixed projection produces ambiguous matching evidence near occlusion boundaries and thin structures. Although existing methods mitigate this by down-weighting unreliable views, they do not resolve the underlying discrepancy because context formation and cross-view fusion remain tied to rigid fisheye-to-ERP sampling. We present OmniDS, an iterative depth refinement framework that replaces rigid aggregation by combining dynamic context fusion with consensus-aware multi-view similarity. A dual-stream encoder pairs a lightweight CNN for geometric detail with a frozen DINOv3 for semantic priors; their features are reprojected into ERP space at each refinement step via learned view weighting and deformable cross-attention with geometric distortion bias. In parallel, a multi-view consensus volume captures global cross-camera agreement through group-wise correlation and feature variance, regularized by a 3D U-Net. For efficient deployment, we distill the dual-stream representation into a single MobileNet-based encoder. OmniDS achieves state-of-the-art performance on the OmniThings, OmniHouse, and Sunny benchmarks while maintaining competitive inference speed.
Motivation
(a) A wide-baseline multi-fisheye rig induces strong parallax and self-occlusion: different cameras observe different visible faces of the same nearby object. (b) Top: ground-truth depth. Below: four per-camera renderings obtained by projecting each fisheye image into a common rig-centric ERP coordinate system using the GT depth. Even with perfect geometry, ERP-aligned samples can be inconsistent due to visibility conflicts (occlusion leakage) and self-occlusion (different surface faces mapped to the same ERP cell). (c) Top: ground-truth depth. Middle: the ERP context produced by our dynamic ERP context fusion. Bottom: a baseline ERP context built by simple grid sampling after fisheye-to-ERP projection. Our fusion yields a more coherent context around thin structures and occlusion boundaries.
Method
Results
Quantitative Comparison
Quantitative comparison on OmniThings and OmniHouse datasets. Bold: best, underline: second best.
| Method | OmniThings | OmniHouse | Time (ms) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| >1 | >3 | >5 | MAE | RMS | >1 | >3 | >5 | MAE | RMS | ||
| Trained on OmniThings only | |||||||||||
| OmniMVS | 47.72 | 15.12 | 8.91 | 2.40 | 5.27 | 30.53 | 10.29 | 6.27 | 1.72 | 4.05 | 289 |
| OmniMVS+32 | 20.70 | 8.18 | 5.49 | 1.37 | 4.11 | 19.89 | 5.89 | 3.99 | 1.30 | 2.64 | 289 |
| S-OmniMVS | 28.03 | 10.40 | 6.33 | 1.48 | 3.68 | 18.86 | 8.05 | 4.90 | 1.06 | 2.41 | -- |
| RomniStereo32 | 20.42 | 8.49 | 5.81 | 1.39 | 4.22 | 12.13 | 4.73 | 3.02 | 0.80 | 1.85 | 83 |
| RomniStereo64 | 17.77 | 7.52 | 5.00 | 1.22 | 3.90 | 10.52 | 4.05 | 2.69 | 0.74 | 1.73 | 161 |
| MDP-Omni | 18.37 | 7.09 | 4.59 | 1.20 | 3.79 | 17.13 | 7.20 | 4.68 | 1.16 | 2.62 | 117 |
| Ours | 14.89 | 5.94 | 3.96 | 1.04 | 3.59 | 9.42 | 3.28 | 2.01 | 0.64 | 1.61 | 148 |
| Fine-tuned on OmniHouse and Sunny | |||||||||||
| OmniMVS-ft | 50.28 | 22.78 | 15.60 | 3.52 | 7.44 | 21.09 | 4.63 | 2.58 | 1.04 | 1.97 | 289 |
| OmniMVS+32-ft | 44.79 | 27.17 | 20.41 | 4.23 | 8.42 | 9.70 | 3.51 | 2.13 | 0.64 | 1.69 | 289 |
| S-OmniMVS-ft | -- | -- | -- | -- | -- | 6.99 | 1.79 | 0.97 | 0.42 | 1.06 | -- |
| RomniStereo32-ft | 34.32 | 19.76 | 14.22 | 2.81 | 6.47 | 6.02 | 2.49 | 1.73 | 0.49 | 1.31 | 83 |
| RomniStereo64-ft | 29.84 | 16.21 | 11.28 | 2.26 | 5.60 | 5.28 | 2.22 | 1.51 | 0.42 | 1.14 | 161 |
| MDP-Omni-ft | 29.53 | 14.05 | 9.12 | 1.96 | 5.08 | 5.61 | 2.19 | 1.50 | 0.44 | 1.15 | 117 |
| Ours-ft | 25.14 | 11.72 | 7.67 | 1.70 | 4.69 | 3.92 | 1.17 | 0.64 | 0.28 | 0.89 | 148 |
| Ours-ft (distilled) | 26.22 | 13.69 | 9.51 | 2.11 | 5.49 | 3.41 | 1.16 | 0.73 | 0.28 | 0.89 | 102 |
Synthetic Data (with Ground Truth)
Omnidirectional depth prediction on synthetic sequences where ground-truth depth is available for comparison.
Real-World Data
Omnidirectional depth prediction on a real-world dataset.
Citation
Acknowledgements
We thank the research community for providing the benchmarks and datasets used in this research. This work was supported by ME & IPAI at Seoul National University.
The website template was borrowed from Michaël Gharbi.