\begin{abstract}
Structure-from-Motion (SfM) is a cornerstone of 3D perception, yet current methods often fail when applied to complex videos involving challenging camera motions or dynamic scenes.
Compounding the problem, the field lacks reliable ground-truth benchmarks for such difficult scenarios, making it hard to gauge real-world progress, or pinpoint where improvements are most needed.
To address this gap, we introduce a new benchmark for evaluating camera pose estimation.
Our key insight is to leverage online panoramic 360° as a source of data from which to construct challenging clips, while still enabling robust ground-truth trajectory recovery.
The panoramic nature of these videos provides richer visual context for tracking camera motion, even when parts of the view are affected by blur, motion, or dynamic objects.
By tracking camera motion across full 360° videos, we crop and reproject selected portions to generate perspective-view clips that serve as our benchmark---ORBIT---a diverse collection of 100 video clips.
% of varying lengths.
Experiments show that COLMAP and other state-of-the-art SfM methods struggle to accurately estimate camera positions on our benchmark, indicating that it remains a challenging and open problem space for future research.
As a result, ORBIT provides a valuable testbed where researchers can meaningfully compete and measure progress on truly challenging, real-world SfM problems.
\end{abstract}

% --- previous version
\endinput

\begin{abstract}
Structure-from-Motion (SfM) is an essential ingredient for 3D perception, but current methods often fail on complex videos featuring difficult camera motion and dynamic objects.
Worse, the community lacks proper ground truth benchmarks to even measure performance on such challenging videos, making it unclear whether our field is making progress on real-world data and difficult to identify areas where improvements are needed.
We address this gap through a new benchmark for evaluating camera pose estimation.
Our key idea is to use online panoramic 360$^\circ$ videos as a data source for finding challenging clips where we can still robustly determine ground truth camera trajectories. 
The 360$^\circ$ nature of the raw video yields much more visual information for use in tracking camera motion, even if parts of the field of view are difficult for SfM methods (e.g., due to blurry content, moving objects, etc).
We use full videos to track camera motion, and crop and reproject portions of these to create the challenging perspective videos that form our benchmark, called  ORBIT, comprising 100 video clips of varying length. 
We show that COLMAP and other state-of-the-art SfM methods struggle to estimate camera positions on our benchmark. 
While on the currently available benchmarks many SOTA methods outperform simpler methods such as COLMAP, our benchmark can better highlight  advantages and disadvantages of these methods.
\end{abstract}

\endinput

\begin{abstract}
Structure-from-Motion (SfM) is an essential ingredient for 3D perception, but current methods often fail on complex videos featuring difficult camera motion and dynamic objects.
% , which are common in the real world. 
Worse, the community lacks proper ground truth benchmarks to even measure performance on such challenging videos, making it unclear whether our field is making progress on real-world data and difficult to identify areas where improvements are needed.
% Having real world challenging scenarios with both camera movement and object movement in video accompanied by ground truth trajectories is currently missing and crucial for developing methods and models dependant on camera controls such as structure from motion (SFM) methods. 
We address this gap through a new benchmark for evaluating camera pose estimation.
Our key idea is to use online panoramic 360$^\circ$ videos as a data source for finding challenging clips where we can still robustly determine ground truth camera trajectories. 
% Our premise is in most scenarios exists a reliable view that is easy to track and calibrate camera for. 
The 360$^\circ$ nature of the raw video yields much more visual information for use in tracking camera motion, even if parts of the field of view are difficult for SfM methods (e.g., due to blurry content, moving objects, etc).
We use full videos to track camera motion, and crop and reproject portions of these to create the challenging perspective videos that form our benchmark, called  \benchmark, comprising 
% Furthermore, we can use non overlapping views to cross validate the trajectories and increase the robustness of ground truth to dynamic objects. 
% \benchmark consists of 
100 video clips of varying length. 
We show that COLMAP and other state-of-the-art SfM methods struggle to estimate camera positions on our benchmark. 
While on the currently available benchmarks many SOTA methods outperform simpler methods such as COLMAP, our benchmark can better highlight  advantages and disadvantages of these methods.
\end{abstract}