Joint Vehicle Pose and Extent Estimation in the Context of Multi-Camera Traffic Surveillance

Leah Strand1, Jens Honer2, Alois C. Knoll1
1Technical University of Munich 2Software Products and Services Product Line (DSW), Valeo Schalter und Sensoren GmbH

Motivation

 

The objective of our work is to fuse the data from a multi-camera traffic surveillance system with the task of localizing passing cars.



 

The inherent difficulty in accurately estimating the real-world object state from a two-dimensional representation of the scene poses a significant challenge. We present a novel measurement model that compares the measured and the expected spatial expansion of the object within the image frame to estimate its pose and extents.

Abstract

In this paper, we introduce a novel method for the estimation of vehicle pose and extent in traffic surveillance scenarios based on camera data. The state estimation is performed in a common world frame, enabling the seamless integration of the image data from different viewpoints. Our approach incorporates the non-linear transformation between the measurements and the states directly into the framework of an Unscented Kalman filter. Two measurement models are proposed: one designed for bounding boxes and another for discretized object contours extracted from segmentation masks. The method is evaluated using data from a real-world traffic surveillance system, demonstrating the high effectiveness and good feasibility of our approach for localizing passing cars.

Overview

 

Illustration of the proposed method which combines deep-learning for extracting the observation with filtering of the object state in the world frame. Two measurement models are presented: one tailored to bounding boxes (a) and one to discretized contours (b).

Evaluation

 

For the evaluation, we recorded five datasets by equipping different vehicles with a high-precision RTK-GPS device and recording in total 39 trajectories at the intersection.

Exemplary Visualized Results

 
 
 
 
 
 

Funded by

as part of the research project VIDETEC-2