Depth and thermal image fusion for human detection with occlusion handling under poor illumination from mobile robot

Abstract

In this paper we present a vision-based approach to detect multiple persons with occlusion handling from a mobile robot in real-world scenarios under two lighting conditions, good illumination (lighted) and poor illumination (dark). We use depth and thermal information that are fused for occlusion handling. First, a classifier is trained using thermal images of the human upper-body. This classifier is used to obtain the bounding box coordinates of human. The depth image is later fused with the region of interest obtained from the thermal image. Using the initial bounding box, occlusion handling is performed to determine the final position of human in the image. The proposed method significantly improves human detection even in crowded scene and poor illumination.