Compositional convolutional neural networks: a robust and interpretable model for object recognition under occlusion

DOI10.1007/S11263-020-01401-3zbMATH Open1483.68433arXiv2006.15538OpenAlexW3107828565MaRDI QIDQ2056441FDOQ2056441

Authors: Yanyan Li 

Publication date: 2 December 2021

Published in: International Journal of Computer Vision (Search for Journal in Brave)

Abstract: Computer vision systems in real-world applications need to be robust to partial occlusion while also being explainable. In this work, we show that black-box deep convolutional neural networks (DCNNs) have only limited robustness to partial occlusion. We overcome these limitations by unifying DCNNs with part-based models into Compositional Convolutional Neural Networks (CompositionalNets) - an interpretable deep architecture with innate robustness to partial occlusion. Specifically, we propose to replace the fully connected classification head of DCNNs with a differentiable compositional model that can be trained end-to-end. The structure of the compositional model enables CompositionalNets to decompose images into objects and context, as well as to further decompose object representations in terms of individual parts and the objects' pose. The generative nature of our compositional model enables it to localize occluders and to recognize objects based on their non-occluded parts. We conduct extensive experiments in terms of image classification and object detection on images of artificially occluded objects from the PASCAL3D+ and ImageNet dataset, and real images of partially occluded vehicles from the MS-COCO dataset. Our experiments show that CompositionalNets made from several popular DCNN backbones (VGG-16, ResNet50, ResNext) improve by a large margin over their non-compositional counterparts at classifying and detecting partially occluded objects. Furthermore, they can localize occluders accurately despite being trained with class-level supervision only. Finally, we demonstrate that CompositionalNets provide human interpretable predictions as their individual components can be understood as detecting parts and estimating an objects' viewpoint.

Full work available at URL: https://arxiv.org/abs/2006.15538

Recommendations

• Visual concepts and compositional voting
• Partially occluded object recognition using statistical models
• Localized Scene Interpretation from 3D Models, Range, and Optical Data
• Recovering occlusion boundaries from an image
• Recursive compositional models for vision: description and review of recent work

zbMATH Keywords

image classificationobject detectioncompositional modelsout-of-distribution generalizationrobustness to partial occlusion

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Machine vision and scene understanding (68T45)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• AND/OR search spaces for graphical models
• Visual concepts and compositional voting

Cited In (2)

• \textsc{OccRob}: efficient SMT-based occlusion robustness verification of deep neural networks
• Understanding the role of pathways in a deep neural network

Uses Software

• ImageNet
• k-means++
• Adam
• MS-COCO
• AlexNet
• AutoAugment
• Faster R-CNN

This page was built for publication: Compositional convolutional neural networks: a robust and interpretable model for object recognition under occlusion

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2056441)