OCL: Object Concept Learning

MVIG-RHOS, SJTU

demo
Understanding objects is a central building block of artificial intelligence, especially for embodied AI. Even though object recognition excels with deep learning, current machines still struggle to learn higher-level knowledge, e.g., what attributes does an object have, what can we do with an object. In this work, we propose a challenging Object Concept Learning (OCL) task to push the envelope of object understanding. It requires machines to reason out object affordances and simultaneously give the reason: what attributes make an object possesses these affordances. To support OCL, we build a densely annotated knowledge base including extensive labels for three levels of object concept: categories, attributes, and affordances, together with their causal relations. By analyzing the causal structure of OCL, we present a strong baseline, Object Concept Reasoning Network (OCRN). It leverages causal intervention and concept instantiation to infer the three levels following their causal relations.

Demo

Left-top: object (in yellow box)

Right-top: key causal graph

Left-bottom: affordance prediction score

Right-bottom: key causal relations

News and Olds

[2022.12] Our paper is available on arXiv.
[2022.11] Trail run

Download

Our data (image source and annotations) and code will come very coon!

Publications

Before using our data and code in your project, please cite:
@ARTICLE{ocl,
  title={Beyond Object Recognition: A New Benchmark towards Object Concept Learning},
  author={Li, Yong-Lu and Xu, Yue and Xu, Xinyu and Mao, Xiaohan 
          and Yao, Yuan and Liu, Siqi and Lu, Cewu},
  journal={arXiv preprint arXiv:2212.02710},
  year={2022},
}

Disclaimer

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
In our database, 75,578 images and their anntations are extracted from existing datasets (COCOa, ImageNet-150K, aPY, SUN). 4,885 images are from internet. We only provide image links for research purposes.

© Copyright 2022 MVIG-RHOS • Based on tbakerx