Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PANDAS: Prototype-based Novel Class Discovery and Detection (2402.17420v2)

Published 27 Feb 2024 in cs.CV and cs.AI

Abstract: Object detectors are typically trained once and for all on a fixed set of classes. However, this closed-world assumption is unrealistic in practice, as new classes will inevitably emerge after the detector is deployed in the wild. In this work, we look at ways to extend a detector trained for a set of base classes so it can i) spot the presence of novel classes, and ii) automatically enrich its repertoire to be able to detect those newly discovered classes together with the base ones. We propose PANDAS, a method for novel class discovery and detection. It discovers clusters representing novel classes from unlabeled data, and represents old and new classes with prototypes. During inference, a distance-based classifier uses these prototypes to assign a label to each detected object instance. The simplicity of our method makes it widely applicable. We experimentally demonstrate the effectiveness of PANDAS on the VOC 2012 and COCO-to-LVIS benchmarks. It performs favorably against the state of the art for this task while being computationally more affordable.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Zero-shot object detection. In Proceedings of the European Conference in Computer Vision (ECCV), pp.  384–400, 2018.
  2. Towards open world recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  1893–1902, 2015.
  3. Towards open set deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  1563–1572, 2016.
  4. Open-world semi-supervised learning. In International Conference on Learning Representations (ICLR), 2022.
  5. Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems (NeurIPS), 33:9912–9924, 2020.
  6. Incremental learning in semantic segmentation from image labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  7. Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  1201–1210, 2015.
  8. The overlooked elephant of object detection: Open set. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp.  1021–1030, 2020.
  9. The pascal visual object classes (voc) challenge. International Journal of Computer Vision (IJCV), 88:303–338, 2010.
  10. A unified objective for novel class discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.  9284–9292, 2021.
  11. Learning to discover and detect objects. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp.  8746–8759, 2022.
  12. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  5356–5364, 2019.
  13. Ow-detr: Open-world detection transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  9235–9244, 2022.
  14. Learning to discover novel visual categories via deep transfer clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  8401–8409, 2019.
  15. Automatically discovering and learning new visual categories with ranking statistics. In International Conference on Learning Representations (ICLR), 2020.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  17. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  9729–9738, 2020.
  18. Deep image category discovery using a transferred similarity function. arXiv preprint arXiv:1612.01253, 2016.
  19. Learning to cluster in order to transfer across domains and tasks. In International Conference on Learning Representations (ICLR), 2018.
  20. Multi-class open set recognition using probability of inclusion. In Proceedings of the European Conference in Computer Vision (ECCV), pp.  393–409. Springer, 2014.
  21. Joint representation learning and novel category discovery on single-and multi-modal data. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.  610–619, 2021.
  22. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547, 2019.
  23. Towards open world object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  5830–5840, 2021.
  24. Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83–97, 1955.
  25. Object-graphs for context-aware category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  1–8. IEEE, 2010.
  26. Learning the easy things first: Self-paced visual category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  1721–1728. IEEE, 2011.
  27. Microsoft coco: Common objects in context. In Advances in Neural Information Processing Systems (NeurIPS), pp.  740–755. Springer, 2014.
  28. Feature pyramid networks for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  2117–2125, 2017.
  29. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24:109–165, 1989.
  30. Dropout sampling for robust object detection in open-set conditions. In IEEE International Conference on Robotics and Automation (ICRA), pp.  3243–3249. IEEE, 2018.
  31. The pursuit of knowledge: Discovering and localizing novel categories using dual memory. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.  9153–9163, 2021.
  32. Most: Multiple object localization with self-supervised transformers for object discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
  33. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems (NeurIPS), 28, 2015.
  34. Toward open set recognition. IEEE TPAMI, 35(7):1757–1772, 2012.
  35. Probability models for open set recognition. IEEE TPAMI, 36(11):2317–2324, 2014.
  36. Unsupervised object localization in the era of self-supervised vits: A survey. arXiv preprint arXiv:2310.12904, 2023.
  37. Novel class discovery: an introduction and key concepts. arXiv preprint arXiv:2302.12028, 2023.
  38. Unsupervised object discovery: A comparison. International Journal of Computer Vision (IJCV), 88:284–302, 2010.
  39. Generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  7492–7501, 2022.
  40. Large-scale unsupervised object discovery. Advances in Neural Information Processing Systems (NeurIPS), 34:16764–16778, 2021.
  41. Freesolo: Learning to segment objects without annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  14176–14186, 2022.
  42. Unsupervised discovery of the long-tail in instance segmentation using hierarchical self-supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  2603–2612, 2021.
  43. UC-OWOD: Unknown-classified open world object detection. In Proceedings of the European Conference in Computer Vision (ECCV), pp.  193–210, 2022.
  44. Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly. IEEE TPAMI, 41(9):2251–2265, 2018.
  45. Novel visual category discovery with dual ranking statistics and mutual knowledge distillation. Advances in Neural Information Processing Systems (NeurIPS), 34:22982–22994, 2021.
  46. Novel class discovery in semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  4340–4349, 2022.
  47. Towards open-set object detection and discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.  3961–3970, 2022.
  48. Neighborhood contrastive learning for novel class discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  10867–10875, 2021a.
  49. Openmix: Reviving known knowledge for discovering novel visual categories in an open world. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  9462–9470, 2021b.
  50. Detecting twenty-thousand classes using image-level supervision. In Proceedings of the European Conference in Computer Vision (ECCV), pp.  350–368. Springer, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Tyler L. Hayes (24 papers)
  2. César R. de Souza (1 paper)
  3. Namil Kim (8 papers)
  4. Jiwon Kim (50 papers)
  5. Riccardo Volpi (30 papers)
  6. Diane Larlus (41 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now