Home About Research Publications Lectures Teaching Academic Collaborators Personalia
Topics
Projects







ShapeGoogle: Feature-based shape analysis

Bag of features representation of non-rigid shapes
Representation of shapes as bags of features (geometric words) invariant to non-rigid transformations.

The availability of large public-domain databases of 3D models such as Google 3D Warehouse has created the demand for shape search and retrieval algorithms capable of finding similar shapes in the same way a search engine responds to text quires. However, while text search methods are sufficiently good to be ubiquitously use e.g. in Web application, the search and retrieval of 3D shapes is a much more challenging problem. Shape retrieval based on text metadata (annotations and tags added by human) is often insufficient to provide the same experience as a text search engine does. Content-based shape retrieval using the shape itself as a query and based on comparison of shape properties is complicated by the fact that many 3D objects manifest rich variability, and invariance to different classes of transformations and shape variations is required. One of the most challenging settings addressed in this paper is the case of non-rigid or deformable shapes, in which the class of transformations may be very wide due to the capability of such shapes to bend and assume different forms.

In the image domain, an analogous problem is image retrieval, the problem of finding images depicting similar scenes or objects. Images, as well as three-dimensional shapes, may manifest significant variability. The computer vision and pattern recognition communities have recently witnessed a wide adoption of feature-based methods in object recognition and image retrieval applications. These methods allow to represent images as collections of "visual words" and treat them using text search approaches such as the "bag of features" paradigm. Feature-based methods are popular in computer vision and pattern recognition communities, in applications such as object recognition and image retrieval.

We explored analogous approaches in the 3D world applied to the problem of non-rigid shape retrieval in large databases. Using multiscale diffusion heat kernels as "geometric words", we construct shape descriptors by means of the "bag of features" approach. We also show that considering pairs of spatially-close "geometric words" ("geometric expressions") allows to create spatially-sensitive bags of features with better discriminativity.

Papers

  • I. Kokkinos, M. M. Bronstein, R. Litman, A. M. Bronstein, "Intrinsic shape context descriptors for deformable shapes", Proc. Computer Vision and Pattern Recognition (CVPR), 2012.

  • A. Kovnatsky, D. Raviv, M. M. Bronstein, A. M. Bronstein, R. Kimmel "Affine-invariant photometric heat kernel signatures", Proc. Workshop on 3D Object Retrieval (3DOR), 2012.

  • R. Litman, A. M. Bronstein, M. M. Bronstein, "Stable volumetric features in deformable shapes", Computers & Graphics, Vol. 36/3, 2012.

  • R. Litman, A. M. Bronstein, M. M. Bronstein, "Diffusion-geometric maximally stable component detection in deformable shapes", Computers & Graphics, Vol. 35/3, pp. 549-560, June 2011.

  • A. Kovnatsky, M. M. Bronstein, A. M. Bronstein, R. Kimmel, "Photometric heat kernel signatures", Proc. Conf. on Scale Space and Variational Methods in Computer Vision (SSVM), 2011.

  • E. Boyer, A. M. Bronstein, M. M. Bronstein, B. Bustos, T. Darom, R. Horaud, I. Hotz, Y. Keller, J. Keustermans, A. Kovnatsky, R. Litman, J. Reininghaus, I. Sipiran, D. Smeets, P. Suetens, D. Vandermeulen, A. Zaharescu, V. Zobel, "SHREC 2011: robust feature detection and description benchmark", Proc. EUROGRAPHICS Workshop on 3D Object Retrieval (3DOR), 2011.

  • A. M. Bronstein, M. M. Bronstein, M. Ovsjanikov, L. J. Guibas, "Shape Google: geometric words and expressions for invariant shape retrieval", ACM Trans. Graphics (TOG), Vol. 30/1, pp. 1-20, January 2011.

  • D. Raviv, M. M. Bronstein, A. M. Bronstein, R. Kimmel, "Volumetric heat kernel signatures", Proc. Intl. Workshop on 3D Object Retrieval, ACM Multimedia, 2010.

  • A. M. Bronstein, M. M. Bronstein, U. Castellani, B. Falcidieno, A. Fusiello, A. Godil, L. J. Guibas, I. Kokkinos, Z. Lian, M. Ovsjanikov, G. Patané, M. Spagnuolo, R. Toldo, "SHREC 2010: robust large-scale shape retrieval benchmark", Proc. EUROGRAPHICS Workshop on 3D Object Retrieval (3DOR), 2010.

  • A. M. Bronstein, M. M. Bronstein, B. Bustos, U. Castellani, M. Crisani, B. Falcidieno, L. J. Guibas, I. Kokkinos, V. Murino, M. Ovsjanikov, G. Patané, I. Sipiran, M. Spagnuolo, J. Sun, "SHREC 2010: robust feature detection and description benchmark", Proc. EUROGRAPHICS Workshop on 3D Object Retrieval (3DOR), 2010.

  • M. M. Bronstein, I. Kokkinos, "Scale-invariant heat kernel signatures for non-rigid shape recognition", Proc. Computer Vision and Pattern Recognition (CVPR), 2010.

  • M. Ovsjanikov, A. M. Bronstein, M. M. Bronstein, L. J. Guibas, "ShapeGoogle: a computer vision approach for invariant shape retrieval", Proc. Workshop on Nonrigid Shape Analysis and Deformable Image Alignment (NORDIA), 2009.

    • See also

  • Scale-invariant heat kernel signatures (CVPR trailer video)

  • Non-rigid shape similarity and correspondence

  • Partial similarity and correspondence