Publications
262 results found
Gasparetto A, Cosmo L, Rodola E, et al., 2018, Spatial Maps: From low rank spectral to sparse spatial functional representations, Pages: 477-485
Functional representation is a well-established approach to represent dense correspondences between deformable shapes. The approach provides an efficient low rank representation of a continuous mapping between two shapes, however under that framework the correspondences are only intrinsically captured, which implies that the induced map is not guaranteed to map the whole surface, much less to form a continuous mapping. In this work, we define a novel approach to the computation of a continuous bijective map between two surfaces moving from the low rank spectral representation to a sparse spatial representation. Key to this is the observation that continuity and smoothness of the optimal map induces structure both on the spectral and the spatial domain, the former providing effective low rank approximations, while the latter exhibiting strong sparsity and locality that can be used in the solution of large-scale problems. We cast our approach in terms of the functional transfer through a fuzzy map between shapes satisfying infinitesimal mass transportation at each point. The result is that, not only the spatial map induces a sub-vertex correspondence between the surfaces, but also the transportation of the whole surface, and thus the bijectivity of the induced map is assured. The performance of the proposed method is assessed on several popular benchmarks.
Nogneng D, Melzi S, Rodolá E, et al., 2018, Improved functional mappings via product preservation, Computer Graphics Forum: the international journal of the Eurographics Association, Vol: 37, Pages: 179-190, ISSN: 0167-7055
In this paper, we consider the problem of information transfer across shapes and propose an extension to the widely used functional map representation. Our main observation is that in addition to the vector space structure of the functional spaces, which has been heavily exploited in the functional map framework, the functional algebra (i.e., the ability to take pointwise products of functions) can significantly extend the power of this framework. Equipped with this observation, we show how to improve one of the key applications of functional maps, namely transferring real-valued functions without conversion to point-to-point correspondences. We demonstrate through extensive experiments that by decomposing a given function into a linear combination consisting not only of basis functions but also of their pointwise products, both the representation power and the quality of the function transfer can be improved significantly. Our modification, while computationally simple, allows us to achieve higher transfer accuracy while keeping the size of the basis and the functional map fixed. We also analyze the computational complexity of optimally representing functions through linear combinations of products in a given basis and prove NP-completeness in some general cases. Finally, we argue that the use of function products can have a wide-reaching effect in extending the power of functional maps in a variety of applications, in particular by enabling the transfer of highfrequency functions without changing the representation size or complexity.
Svoboda J, Bronstein M, 2018, New achievements in 3D hand shape recognition, Hand-Based Biometrics: Methods and Technology, Pages: 309-336, ISBN: 9781785612244
With the recent developments in low-cost 3D sensing, the interest in employing 3D biometrics has increased. We have wrapped up the current state of the art in 3D hand biometric recognition. Depending on the representation of the acquired geometry, we are offered different feature extraction methods. Typically, the hand scan is stored as range map. The neighborhood structure is well defined on such surfaces, and it is therefore easy to compute approximate geodesic distances as well as perform curvature analysis. Those are, in fact, the paths that most of the researchers have followed in the past. The features are typically stored as n-D vectors and are well suited to be compared using a specific distance metric. As shown in the article, in order to find the best separation of the subjects, dimensionality reduction and metric learning can be employed to exploit the information better. To encourage future development, we show some unconventional representations of the hand geometry as well. Independently, the field of hand tracking advanced very fast, currently offering fast and precise 3D hand-tracking models. Employing such models in 3D hand biometric recognition is very promising, as it could potentially boost the precision of the hand annotation that would naturally result in better stability and performance of the 3D hand recognition. As the hand-tracking models usually work in real time, new possibilities (e.g., continuous user verification, etc.) come up as well. Chapter Contents: • 13.1 Data acquisition • 13.1.1 Methods • 13.1.1.1 Passive stereo • 13.1.1.2 Structured light • 13.1.1.3 Time-of-flight • 13.1.2 Devices • 13.1.3 RGB-D data refinement • 13.1.4 Acquisition setup • 13.1.4.1 Constrained acquisition • 13.1.4.2 Less-constrained acquisition • 13.1.4.3 Toward unconstrained acquisition • 13.2 Preprocessing • 13.2.1 Input smoothing • 13.2.2 Hand segmentation • 13.2.3 Hand annotation &b
Rodolà E, Lähner Z, Bronstein AM, et al., 2018, Functional Maps on Product Manifolds, Pages: 9-10, ISSN: 1727-8384
We consider the tasks of representing, analyzing and manipulating maps between shapes. We model maps as densities over the product manifold of the input shapes; these densities can be treated as scalar functions and therefore are manipulable using the language of signal processing on manifolds. Being a manifold itself, the product space endows the set of maps with a geometry of its own, which we exploit to define map operations in the spectral domain. To apply these ideas in practice, we introduce localized spectral analysis of the product manifold as a novel tool for map processing.
Kurpas D, 2018, Preface, FAMILY MEDICINE AND PRIMARY CARE REVIEW, Vol: 20, ISSN: 1734-3402
Litany O, Remez T, Rodola E, et al., 2017, Deep Functional Maps: Structured Prediction for Dense Shape Correspondence, Pages: 5660-5668, ISSN: 1550-5499
We introduce a new framework for learning dense correspondence between deformable 3D shapes. Existing learning based approaches model shape correspondence as a labelling problem, where each point of a query shape receives a label identifying a point on some reference domain; the correspondence is then constructed a posteriori by composing the label predictions of two input shapes. We propose a paradigm shift and design a structured prediction model in the space of functional maps, linear operators that provide a compact representation of the correspondence. We model the learning process via a deep residual network which takes dense descriptor fields defined on two shapes as input, and outputs a soft map between the two given objects. The resulting correspondence is shown to be accurate on several challenging benchmarks comprising multiple categories, synthetic models, real scans with acquisition artifacts, topological noise, and partiality.
Monti F, Bronstein M, Bresson X, 2017, Geometric matrix completion with recurrent multi-graph neural networks, Thirty-first Conference on Neural Information Processing Systems, Pages: 3697-3707
Monti F, Bronstein MM, Bresson X, 2017, Geometric matrix completion with recurrent multi-graph neural networks, Neural Information Processing Systems, Publisher: Curran Associates Inc., Pages: 3700-3710, ISSN: 1049-5258
Matrix completion models are among the most common formulations of recommender systems. Recent works have showed a boost of performance of these techniques when introducing the pairwise relationships between users/items in the form of graphs, and imposing smoothness priors on these graphs. However, such techniques do not fully exploit the local stationary structures on user/item graphs, and the number of parameters to learn is linear w.r.t. the number of users and items. We propose a novel approach to overcome these limitations by using geometric deep learning on graphs. Our matrix completion architecture combines a novel multi-graph convolutional neural network that can learn meaningful statistical graph-structured patterns from users and items, and a recurrent neural network that applies a learnable diffusion on the score matrix. Our neural network system is computationally attractive as it requires a constant number of parameters independent of the matrix size. We apply our method on several standard datasets, showing that it outperforms state-of-the-art matrix completion techniques.
Monti F, Boscaini D, Masci J, et al., 2017, Geometric deep learning on graphs and manifolds using mixture model CNNs, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Publisher: IEEE, Pages: 5425-5434, ISSN: 1063-6919
Deep learning has achieved a remarkable performance breakthrough in several fields, most notably in speech recognition, natural language processing, and computer vision. In particular, convolutional neural network (CNN) architectures currently produce state-of-the-art performance on a variety of image analysis tasks such as object detection and recognition. Most of deep learning research has so far focused on dealing with 1D, 2D, or 3D Euclidean-structured data such as acoustic signals, images, or videos. Recently, there has been an increasing interest in geometric deep learning, attempting to generalize deep learning methods to non-Euclidean structured data such as graphs and manifolds, with a variety of applications from the domains of network analysis, computational social science, or computer graphics. In this paper, we propose a unified framework allowing to generalize CNN architectures to non-Euclidean domains (graphs and manifolds) and learn local, stationary, and compositional task-specific features. We show that various non-Euclidean CNN methods previously proposed in the literature can be considered as particular instances of our framework. We test the proposed method on standard tasks from the realms of image-, graph-and 3D shape analysis and show that it consistently outperforms previous approaches.
Monti F, Boscaini D, Masci J, et al., 2017, Geometric deep learning on graphs and manifolds using mixture model CNNs, 2017 IEEE Conference on Computer Vision and Pattern Recognition, Pages: 3-3
Ovsjanikov M, Corman E, Bronstein M, et al., 2017, Computing and processing correspondences with functional maps
Notions of similarity and correspondence between geometric shapes and images are central to many tasks in geometry processing, computer vision, and computer graphics. The goal of this course is to familiarize the audience with a set of recent techniques that greatly facilitate the computation of mappings or correspondences between geometric datasets, such as 3D shapes or 2D images by formulating them as mappings between functions rather than points or triangles. Methods based on the functional map framework have recently led to state-of-the-art results in problems as diverse as non-rigid shape matching, image co-segmentation and even some aspects of tangent vector field design. One challenge in adopting these methods in practice, however, is that their exposition often assumes a significant amount of background in geometry processing, spectral methods and functional analysis, which can make it difficult to gain an intuition about their performance or about their applicability to real-life problems. In this course, we try to provide all the tools necessary to appreciate and use these techniques, while assuming very little background knowledge. We also give a unifying treatment of these techniques, which may be difficult to extract from the individual publications and, at the same time, hint at the generality of this point of view, which can help tackle many problems in the analysis and creation of visual content. This course is structured as a half day course. We will assume that the participants have knowledge of basic linear algebra and some knowledge of differential geometry, to the extent of being familiar with the concepts of a manifold and a tangent vector space. We will discuss in detail the functional approach to finding correspondences between non-rigid shapes, the design and analysis of tangent vector fields on surfaces, consistent map estimation in networks of shapes and applications to shape and image segmentation, shape variability analysis, and other ar
Bronstein MM, Bruna J, LeCun Y, et al., 2017, Geometric Deep Learning Going beyond Euclidean data, IEEE SIGNAL PROCESSING MAGAZINE, Vol: 34, Pages: 18-42, ISSN: 1053-5888
Svoboda J, Monti F, Bronstein MM, 2017, Generative convolutional networks for latent fingerprint reconstruction, Pages: 429-436
Performance of fingerprint recognition depends heavily on the extraction of minutiae points. Enhancement of the fingerprint ridge pattern is thus an essential pre-processing step that noticeably reduces false positive and negative detection rates. A particularly challenging setting is when the fingerprint images are corrupted or partially missing. In this work, we apply generative convolutional networks to denoise visible minutiae and predict the missing parts of the ridge pattern. The proposed enhancement approach is tested as a pre-processing step in combination with several standard feature extraction methods such as MINDTCT, followed by biometric comparison using MCC and BO-ZORTH3. We evaluate our method on several publicly available latent fingerprint datasets captured using different sensors.
Litany O, Rodolà E, Bronstein AM, et al., 2017, Fully Spectral Partial Shape Matching, Computer Graphics Forum, Vol: 36, Pages: 247-258, ISSN: 0167-7055
© 2017 The Author(s) Computer Graphics Forum © 2017 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd. We propose an efficient procedure for calculating partial dense intrinsic correspondence between deformable shapes performed entirely in the spectral domain. Our technique relies on the recently introduced partial functional maps formalism and on the joint approximate diagonalization (JAD) of the Laplace-Beltrami operators previously introduced for matching non-isometric shapes. We show that a variant of the JAD problem with an appropriately modified coupling term (surprisingly) allows to construct quasi-harmonic bases localized on the latent corresponding parts. This circumvents the need to explicitly compute the unknown parts by means of the cumbersome alternating minimization used in the previous approaches, and allows performing all the calculations in the spectral domain with constant complexity independent of the number of shape vertices. We provide an extensive evaluation of the proposed technique on standard non-rigid correspondence benchmarks and show state-of-the-art performance in various settings, including partiality and the presence of topological noise.
Rodolà E, Cosmo L, Bronstein MM, et al., 2017, Partial functional correspondence, Computer Graphics Forum: the international journal of the Eurographics Association, Vol: 36, Pages: 222-236, ISSN: 0167-7055
In this paper, we propose a method for computing partial functional correspondence between non‐rigid shapes. We use perturbation analysis to show how removal of shape parts changes the Laplace–Beltrami eigenfunctions, and exploit it as a prior on the spectral representation of the correspondence. Corresponding parts are optimization variables in our problem and are used to weight the functional correspondence; we are looking for the largest and most regular (in the Mumford–Shah sense) parts that minimize correspondence distortion. We show that our approach can cope with very challenging correspondence settings.
Melzi S, Rodolà E, Castellani U, et al., 2017, Localized Manifold Harmonics for Spectral Shape Analysis, Pages: 5-6, ISSN: 1727-8384
The use of Laplacian eigenfunctions is ubiquitous in a wide range of computer graphics and geometry processing applications. In particular, Laplacian eigenbases allow generalizing the classical Fourier analysis to manifolds. A key drawback of such bases is their inherently global nature, as the Laplacian eigenfunctions carry geometric and topological structure of the entire manifold. In this paper, we introduce a new framework for local spectral shape analysis. We show how to efficiently construct localized orthogonal bases by solving an optimization problem that in turn can be posed as the eigendecomposition of a new operator obtained by a modification of the standard Laplacian. We study the theoretical and computational aspects of the proposed framework and showcase our new construction on the classical problems of shape approximation and correspondence.
Boyarski A, Bronstein AM, Bronstein MM, 2017, Subspace least squares multidimensional scaling, Pages: 681-693, ISSN: 0302-9743
Multidimensional Scaling (MDS) is one of the most popular methods for dimensionality reduction and visualization of high dimensional data. Apart from these tasks, it also found applications in the field of geometry processing for the analysis and reconstruction of nonrigid shapes. In this regard, MDS can be thought of as a shape from metric algorithm, consisting of finding a configuration of points in the Euclidean space that realize, as isometrically as possible, some given distance structure. In the present work we cast the least squares variant of MDS (LS-MDS) in the spectral domain. This uncovers a multiresolution property of distance scaling which speeds up the optimization by a significant amount, while producing comparable, and sometimes even better, embeddings.
Rodolà E, Cosmo L, Litany O, et al., 2017, SHREC'17: Deformable shape retrieval with missing parts, Pages: 85-94, ISSN: 1997-0463
Partial similarity problems arise in numerous applications that involve real data acquisition by 3D sensors, inevitably leading to missing parts due to occlusions and partial views. In this setting, the shapes to be retrieved may undergo a variety of transformations simultaneously, such as non-rigid deformations (changes in pose), topological noise, and missing parts - a combination of nuisance factors that renders the retrieval process extremely challenging. With this benchmark, we aim to evaluate the state of the art in deformable shape retrieval under such kind of transformations. The benchmark is organized in two sub-challenges exemplifying different data modalities (3D vs. 2.5D). A total of 15 retrieval algorithms were evaluated in the contest; this paper presents the details of the dataset, and shows thorough comparisons among all competing methods.
Eynard D, Rodola E, Glashoff K, et al., 2016, Coupled functional maps, Pages: 399-407
Classical formulations of the shape matching problem involve the definition of a matching cost that directly depends on the action of the desired map when applied to some input data. Such formulations are typically one-sided - they seek for a mapping from one shape to the other, but not vice versa. In this paper we consider an unbiased formulation of this problem, in which we solve simultaneously for a low-distortion map relating the two given shapes and its inverse. We phrase the problem in the spectral domain using the language of functional maps, resulting in an especially compact and efficient optimization problem. The benefits of our proposed regularization are especially evident in the scarce data setting, where we demonstrate highly competitive results with respect to the state of the art.
Melzi S, Rodola E, Castellani U, et al., 2016, Shape analysis with anisotropic windowed Fourier transform, Pages: 470-478
We propose Anisotropic Windowed Fourier Transform (AWFT), a framework for localized space-frequency analysis of deformable 3D shapes. With AWFT, we are able to extract meaningful intrinsic localized orientation-sensitive structures on surfaces, and use them in applications such as shape segmentation, salient point detection, feature point description, and matching. Our method outperforms previous approaches in the considered applications.
Cosmo L, Rodola E, Masci J, et al., 2016, Matching deformable objects in clutter, Pages: 1-10
We consider the problem of deformable object detection and dense correspondence in cluttered 3D scenes. Key ingredient to our method is the choice of representation: we formulate the problem in the spectral domain using the functional maps framework, where we seek for the most regular nearly-isometric parts in the model and the scene that minimize correspondence error. The problem is initialized by solving a sparse relaxation of a quadratic assignment problem on features obtained via data-driven metric learning. The resulting matching pipeline is solved efficiently, and yields accurate results in challenging settings that were previously left unexplored in the literature.
Lahner Z, Rodola E, Schmidt FR, et al., 2016, Efficient Globally Optimal 2D-to-3D Deformable Shape Matching, Pages: 2185-2193, ISSN: 1063-6919
We propose the first algorithm for non-rigid 2D-to-3D shape matching, where the input is a 2D query shape as well as a 3D target shape and the output is a continuous matching curve represented as a closed contour on the 3D shape. We cast the problem as finding the shortest circular path on the product 3-manifold of the two shapes. We prove that the optimal matching can be computed in polynomial time with a (worst-case) complexity of O(mn2 log(n)), wherem and n denote the number of vertices on the 2D and the 3D shape respectively. Quantitative evaluation confirms that the method provides excellent results for sketch-based deformable 3D shape retrieval.
Masci J, Rodolà E, Boscaini D, et al., 2016, Geometric deep learning
- Cite
- Citations: 32
Ovsjanikov M, Corman E, Bronstein M, et al., 2016, Computing and processing correspondences with functional maps
Notions of similarity and correspondence between geometric shapes and images are central to many tasks in geometry processing, computer vision, and computer graphics. The goal of this course is to familiarize the audience with a set of recent techniques that greatly facilitate the computation of mappings or correspondences between geometric datasets, such as 3D shapes or 2D images by formulating them as mappings between functions rather than points or triangles. Methods based on the functional map framework have recently led to state-of-the-art results in problems as diverse as non-rigid shape matching, image co-segmentation and even some aspects of tangent vector field design. One challenge in adopting these methods in practice, however, is that their exposition often assumes a significant amount of background in geometry processing, spectral methods and functional analysis, which can make it difficult to gain an intuition about their performance or about their applicability to real-life problems. In this course, we try to provide all the tools necessary to appreciate and use these techniques, while assuming very little background knowledge. We also give a unifying treatment of these techniques, which may be difficult to extract from the individual publications and, at the same time, hint at the generality of this point of view, which can help tackle many problems in the analysis and creation of visual content. This course is structured as a half day course. We will assume that the participants have knowledge of basic linear algebra and some knowledge of differential geometry, to the extent of being familiar with the concepts of a manifold and a tangent vector space. We will discuss in detail the functional approach to finding correspondences between non-rigid shapes, the design and analysis of tangent vector fields on surfaces, consistent map estimation in networks of shapes and applications to shape and image segmentation, shape variability analysis, and other ar
Pickup D, Sun X, Rosin PL, et al., 2016, Shape Retrieval of Non-rigid 3D Human Models, International Journal of Computer Vision, Vol: 120, Pages: 169-193, ISSN: 0920-5691
© 2016, The Author(s). 3D models of humans are commonly used within computer graphics and vision, and so the ability to distinguish between body shapes is an important shape retrieval problem. We extend our recent paper which provided a benchmark for testing non-rigid 3D shape retrieval algorithms on 3D human models. This benchmark provided a far stricter challenge than previous shape benchmarks. We have added 145 new models for use as a separate training set, in order to standardise the training data used and provide a fairer comparison. We have also included experiments with the FAUST dataset of human scans. All participants of the previous benchmark study have taken part in the new tests reported here, many providing updated results using the new data. In addition, further participants have also taken part, and we provide extra analysis of the retrieval results. A total of 25 different shape retrieval methods are compared.
Biasotti S, Cerri A, Bronstein A, et al., 2016, Recent Trends, Applications, and Perspectives in 3D Shape Similarity Assessment, Computer Graphics Forum, Vol: 35, Pages: 87-119, ISSN: 0167-7055
© 2015 The Authors Computer Graphics Forum © 2015 The Eurographics Association and John Wiley & Sons Ltd. The recent introduction of 3D shape analysis frameworks able to quantify the deformation of a shape into another in terms of the variation of real functions yields a new interpretation of the 3D shape similarity assessment and opens new perspectives. Indeed, while the classical approaches to similarity mainly quantify it as a numerical score, map-based methods also define (dense) shape correspondences. After presenting in detail the theoretical foundations underlying these approaches, we classify them by looking at their most salient features, including the kind of structure and invariance properties they capture, as well as the distances and the output modalities according to which the similarity between shapes is assessed and returned. We also review the usage of these methods in a number of 3D shape application domains, ranging from matching and retrieval to annotation and segmentation. Finally, the most promising directions for future research developments are discussed.
Litany O, Rodolà E, Bronstein AM, et al., 2016, Non-rigid puzzles, Computer Graphics Forum, Vol: 35, Pages: 135-143
Boscaini D, Masci J, Rodolà E, et al., 2016, Anisotropic diffusion descriptors, 37th Annual Conference of the European Association for Computer Graphics, Pages: 431-441
Masci J, Boscaini D, Bronstein MM, et al., 2016, Geodesic Convolutional Neural Networks on Riemannian Manifolds, Pages: 832-840, ISSN: 1550-5499
Feature descriptors play a crucial role in a wide range of geometry analysis and processing applications, including shape correspondence, retrieval, and segmentation. In this paper, we introduce Geodesic Convolutional Neural Networks (GCNN), a generalization of the convolutional neural networks (CNN) paradigm to non-Euclidean manifolds. Our construction is based on a local geodesic system of polar coordinates to extract "patches", which are then passed through a cascade of filters and linear and non-linear operators. The coefficients of the filters and linear combination weights are optimization variables that are learned to minimize a task-specific cost function. We use ShapeNet to learn invariant shape features, allowing to achieve state-of-The-Art performance in problems such as shape description, retrieval, and correspondence.
Kovnatsky A, Glashoff K, Bronstein MM, 2016, MADMM: A generic algorithm for non-smooth optimization on manifolds, Pages: 680-696, ISSN: 0302-9743
Numerous problems in computer vision, pattern recognition, and machine learning are formulated as optimization with manifold constraints. In this paper, we propose the Manifold Alternating Directions Method of Multipliers (MADMM), an extension of the classical ADMM scheme for manifold-constrained non-smooth optimization problems. To our knowledge, MADMM is the first generic non-smooth manifold optimization method. We showcase our method on several challenging problems in dimensionality reduction, non-rigid correspondence, multi-modal clustering, and multidimensional scaling.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.