184 results found
Gainza P, Sverrisson F, Monti F, et al., 2019, Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning., Nat Methods
Predicting interactions between proteins and other biomolecules solely based on structure remains a challenge in biology. A high-level representation of protein structure, the molecular surface, displays patterns of chemical and geometric features that fingerprint a protein's modes of interactions with other biomolecules. We hypothesize that proteins participating in similar interactions may share common fingerprints, independent of their evolutionary history. Fingerprints may be difficult to grasp by visual analysis but could be learned from large-scale datasets. We present MaSIF (molecular surface interaction fingerprinting), a conceptual framework based on a geometric deep learning method to capture fingerprints that are important for specific biomolecular interactions. We showcase MaSIF with three prediction challenges: protein pocket-ligand prediction, protein-protein interaction site prediction and ultrafast scanning of protein surfaces for prediction of protein-protein complexes. We anticipate that our conceptual framework will lead to improvements in our understanding of protein function and design.
Rodolà E, Lähner Z, Bronstein AM, et al., 2019, Functional Maps Representation On Product Manifolds, Computer Graphics Forum, ISSN: 0167-7055
© 2019 The Authors Computer Graphics Forum © 2019 The Eurographics Association and John Wiley & Sons Ltd. We consider the tasks of representing, analysing and manipulating maps between shapes. We model maps as densities over the product manifold of the input shapes; these densities can be treated as scalar functions and therefore are manipulable using the language of signal processing on manifolds. Being a manifold itself, the product space endows the set of maps with a geometry of its own, which we exploit to define map operations in the spectral domain; we also derive relationships with other existing representations (soft maps and functional maps). To apply these ideas in practice, we discretize product manifolds and their Laplace–Beltrami operators, and we introduce localized spectral analysis of the product manifold as a novel tool for map processing. Our framework applies to maps defined between and across 2D and 3D shapes without requiring special adjustment, and it can be implemented efficiently with simple operations on sparse matrices.
Choma N, Monti F, Gerhardt L, et al., 2019, Graph Neural Networks for IceCube Signal Classification, Pages: 386-391
© 2018 IEEE. Tasks involving the analysis of geometric (graph-and manifold-structured) data have recently gained prominence in the machine learning community, giving birth to a rapidly developing field of geometric deep learning. In this work, we leverage graph neural networks to improve signal detection in the IceCube neutrino observatory. The IceCube detector array is modeled as a graph, where vertices are sensors and edges are a learned function of the sensors spatial coordinates. As only a subset of IceCubes sensors is active during a given observation, we note the adaptive nature of our GNN, wherein computation is restricted to the input signal support. We demonstrate the effectiveness of our GNN architecture on a task classifying IceCube events, where it outperforms both a traditional physics-based method as well as classical 3D convolution neural networks.
Svoboda J, Masci J, Monti F, et al., 2019, Peernets: Exploiting peer wisdom against adversarial attacks
© 7th International Conference on Learning Representations, ICLR 2019. All Rights Reserved. Deep learning systems have become ubiquitous in many aspects of our lives. Unfortunately, it has been shown that such systems are vulnerable to adversarial attacks, making them prone to potential unlawful uses. Designing deep neural networks that are robust to adversarial attacks is a fundamental step in making such systems safer and deployable in a broader variety of applications (e.g. autonomous driving), but more importantly is a necessary step to design novel and more advanced architectures built on new computational paradigms rather than marginally building on the existing ones. In this paper we introduce PeerNets, a novel family of convolutional networks alternating classical Euclidean convolutions with graph convolutions to harness information from a graph of peer samples. This results in a form of non-local forward propagation in the model, where latent features are conditioned on the global structure induced by the graph, that is up to 3× more robust to a variety of white- and black-box adversarial attacks compared to conventional architectures with almost no drop in accuracy.
Litany O, Bronstein A, Bronstein M, et al., 2018, Deformable Shape Completion with Graph Convolutional Autoencoders, Pages: 1886-1895, ISSN: 1063-6919
© 2018 IEEE. The availability of affordable and portable depth sensors has made scanning objects and people simpler than ever. However, dealing with occlusions and missing parts is still a significant challenge. The problem of reconstructing a (possibly non-rigidly moving) 3D object from a single or multiple partial scans has received increasing attention in recent years. In this work, we propose a novel learning-based method for the completion of partial shapes. Unlike the majority of existing approaches, our method focuses on objects that can undergo non-rigid deformations. The core of our method is a variational autoencoder with graph convolutional operations that learns a latent space for complete realistic shapes. At inference, we optimize to find the representation in this latent space that best fits the generated shape to the known partial input. The completed shape exhibits a realistic appearance on the unknown part. We show promising results towards the completion of synthetic and real scans of human body and face meshes exhibiting different styles of articulation and partiality.
Levie R, Monti F, Bresson X, et al., 2018, CayleyNets: Graph Convolutional Neural Networks with Complex Rational Spectral Filters, IEEE Transactions on Signal Processing, Vol: 67, Pages: 97-109, ISSN: 1053-587X
© 1991-2012 IEEE. The rise of graph-structured data such as social networks, regulatory networks, citation graphs, and functional brain networks, in combination with resounding success of deep learning in various applications, has brought the interest in generalizing deep learning models to non-Euclidean domains. In this paper, we introduce a new spectral domain convolutional architecture for deep learning on graphs. The core ingredient of our model is a new class of parametric rational complex functions (Cayley polynomials) allowing to efficiently compute spectral filters on graphs that specialize on frequency bands of interest. Our model generates rich spectral filters that are localized in space, scales linearly with the size of the input data for sparsely connected graphs, and can handle different constructions of Laplacian operators. Extensive experimental results show the superior performance of our approach, in comparison to other spectral domain convolutional architectures, on spectral image classification, community detection, vertex classification, and matrix completion tasks.
Litany O, Rodolà E, Bronstein A, et al., 2018, Partial Single- and Multishape Dense Correspondence Using Functional Maps, Handbook of Numerical Analysis, Vol: 19, Pages: 55-90, ISSN: 1570-8659
© 2018 Elsevier B.V. Shape correspondence is a fundamental problem in computer graphics and vision, with applications in various problems including animation, texture mapping, robotic vision, medical imaging, archaeology and many more. In settings where the shapes are allowed to undergo nonrigid deformations and only partial views are available, the problem becomes very challenging. In this chapter we describe recent techniques designed to tackle such problems. Specifically, we explain how the renown functional maps framework can be extended to tackle the partial setting. We then present a further extension to the multipart case in which one tries to establish correspondence between a collection of shapes. Finally, we focus on improving the technique efficiency, by disposing of its spatial ingredient and thus keeping the computation in the spectral domain. Extensive experimental results are provided along with the theoretical explanations, to demonstrate the effectiveness of the described methods in these challenging scenarios.
Monti F, Bronstein MM, Bresson X, 2018, Deep geometric matrix completion: A new way for recommender systems, Pages: 6852-6856, ISSN: 1520-6149
© 2018 IEEE. In the last years, Graph Convolutional Neural Networks gained popularity in the Machine Learning community for their capability of extracting local compositional features on signals defined on non-Euclidean domains. Shape correspondence, document classification, molecular properties predictions are just few of the many different problems where these techniques have been successfully applied. In this paper we will present Deep Geometric Matrix Completion, a recent application of Graph Convolutional Neural Networks to the matrix completion problem. We will illustrate MGCNN (a multi-graph CNN able to deal with signals defined over multiple domains) and we will show how coupling such technique with a RNN, a learnable diffusion process can be realized for reconstructing the desired information. Extensive experimental evaluation shows how Geometric Deep Learning techniques allow to outperform previous state of the art solutions on the matrix completion problem.
Monti F, Otness K, Bronstein MM, 2018, MOTIFNET: A MOTIF-BASED GRAPH CONVOLUTIONAL NETWORK for DIRECTED GRAPHS, Pages: 225-228
© 2018 IEEE. Deep learning on graphs and in particular, graph convolutional neural networks, have recently attracted significant attention in the machine learning community. Many of such techniques explore the analogy between the graph Laplacian eigenvectors and the classical Fourier basis, allowing to formulate the convolution as a multiplication in the spectral domain. One of the key drawback of spectral CNNs is their explicit assumption of an undirected graph, leading to a symmetric Laplacian matrix with orthogonal eigendecomposition. In this work we propose MotifNet, a graph CNN capable of dealing with directed graphs by exploiting local graph motifs. We present experimental evidence showing the advantage of our approach on real data.
Melzi S, Rodolà E, Castellani U, et al., 2018, Localized Manifold Harmonics for Spectral Shape Analysis, Computer Graphics Forum, Vol: 37, Pages: 20-34, ISSN: 0167-7055
© 2017 The Authors Computer Graphics Forum © 2017 The Eurographics Association and John Wiley & Sons Ltd. The use of Laplacian eigenfunctions is ubiquitous in a wide range of computer graphics and geometry processing applications. In particular, Laplacian eigenbases allow generalizing the classical Fourier analysis to manifolds. A key drawback of such bases is their inherently global nature, as the Laplacian eigenfunctions carry geometric and topological structure of the entire manifold. In this paper, we introduce a new framework for local spectral shape analysis. We show how to efficiently construct localized orthogonal bases by solving an optimization problem that in turn can be posed as the eigendecomposition of a new operator obtained by a modification of the standard Laplacian. We study the theoretical and computational aspects of the proposed framework and showcase our new construction on the classical problems of shape approximation and correspondence. We obtain significant improvement compared to classical Laplacian eigenbases as well as other alternatives for constructing localized bases.
Gasparetto A, Cosmo L, Rodola E, et al., 2018, Spatial Maps: From low rank spectral to sparse spatial functional representations, Pages: 477-485
© 2017 IEEE. Functional representation is a well-established approach to represent dense correspondences between deformable shapes. The approach provides an efficient low rank representation of a continuous mapping between two shapes, however under that framework the correspondences are only intrinsically captured, which implies that the induced map is not guaranteed to map the whole surface, much less to form a continuous mapping. In this work, we define a novel approach to the computation of a continuous bijective map between two surfaces moving from the low rank spectral representation to a sparse spatial representation. Key to this is the observation that continuity and smoothness of the optimal map induces structure both on the spectral and the spatial domain, the former providing effective low rank approximations, while the latter exhibiting strong sparsity and locality that can be used in the solution of large-scale problems. We cast our approach in terms of the functional transfer through a fuzzy map between shapes satisfying infinitesimal mass transportation at each point. The result is that, not only the spatial map induces a sub-vertex correspondence between the surfaces, but also the transportation of the whole surface, and thus the bijectivity of the induced map is assured. The performance of the proposed method is assessed on several popular benchmarks.
Vestner M, Lahner Z, Boyarski A, et al., 2018, Efficient deformable shape correspondence via kernel matching, Pages: 517-526
© 2017 IEEE. We present a method to match three dimensional shapes under non-isometric deformations, topology changes and partiality. We formulate the problem as matching between a set of pair-wise and point-wise descriptors, imposing a continuity prior on the mapping, and propose a projected descent optimization procedure inspired by difference of convex functions (DC) programming.
Nogneng D, Melzi S, Rodolá E, et al., 2018, Improved functional mappings via product preservation, Computer Graphics Forum, Vol: 37, Pages: 1-12, ISSN: 0167-7055
© 2018 The Author(s) and 2018 The Eurographics Association and John Wiley & Sons Ltd. In this paper, we consider the problem of information transfer across shapes and propose an extension to the widely used functional map representation. Our main observation is that in addition to the vector space structure of the functional spaces, which has been heavily exploited in the functional map framework, the functional algebra (i.e., the ability to take pointwise products of functions) can significantly extend the power of this framework. Equipped with this observation, we show how to improve one of the key applications of functional maps, namely transferring real-valued functions without conversion to point-to-point correspondences. We demonstrate through extensive experiments that by decomposing a given function into a linear combination consisting not only of basis functions but also of their pointwise products, both the representation power and the quality of the function transfer can be improved significantly. Our modification, while computationally simple, allows us to achieve higher transfer accuracy while keeping the size of the basis and the functional map fixed. We also analyze the computational complexity of optimally representing functions through linear combinations of products in a given basis and prove NP-completeness in some general cases. Finally, we argue that the use of function products can have a wide-reaching effect in extending the power of functional maps in a variety of applications, in particular by enabling the transfer of highfrequency functions without changing the representation size or complexity.
Svoboda J, Monti F, Bronstein MM, 2018, Generative convolutional networks for latent fingerprint reconstruction, Pages: 429-436
© 2017 IEEE. Performance of fingerprint recognition depends heavily on the extraction of minutiae points. Enhancement of the fingerprint ridge pattern is thus an essential pre-processing step that noticeably reduces false positive and negative detection rates. A particularly challenging setting is when the fingerprint images are corrupted or partially missing. In this work, we apply generative convolutional networks to denoise visible minutiae and predict the missing parts of the ridge pattern. The proposed enhancement approach is tested as a pre-processing step in combination with several standard feature extraction methods such as MINDTCT, followed by biometric comparison using MCC and BO-ZORTH3. We evaluate our method on several publicly available latent fingerprint datasets captured using different sensors.
Kurpas D, 2018, Preface, FAMILY MEDICINE AND PRIMARY CARE REVIEW, Vol: 20, ISSN: 1734-3402
Litany O, Remez T, Rodola E, et al., 2017, Deep Functional Maps: Structured Prediction for Dense Shape Correspondence, Pages: 5660-5668, ISSN: 1550-5499
© 2017 IEEE. We introduce a new framework for learning dense correspondence between deformable 3D shapes. Existing learning based approaches model shape correspondence as a labelling problem, where each point of a query shape receives a label identifying a point on some reference domain; the correspondence is then constructed a posteriori by composing the label predictions of two input shapes. We propose a paradigm shift and design a structured prediction model in the space of functional maps, linear operators that provide a compact representation of the correspondence. We model the learning process via a deep residual network which takes dense descriptor fields defined on two shapes as input, and outputs a soft map between the two given objects. The resulting correspondence is shown to be accurate on several challenging benchmarks comprising multiple categories, synthetic models, real scans with acquisition artifacts, topological noise, and partiality.
Monti F, Bronstein M, Bresson X, 2017, Geometric matrix completion with recurrent multi-graph neural networks, Thirty-first Conference on Neural Information Processing Systems, Pages: 3697-3707
Monti F, Boscaini D, Masci J, et al., 2017, Geometric deep learning on graphs and manifolds using mixture model CNNs, 2017 IEEE Conference on Computer Vision and Pattern Recognition, Pages: 3-3
Monti F, Boscaini D, Masci J, et al., 2017, Geometric deep learning on graphs and manifolds using mixture model CNNs, Pages: 5425-5434
© 2017 IEEE. Deep learning has achieved a remarkable performance breakthrough in several fields, most notably in speech recognition, natural language processing, and computer vision. In particular, convolutional neural network (CNN) architectures currently produce state-of-the-art performance on a variety of image analysis tasks such as object detection and recognition. Most of deep learning research has so far focused on dealing with 1D, 2D, or 3D Euclidean-structured data such as acoustic signals, images, or videos. Recently, there has been an increasing interest in geometric deep learning, attempting to generalize deep learning methods to non-Euclidean structured data such as graphs and manifolds, with a variety of applications from the domains of network analysis, computational social science, or computer graphics. In this paper, we propose a unified framework allowing to generalize CNN architectures to non-Euclidean domains (graphs and manifolds) and learn local, stationary, and compositional task-specific features. We show that various non-Euclidean CNN methods previously proposed in the literature can be considered as particular instances of our framework. We test the proposed method on standard tasks from the realms of image-, graph-and 3D shape analysis and show that it consistently outperforms previous approaches.
Monti F, Bronstein MM, Bresson X, 2017, Geometric matrix completion with recurrent multi-graph neural networks, International Conference on Machine Learning, Pages: 3698-3708, ISSN: 1049-5258
© 2017 Neural information processing systems foundation. All rights reserved. Matrix completion models are among the most common formulations of recommender systems. Recent works have showed a boost of performance of these techniques when introducing the pairwise relationships between users/items in the form of graphs, and imposing smoothness priors on these graphs. However, such techniques do not fully exploit the local stationary structures on user/item graphs, and the number of parameters to learn is linear w.r.t. the number of users and items. We propose a novel approach to overcome these limitations by using geometric deep learning on graphs. Our matrix completion architecture combines a novel multi-graph convolutional neural network that can learn meaningful statistical graph-structured patterns from users and items, and a recurrent neural network that applies a learnable diffusion on the score matrix. Our neural network system is computationally attractive as it requires a constant number of parameters independent of the matrix size. We apply our method on several standard datasets, showing that it outperforms state-of-the-art matrix completion techniques.
Ovsjanikov M, Corman E, Bronstein M, et al., 2017, Computing and processing correspondences with functional maps
Notions of similarity and correspondence between geometric shapes and images are central to many tasks in geometry processing, computer vision, and computer graphics. The goal of this course is to familiarize the audience with a set of recent techniques that greatly facilitate the computation of mappings or correspondences between geometric datasets, such as 3D shapes or 2D images by formulating them as mappings between functions rather than points or triangles. Methods based on the functional map framework have recently led to state-of-the-art results in problems as diverse as non-rigid shape matching, image co-segmentation and even some aspects of tangent vector field design. One challenge in adopting these methods in practice, however, is that their exposition often assumes a significant amount of background in geometry processing, spectral methods and functional analysis, which can make it difficult to gain an intuition about their performance or about their applicability to real-life problems. In this course, we try to provide all the tools necessary to appreciate and use these techniques, while assuming very little background knowledge. We also give a unifying treatment of these techniques, which may be difficult to extract from the individual publications and, at the same time, hint at the generality of this point of view, which can help tackle many problems in the analysis and creation of visual content. This course is structured as a half day course. We will assume that the participants have knowledge of basic linear algebra and some knowledge of differential geometry, to the extent of being familiar with the concepts of a manifold and a tangent vector space. We will discuss in detail the functional approach to finding correspondences between non-rigid shapes, the design and analysis of tangent vector fields on surfaces, consistent map estimation in networks of shapes and applications to shape and image segmentation, shape variability analysis, and other ar
Bronstein MM, Bruna J, LeCun Y, et al., 2017, Geometric Deep Learning Going beyond Euclidean data, IEEE SIGNAL PROCESSING MAGAZINE, Vol: 34, Pages: 18-42, ISSN: 1053-5888
© 2017 The Author(s) Computer Graphics Forum © 2017 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd. We propose an efficient procedure for calculating partial dense intrinsic correspondence between deformable shapes performed entirely in the spectral domain. Our technique relies on the recently introduced partial functional maps formalism and on the joint approximate diagonalization (JAD) of the Laplace-Beltrami operators previously introduced for matching non-isometric shapes. We show that a variant of the JAD problem with an appropriately modified coupling term (surprisingly) allows to construct quasi-harmonic bases localized on the latent corresponding parts. This circumvents the need to explicitly compute the unknown parts by means of the cumbersome alternating minimization used in the previous approaches, and allows performing all the calculations in the spectral domain with constant complexity independent of the number of shape vertices. We provide an extensive evaluation of the proposed technique on standard non-rigid correspondence benchmarks and show state-of-the-art performance in various settings, including partiality and the presence of topological noise.
Rodolà E, Cosmo L, Bronstein MM, et al., 2017, Partial functional correspondence, Computer Graphics Forum, Vol: 36, Pages: 222-236
Boyarski A, Bronstein AM, Bronstein MM, 2017, Subspace least squares multidimensional scaling, Pages: 681-693, ISSN: 0302-9743
© Springer International Publishing AG 2017. Multidimensional Scaling (MDS) is one of the most popular methods for dimensionality reduction and visualization of high dimensional data. Apart from these tasks, it also found applications in the field of geometry processing for the analysis and reconstruction of nonrigid shapes. In this regard, MDS can be thought of as a shape from metric algorithm, consisting of finding a configuration of points in the Euclidean space that realize, as isometrically as possible, some given distance structure. In the present work we cast the least squares variant of MDS (LS-MDS) in the spectral domain. This uncovers a multiresolution property of distance scaling which speeds up the optimization by a significant amount, while producing comparable, and sometimes even better, embeddings.
Rodolà E, Cosmo L, Litany O, et al., 2017, SHREC'17: Deformable shape retrieval with missing parts, Pages: 85-94, ISSN: 1997-0463
© 2017 The Eurographics Association. Partial similarity problems arise in numerous applications that involve real data acquisition by 3D sensors, inevitably leading to missing parts due to occlusions and partial views. In this setting, the shapes to be retrieved may undergo a variety of transformations simultaneously, such as non-rigid deformations (changes in pose), topological noise, and missing parts - a combination of nuisance factors that renders the retrieval process extremely challenging. With this benchmark, we aim to evaluate the state of the art in deformable shape retrieval under such kind of transformations. The benchmark is organized in two sub-challenges exemplifying different data modalities (3D vs. 2.5D). A total of 15 retrieval algorithms were evaluated in the contest; this paper presents the details of the dataset, and shows thorough comparisons among all competing methods.
© 2016 IEEE. We consider the problem of deformable object detection and dense correspondence in cluttered 3D scenes. Key ingredient to our method is the choice of representation: we formulate the problem in the spectral domain using the functional maps framework, where we seek for the most regular nearly-isometric parts in the model and the scene that minimize correspondence error. The problem is initialized by solving a sparse relaxation of a quadratic assignment problem on features obtained via data-driven metric learning. The resulting matching pipeline is solved efficiently, and yields accurate results in challenging settings that were previously left unexplored in the literature.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.