Publications
637 results found
Khalifa Y, Mandic D, Sejdic E, 2021, A review of Hidden Markov models and Recurrent Neural Networks for event detection and localization in biomedical signals, INFORMATION FUSION, Vol: 69, Pages: 52-72, ISSN: 1566-2535
Ruiz-Garcia A, Schmidhuber J, Palade V, et al., 2021, Deep neural network representation and Generative Adversarial Learning., Neural Netw, Vol: 139, Pages: 199-200
Calvi GG, Dees BS, Mandic DP, 2021, A lower bound on the tensor rank based on its maximally square matrix unfolding, SIGNAL PROCESSING, Vol: 180, ISSN: 0165-1684
Xu D, Zhang S, Zhang H, et al., 2021, Convergence of the RMSProp deep learning method with penalty for nonconvex optimization., Neural Netw, Vol: 139, Pages: 17-23
A norm version of the RMSProp algorithm with penalty (termed RMSPropW) is introduced into the deep learning framework and its convergence is addressed both analytically and numerically. For rigour, we consider the general nonconvex setting and prove the boundedness and convergence of the RMSPropW method in both deterministic and stochastic cases. This equips us with strict upper bounds on both the moving average squared norm of the gradient and the norm of weight parameters throughout the learning process, owing to the penalty term within the proposed cost function. In the deterministic (batch) case, the boundedness of the moving average squared norm of the gradient is employed to prove that the gradient sequence converges to zero when using a fixed step size, while with diminishing stepsizes, the minimum of the gradient sequence converges to zero. In the stochastic case, due to the boundedness of the weight evolution sequence, it is further shown that the weight sequence converges to a stationary point with probability 1. Finally, illustrative simulations are provided to support the theoretical analysis, including a comparison with the standard RMSProp on MNIST, CIFAR-10, and IMDB datasets.
Brajović M, Stanković I, Daković M, et al., 2021, On the Number of Channels in Multicomponent Nonstationary Noisy Signal Decomposition
If acquired using multiple sensors, non-stationary multicomponent signals can be decomposed into individual components by exploiting interdependences of signals from different channels. Earlier, we have proposed a decomposition approach being able to extract individual non-stationary signal components even in the challenging cases when their domains of support overlap in the time, frequency or joint time-frequency (TF) domains. The approach is based upon the eigenvalue analysis of the multichannel autocorrelation matrix and minimizations of concentration measures calculated using TF representations. In this paper, we investigate the influence of the number of sensors (channels) and external noise variance to the outcome of the decomposition process.
Haliassos A, Konstantinidis K, Mandic DP, 2021, Supervised Learning for Nonsequential Data: A Canonical Polyadic Decomposition Approach, IEEE Transactions on Neural Networks and Learning Systems, ISSN: 2162-237X
Efficient modeling of feature interactions underpins supervised learning for nonsequential tasks, characterized by a lack of inherent ordering of features (variables). The brute force approach of learning a parameter for each interaction of every order comes at an exponential computational and memory cost (curse of dimensionality). To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor, the order of which is equal to the number of features; for efficiency, it can be further factorized into a compact tensor train (TT) format. However, both TT and other tensor networks (TNs), such as tensor ring and hierarchical Tucker, are sensitive to the ordering of their indices (and hence to the features). To establish the desired invariance to feature ordering, we propose to represent the weight tensor through the canonical polyadic (CP) decomposition (CPD) and introduce the associated inference and learning algorithms, including suitable regularization and initialization schemes. It is demonstrated that the proposed CP-based predictor significantly outperforms other TN-based predictors on sparse data while exhibiting comparable performance on dense nonsequential tasks. Furthermore, for enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors. In conjunction with feature vector normalization, this is shown to yield dramatic improvements in performance for dense nonsequential tasks, matching models such as fully connected neural networks.
Gogineni VC, Talebi SP, Werner S, et al., 2020, Fractional-Order Correntropy Filters for Tracking Dynamic Systems in alpha-Stable Environments, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, Vol: 67, Pages: 3557-3561, ISSN: 1549-7747
- Author Web Link
- Cite
- Citations: 1
Stankovic L, Mandic D, Dakovic M, et al., 2020, Vertex-frequency graph signal processing: A comprehensive review, DIGITAL SIGNAL PROCESSING, Vol: 107, ISSN: 1051-2004
Phan A-H, Cichocki A, Uschmajew A, et al., 2020, Tensor Networks for Latent Variable Analysis: Novel Algorithms for Tensor Train Approximation, IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, Vol: 31, Pages: 4622-4636, ISSN: 2162-237X
- Author Web Link
- Cite
- Citations: 2
Cheng H, Xia Y, Lu Z, et al., 2020, Design of Improper Constellations for Optimal Data Rates in Downlink NOMA Systems, Pages: 436-441
This paper addresses the achievable rate improvement for the downlink two-user non-orthogonal multiple access (NOMA) system, in the context of imperfect successive interference cancellation (SIC), by means of improper signaling techniques. We investigate a basic scenario where the strong user transmits data from a proper constellation, subject to a minimum rate constraint, while the weak user adopts an improper signaling scheme. For rigor, the data rates are formulated in terms of the circularity coefficient of the improper constellation, under residual interference due to imperfect SIC. We find that such a scheme always increases the achievable rate of the strong user while the weak user may also benefit when the channel-to-noise ratios (CNRs) of the two users satisfy certain mild conditions. Moreover, analytical expressions for data rates are provided based on the power budget of the weak user, which allows us to find the optimal circularity coefficient for the best improper constellation mapping. Simulations on the downlink NOMA system support the analysis.
Talebi SP, Werner S, Mandic DP, 2020, Quaternion-Valued Distributed Filtering and Control, IEEE TRANSACTIONS ON AUTOMATIC CONTROL, Vol: 65, Pages: 4246-4257, ISSN: 0018-9286
- Author Web Link
- Cite
- Citations: 1
Zhang X, Xia Y, Li C, et al., 2020, Complex Properness Inspired Blind Adaptive Frequency-Dependent I/Q Imbalance Compensation for Wideband Direct-Conversion Receivers, IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, Vol: 19, Pages: 5982-5992, ISSN: 1536-1276
Davies HJ, Williams I, Peters NS, et al., 2020, In-ear SpO2: a tool for wearable, unobtrusive monitoring of core blood oxygen saturation, Sensors (Basel, Switzerland), Vol: 20, ISSN: 1424-8220
The non-invasive estimation of blood oxygen saturation (SpO2) by pulse oximetry is of vital importance clinically, from the detection of sleep apnea to the recent ambulatory monitoring of hypoxemia in the delayed post-infective phase of COVID-19. In this proof of concept study, we set out to establish the feasibility of SpO2 measurement from the ear canal as a convenient site for long term monitoring, and perform a comprehensive comparison with the right index finger-the conventional clinical measurement site. During resting blood oxygen saturation estimation, we found a root mean square difference of 1.47% between the two measurement sites, with a mean difference of 0.23% higher SpO2 in the right ear canal. Using breath holds, we observe the known phenomena of time delay between central circulation and peripheral circulation with a mean delay between the ear and finger of 12.4 s across all subjects. Furthermore, we document the lower photoplethysmogram amplitude from the ear canal and suggest ways to mitigate this issue. In conjunction with the well-known robustness to temperature induced vasoconstriction, this makes conclusive evidence for in-ear SpO2 monitoring being both convenient and superior to conventional finger measurement for continuous non-intrusive monitoring in both clinical and everyday-life settings.
Zhang X, Dees BS, Li C, et al., 2020, Analysis of Least Stochastic Entropy Adaptive Filters for Noncircular Gaussian Signals, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, Vol: 67, Pages: 1364-1368, ISSN: 1549-7747
- Author Web Link
- Cite
- Citations: 3
Phan A-H, Cichocki A, Oseledets I, et al., 2020, Tensor Networks for Latent Variable Analysis: Higher Order Canonical Polyadic Decomposition, IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, Vol: 31, Pages: 2174-2188, ISSN: 2162-237X
Stankovic L, Brajovic M, Dakovic M, et al., 2020, On the decomposition of multichannel nonstationary multicomponent signals, SIGNAL PROCESSING, Vol: 167, ISSN: 0165-1684
- Author Web Link
- Cite
- Citations: 3
Dees BS, Stankovic L, Constantinides AG, et al., 2020, PORTFOLIO CUTS: A GRAPH-THEORETIC FRAMEWORK TO DIVERSIFICATION, IEEE International Conference on Acoustics, Speech, and Signal Processing, Publisher: IEEE, Pages: 8454-8458, ISSN: 1520-6149
- Author Web Link
- Cite
- Citations: 1
Stankovic L, Dakovic M, Mandic D, et al., 2020, A LOW-DIMENSIONALITY METHOD FOR DATA-DRIVEN GRAPH LEARNING, IEEE International Conference on Acoustics, Speech, and Signal Processing, Publisher: IEEE, Pages: 5340-5344, ISSN: 1520-6149
- Author Web Link
- Cite
- Citations: 2
Kisil I, Calvi GG, Scalzo Dees B, et al., 2020, Tensor Decompositions and Practical Applications: A Hands-on Tutorial, Studies in Computational Intelligence, Pages: 69-97
The exponentially increasing availability of big and streaming data comes as a direct consequence of the rapid development and widespread use of multi-sensor technology. The quest to make sense of such large volume and variety of that has both highlighted the limitations of standard flat-view matrix models and the necessity to move toward more versatile data analysis tools. One such model which is naturally suited for data of large volume, variety and veracity are multi-way arrays or tensors. The associated tensor decompositions have been recognised as a viable way to break the “Curse of Dimensionality”, an exponential increase in data volume with the tensor order. Owing to a scalable way in which they deal with multi-way data and their ability to exploit inherent deep data structures when performing feature extraction, tensor decompositions have found application in a wide range of disciplines, from very theoretical ones, such as scientific computing and physics, to the more practical aspects of signal processing and machine learning. It is therefore both timely and important for a wider Data Analytics community to become acquainted with the fundamentals of such techniques. Thus, our aim is not only to provide a necessary theoretical background for multi-linear analysis but also to equip researches and interested readers with an easy to read and understand practical examples in form of a Python code snippets.
Stankovic L, Mandic DP, Dakovic M, et al., 2020, Demystifying the Coherence Index in Compressive Sensing [Lecture Notes], IEEE SIGNAL PROCESSING MAGAZINE, Vol: 37, Pages: 152-162, ISSN: 1053-5888
- Author Web Link
- Cite
- Citations: 5
Li Z, Deng W, Pei W, et al., 2020, SINR ANALYSIS OF MIMO SYSTEMS WITH WIDELY LINEAR MMSE RECEIVERS FOR THE RECEPTION OF REAL-VALUED CONSTELLATIONS, 31st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (IEEE PIMRC), Publisher: IEEE, ISSN: 2166-9570
Chen Z, Dees BS, Mandic DP, 2020, A Probabilistic Beat-to-Beat Filtering Model for Continuous and Accurate Blood Pressure Estimation, International Joint Conference on Neural Networks (IJCNN) held as part of the IEEE World Congress on Computational Intelligence (IEEE WCCI), Publisher: IEEE, ISSN: 2161-4393
Xiang M, Xia Y, Mandic DP, 2020, Performance Analysis of Deficient Length Quaternion Least Mean Square Adaptive Filters, IEEE TRANSACTIONS ON SIGNAL PROCESSING, Vol: 68, Pages: 65-80, ISSN: 1053-587X
- Author Web Link
- Cite
- Citations: 1
Menguc EC, Acir N, Mandic DP, 2020, Widely Linear Quaternion-Valued Least-Mean Kurtosis Algorithm, IEEE TRANSACTIONS ON SIGNAL PROCESSING, Vol: 68, Pages: 5914-5922, ISSN: 1053-587X
Gogineni VC, Talebi SP, Werner S, et al., 2020, Fractional-Order Correntropy Adaptive Filters for Distributed Processing of alpha-Stable Signals, IEEE SIGNAL PROCESSING LETTERS, Vol: 27, Pages: 1884-1888, ISSN: 1070-9908
Nakamura T, Alqurashi Y, Morrell M, et al., 2020, Hearables: automatic overnight sleep monitoring with standardised in-ear EEG sensor, IEEE Transactions on Biomedical Engineering, Vol: 67, Pages: 203-212, ISSN: 0018-9294
Objective: Advances in sensor miniaturisation and computational power have served as enabling technologies for monitoring human physiological conditions in real-world scenarios. Sleep disruption may impact neural function, and can be a symptom of both physical and mental disorders. This study proposes wearable in-ear electroencephalography (ear- EEG) for overnight sleep monitoring as a 24/7 continuous and unobtrusive technology for sleep quality assessment in the community. Methods: Twenty-two healthy participants took part in overnight sleep monitoring with simultaneous ear-EEG and conventional full polysomnography (PSG) recordings. The ear- EEG data were analysed in the both structural complexity and spectral domains; the extracted features were used for automatic sleep stage prediction through supervised machine learning, whereby the PSG data were manually scored by a sleep clinician. Results: The agreement between automatic sleep stage prediction based on ear-EEG from a single in-ear sensor and the hypnogram based on the full PSG was 74.1% in the accuracy over five sleep stage classification; this is supported by a Substantial Agreement in the kappa metric (0.61). Conclusion: The in-ear sensor is both feasible for monitoring overnight sleep outside the sleep laboratory and mitigates technical difficulties associated with scalp-EEG. It therefore represents a 24/7 continuously wearable alternative to conventional cumbersome and expensive sleep monitoring. Significance: The ‘standardised’ one-size-fits-all viscoelastic in-ear sensor is a next generation solution to monitor sleep - this technology promises to be a viable method for readily wearable sleep monitoring in the community, a key to affordable healthcare and future eHealth.
Stankovic L, Mandic D, Dakovic M, et al., 2020, Data Analytics on Graphs Part III: Machine Learning on Graphs, from Graph Topology to Applications, FOUNDATIONS AND TRENDS IN MACHINE LEARNING, Vol: 13, Pages: 332-530, ISSN: 1935-8237
Stankovic L, Mandic D, Dakovic M, et al., 2020, Data Analytics on Graphs Part II: Signals on Graphs, FOUNDATIONS AND TRENDS IN MACHINE LEARNING, Vol: 13, Pages: 158-331, ISSN: 1935-8237
Stankovic L, Mandic D, Dakovic M, et al., 2020, Data Analytics on Graphs Part I: Graphs and Spectra on Graphs, FOUNDATIONS AND TRENDS IN MACHINE LEARNING, Vol: 13, Pages: 1-157, ISSN: 1935-8237
Bacciu D, Mandic DP, 2020, Tensor decompositions in deep learning, Pages: 441-450
The paper surveys the topic of tensor decompositions in modern machine learning applications. It focuses on three active research topics of significant relevance for the community. After a brief review of consolidated works on multi-way data analysis, we consider the use of tensor decompositions in compressing the parameter space of deep learning models. Lastly, we discuss how tensor methods can be leveraged to yield richer adaptive representations of complex data, including structured information. The paper concludes with a discussion on interesting open research challenges.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.