693 results found
Liu J, Xu D, Lu Y, et al., 2023, Last-iterate convergence analysis of stochastic momentum methods for neural networks, Neurocomputing, Vol: 527, Pages: 27-35, ISSN: 0925-2312
The stochastic momentum method is a commonly used acceleration technique for solving large-scale stochastic optimization problems. Current convergence results of stochastic momentum methods under non-convex stochastic settings mostly discuss convergence in terms of the random output and minimum output, which requires temporal and spatial statistics of historical data. On the other hand, the last-iterate convergence allows us to avoid storing or selecting past output iterates after each iteration, while maintaining rigour in convergence analysis. To this end, we address the convergence of the last iterate output (called last-iterate convergence) of the stochastic momentum methods for non-convex stochastic optimization problems, in a way which is conformal with traditional optimization theory. For generality, we prove the last-iterate convergence of the stochastic momentum methods under a unified framework, covering both stochastic heavy ball momentum and stochastic Nesterov accelerated gradient momentum, whose momentum factors can be either constant or time-varying coefficients. Finally, the last-iterate convergence of the stochastic momentum methods is verified on the benchmark MNIST and CIFAR-10 datasets. The implementation of SUM is available at: https://github.com/xudp100/SUMhttps://github.com/xudp100/SUM.
Li S, Yu Z, Xiang M, et al., 2023, Reciprocal GAN Through Characteristic Functions (RCF-GAN), IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, Vol: 45, Pages: 2246-2263, ISSN: 0162-8828
Qing Z, Ni J, Li Z, et al., 2023, Performance analysis of the augmented complex-valued least mean kurtosis algorithm, SIGNAL PROCESSING, Vol: 203, ISSN: 0165-1684
Sau A, 2023, Artificial intelligence-enabled electrocardiogram to distinguish atrioventricular re-entrant tachycardia from atrioventricular nodal re-entrant tachycardia, Cardiovascular Digital Health Journal, Pages: 1-8, ISSN: 2666-6936
BackgroundAccurately determining arrhythmia mechanism from a 12-lead electrocardiogram (ECG) of supraventricular tachycardia can be challenging. We hypothesized a convolutional neural network (CNN) can be trained to classify atrioventricular re-entrant tachycardia (AVRT) vs atrioventricular nodal re-entrant tachycardia (AVNRT) from the 12-lead ECG, when using findings from the invasive electrophysiology (EP) study as the gold standard.MethodsWe trained a CNN on data from 124 patients undergoing EP studies with a final diagnosis of AVRT or AVNRT. A total of 4962 5-second 12-lead ECG segments were used for training. Each case was labeled AVRT or AVNRT based on the findings of the EP study. The model performance was evaluated against a hold-out test set of 31 patients and compared to an existing manual algorithm.ResultsThe model had an accuracy of 77.4% in distinguishing between AVRT and AVNRT. The area under the receiver operating characteristic curve was 0.80. In comparison, the existing manual algorithm achieved an accuracy of 67.7% on the same test set. Saliency mapping demonstrated the network used the expected sections of the ECGs for diagnoses; these were the QRS complexes that may contain retrograde P waves.ConclusionWe describe the first neural network trained to differentiate AVRT from AVNRT. Accurate diagnosis of arrhythmia mechanism from a 12-lead ECG could aid preprocedural counseling, consent, and procedure planning. The current accuracy from our neural network is modest but may be improved with a larger training dataset.
Liang Y, Xu D, Zhang N, et al., 2023, Almost sure convergence of stochastic composite objective mirror descent for non-convex non-smooth optimization, OPTIMIZATION LETTERS, ISSN: 1862-4472
Yarici MC, Thornton M, Mandic DP, 2023, Ear-EEG sensitivity modeling for neural sources and ocular artifacts, FRONTIERS IN NEUROSCIENCE, Vol: 16
Stankovic L, Mandic D, 2023, Convolutional Neural Networks Demystified: A Matched Filtering Perspective-Based Tutorial, IEEE Transactions on Systems, Man, and Cybernetics: Systems, ISSN: 2168-2216
Deep neural networks (DNNs) and especially convolutional neural networks (CNNs) have revolutionized the way we approach the analysis of large quantities of data. However, the largely ad hoc fashion of their development, albeit one reason for their rapid success, has also brought to light the intrinsic limitations of CNNs—in particular, those related to their black box nature. In addition, the ability to “explain” both the way such systems behave and the results they produce is increasingly becoming an imperative in many practical applications. Therefore, it would be particularly useful to establish physically meaningful mechanisms underpinning the operation of CNNs, thus helping to resolve the issue of interpretability of the processing steps and explain their input-output relationship. To this end, we revisit the operation of CNNs from first principles and show that their very backbone—the convolution operation—represents a matched filter which examines the input for the presence of characteristic patterns in data. Our treatment is based on temporal signals, naturally generated by physical sensors, which admit rigorous analysis through systems science. This serves as a vehicle for a unifying account on the overall functionality of CNNs, whereby both the convolution-activation-pooling chain and learning strategies are shown to admit a compact and elegant interpretation under the umbrella of matched filtering. In addition to helping reveal the physical principles underpinning CNNs and providing an intuitive understanding of their operation, the treatment of CNNs from a matched filtering perspective is also shown to offer a platform to support further developments in this area.
Talebi SP, Godsill SJ, Mandic DP, 2023, Filtering Structures or alpha-Stable Systems, IEEE CONTROL SYSTEMS LETTERS, Vol: 7, Pages: 553-558, ISSN: 2475-1456
Scalzo B, Stankovic L, Dakovic M, et al., 2023, A class of doubly stochastic shift operators for random graph signals and their boundedness, NEURAL NETWORKS, Vol: 158, Pages: 83-88, ISSN: 0893-6080
Lo Giudice M, Ferlazzo E, Mammone N, et al., 2022, Convolutional Neural Network Classification of Rest EEG Signals among People with Epilepsy, Psychogenic Non Epileptic Seizures and Control Subjects, INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH, Vol: 19
Roa-Vicens J, Xu YL, Silva R, et al., 2022, Graph and tensor-train recurrent neural networks for high-dimensional models of limit order books, Pages: 207-213
Recurrent neural networks (RNNs) have proven to be particularly effective for the paradigms of learning and modelling time series. However, sequential data of high dimensions are considerably more difficult and computationally expensive to model, as the number of parameters required to train the RNN grows exponentially with data dimensionality. This is also the case with time series from limit order books, the electronic registries where prices of securities are formed in public markets. To this end, tensorization of neural networks provides an efficient method to reduce the number of model parameters, and has been applied successfully to high-dimensional series such as video sequences and financial time series, for example, using tensor-train RNNs (TTRNNs). However, such TTRNNs suffer from a number of shortcomings, including: (i) model sensitivity to the ordering of core tensor contractions; (ii) training sensitivity to weight initialization; and (iii) exploding or vanishing gradient problems due to the recurrent propagation through the tensor-train topology. Recent studies showed that embedding a multi-linear graph filter to model RNN states (Recurrent Graph Tensor Network, RGTN) provides enhanced flexibility and expressive power to tensor networks, while mitigating the shortcomings of TTRNNs. In this paper, we demonstrate the advantages arising from the use of graph filters to model limit order book sequences of high dimension as compared with the state-of-the-art benchmarks. It is shown that the combination of the graph module (to mitigate problematic gradients) with the radial structure (to make the tensor network architecture flexible) results in substantial improvements in output variance, training time and number of parameters required, without any sacrifice in accuracy.
Powezka K, Pettipher A, Hemakom A, et al., 2022, A Pilot Study of Heart Rate Variability Synchrony as a Marker of Intraoperative Surgical Teamwork and Its Correlation to the Length of Procedure, SENSORS, Vol: 22
Xiao H, Chanwimalueang T, Mandic DP, 2022, Multivariate Multiscale Cosine Similarity Entropy and Its Application to Examine Circularity Properties in Division Algebras, ENTROPY, Vol: 24
- Author Web Link
- Citations: 1
Kisil I, Calvi GG, Konstantinidis K, et al., 2022, Accelerating Tensor Contraction Products via Tensor-Train Decomposition [Tips & Tricks], IEEE SIGNAL PROCESSING MAGAZINE, Vol: 39, Pages: 63-70, ISSN: 1053-5888
Davies HJ, Williams I, Hammour G, et al., 2022, In-ear SpO2 for classification of cognitive workload, IEEE Transactions on Cognitive and Developmental Systems, Pages: 1-1, ISSN: 2379-8920
The brain is the most metabolically active organ in the body, which increases its metabolic activity, and thus oxygen consumption, with increasing cognitive demand. This motivates us to question whether increased cognitive workload may be measurable through changes in blood oxygen saturation. To this end, we explore the feasibility of cognitive workload tracking based on in-ear SpO2 measurements, which are known to be both robust and exhibit minimal delay. We consider cognitive workload assessment based on an N-back task with randomised order. It is shown that the 2-back and 3-back tasks (high cognitive workload) yield either the lowest median absolute SpO2 or largest median decrease in SpO2 in all of the subjects, indicating a measurable and statistically significant decrease in blood oxygen in response to increased cognitive workload. This makes it possible to classify the four N-back task categories, over 5 second epochs, with a mean accuracy of 90.6%, using features derived from in-ear pulse oximetry, including SpO2, pulse rate and respiration rate. These findings suggest that in-ear SpO2 measurements provide sufficient information for the reliable classification of cognitive workload over short time windows, which promises a new avenue for real time cognitive workload tracking.
Thornton M, Mandic D, Reichenbach T, 2022, Robust decoding of the speech envelope from EEG recordings through deep neural networks, JOURNAL OF NEURAL ENGINEERING, Vol: 19, ISSN: 1741-2560
Davies HJ, Williams I, Mandic DP, 2022, Tracking Cognitive Workload in Gaming with In-Ear [Formula: see text]., Annu Int Conf IEEE Eng Med Biol Soc, Vol: 2022, Pages: 4913-4916
The feasibility of using in-ear [Formula: see text] to track cognitive workload induced by gaming is investigated. This is achieved by examining temporal variations in cognitive workload through the game Geometry Dash, with 250 trials across 7 subjects. The relationship between performance and cognitive load in Dark Souls III boss fights is also investigated followed by a comparison of the cognitive workload responses across three different genres of game. A robust decrease in in-ear [Formula: see text] is observed in response to cognitive workload induced by gaming, which is consistent with existing results from memory tasks. The results tentatively suggest that in-ear [Formula: see text] may be able to distinguish cognitive workload alone, whereas heart rate and breathing rate respond similarly to both cognitive workload and stress. This study demonstrates the feasibility of low cost wearable cognitive workload tracking in gaming with in-ear [Formula: see text], with applications to the play testing of games and biofeedback in games of the future.
Normahani P, Sounderajah V, Mandic D, et al., 2022, Machine learning-based classification of arterial spectral waveforms for the diagnosis of peripheral artery disease in the context of diabetes: A proof-of-concept study, VASCULAR MEDICINE, Vol: 27, Pages: 450-456, ISSN: 1358-863X
Menguc EC, Xiang M, Mandic DP, 2022, Online censoring based complex-valued adaptive filters, SIGNAL PROCESSING, Vol: 200, ISSN: 0165-1684
Took CC, Mandic D, 2022, Weight sharing for LMS algorithms: Convolutional neural networks inspired multichannel adaptive filtering, DIGITAL SIGNAL PROCESSING, Vol: 127, ISSN: 1051-2004
- Author Web Link
- Citations: 2
Li S, Mandic D, 2022, Von Mises-Fisher Elliptical Distribution, IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, ISSN: 2162-237X
Prochazka A, Charvat J, Vysata O, et al., 2022, Incremental deep learning for reflectivity data recognition in stomatology, NEURAL COMPUTING & APPLICATIONS, Vol: 34, Pages: 7081-7089, ISSN: 0941-0643
Davies HJ, Bachtiger P, Williams I, et al., 2022, Wearable in-ear PPG: detailed respiratory variations enable classification of COPD, IEEE Transactions on Biomedical Engineering, Vol: 69, ISSN: 0018-9294
An ability to extract detailed spirometry-like breath-ing waveforms from wearable sensors promises to greatly improve respiratory health monitoring. Photoplethysmography (PPG) has been researched in depth for estimation of respiration rate, given that it varies with respiration through overall intensity, pulse amplitude and pulse interval. We compare and contrast the extraction of these three respiratory modes from both the ear canal and finger and show a marked improvement in the respiratory power for respiration induced intensity variations and pulse amplitude variations when recording from the ear canal. We next employ a data driven multi-scale method, noise assisted multivariate empirical mode decomposition (NA-MEMD), which allows for simultaneous analysis of all three respiratory modes to extract detailed respiratory waveforms from in-ear PPG. For rigour, we considered in-ear PPG recordings from healthy subjects, both older and young, patients with chronic obstructive pulmonary disease (COPD) and idiopathic pulmonary fibrosis (IPF) and healthy subjects with artificially obstructed breathing. Specific in-ear PPG waveform changes are observed for COPD, such as a decreased inspiratory duty cycle and an increased inspiratory magnitude, when compared with expiratory magnitude. These differences are used to classify COPD from healthy and IPF waveforms with a sensitivity of 87% and an overall accuracy of 92%. Our findings indicate the promise of in-ear PPG for COPD screening and unobtrusive respiratory monitoring in ambulatory scenarios and in consumer wearables.
Michalas L, Konstantinidis K, Farinelli P, et al., 2022, RF MEMS Switch Design Methodology by Electromagnetic Simulations and Machine Learning, 52nd European Microwave Conference (EuMC), Publisher: IEEE, Pages: 369-372, ISSN: 2325-0305
Occhipinti E, Davies HJ, Hammour G, et al., 2022, Hearables: Artefact removal in Ear-EEG for continuous 24/7 monitoring, IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) / IEEE World Congress on Computational Intelligence (IEEE WCCI) / International Joint Conference on Neural Networks (IJCNN) / IEEE Congress on Evolutionary Computation (IEEE CEC), Publisher: IEEE, ISSN: 2161-4393
Talebi SP, Werner S, Mandic DP, 2022, Fractional-Order Learning Systems, IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) / IEEE World Congress on Computational Intelligence (IEEE WCCI) / International Joint Conference on Neural Networks (IJCNN) / IEEE Congress on Evolutionary Computation (IEEE CEC), Publisher: IEEE, ISSN: 2161-4393
Talebi SP, Werner S, Xia Y, et al., 2022, A Joint Particle Filter for Quaternion-Valued alpha-Stable Signals via the Characteristic Function, IEEE 12th Sensor Array and Multichannel Signal Processing Workshop (SAM), Publisher: IEEE, Pages: 390-394
Arroyo A, Scalzo B, Stankovic L, et al., 2022, DYNAMIC PORTFOLIO CUTS: A SPECTRAL APPROACH TO GRAPH-THEORETIC DIVERSIFICATION, 47th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Publisher: IEEE, Pages: 5468-5472, ISSN: 1520-6149
Konstantinidis K, Xu YL, Zhao Q, et al., 2022, VARIATIONAL BAYESIAN TENSOR NETWORKS WITH STRUCTURED POSTERIORS, 47th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Publisher: IEEE, Pages: 3638-3642, ISSN: 1520-6149
Chen Z, Tan X, Wang K, et al., 2022, INFERGRAD: IMPROVING DIFFUSION MODELS FOR VOCODER BY CONSIDERING INFERENCE IN TRAINING, 47th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Publisher: IEEE, Pages: 8432-8436, ISSN: 1520-6149
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.