Transparent Intelligence Publishing Transparent Intelligence Publishing

Search

Search results:
Brain-Wide Mental State Analysis Using Explainable Shallow Neural Networks on HCP fMRI Data
Artificial neural networks (ANNs) have frequently been deployed for classification objectives within functional magnetic resonance imaging (fMRI) research, though such analyses traditionally isolate only specific regional sub-volumes of the brain. Recent methodology has integrated shallow neural networks (SNNs) with explainable artificial intelligence (xAI) frameworks to uncover underlying mechanisms of neural processing. Although prior research validated this paradigm utilizing fMRI data from motor execution tasks, the current investigation implements this framework on Theory of Mind (ToM) cognitive tasks, leveraging data extracted from the Human Connectome Project (HCP) Young Adult repository. Cognitive paradigms introduce greater computational complexity due to the fundamentally non-linear dynamics of neural functions. The HCP multimodal parcellation brain atlas is utilized to partition the cortex, thereby guiding the systematic training, pruning, and subsequent retraining of an SNN. Shapley values are subsequently computed to decode the retrained architecture, with the resulting outputs benchmarked against a General Linear Model (GLM) analysis for empirical validation. The baseline network demonstrated a classification accuracy of 88.2%, which decreased to 80.0% following network pruning, and subsequently recovered to 84.7% after the retraining phase. The SHAP-derived explanations demonstrated strong convergence with both GLM observations and established neuroanatomical regions associated with ToM. This fMRI investigation effectively navigated a highly complex cognitive paradigm, illustrating the efficacy of explainability techniques in deciphering non-linear neurological dynamics. These outcomes indicate that xAI platforms, particularly knowledge extraction frameworks, offer substantial utility for progressing mental health research and neural state decoding applications.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2022 | Article: 13

Reducing Complexity in Neural Network Interpretation Using Visual Analytics and Neuron Clustering
Deep learning (DL) architectures have established state-of-the-art capabilities across numerous fields. Unpacking their internal operations and reasoning paths is critical due to their high architectural complexity and opaque, black-box character, particularly in high-stakes fields like healthcare. Visual analytics (VA) integrated with DL methodologies are frequently employed to extract data insights, yet they regularly face challenges related to visual clutter (VC). This research introduces a streamlined neural network (NN) view design engineered to mitigate visual clutter while clarifying DL model elements for field specialists and everyday users. We applied clustering techniques to aggregate hidden neurons according to similarities in their activation profiles. This framework accommodates both macroscopic and micro-level perspectives of the generated neuron groups. Utilizing a tabular medical dataset as a practical case study, this clustered visualization strategy successfully decreased visual clutter by 54% for neuron graphics and by 88.7% for their interconnections, while successfully exposing correlated neuron activations developed throughout model training.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2023 | Article: 38

Enhancing Trustworthy Graph Neural Network Explanations through Adversarial Generation
Graph neural networks (GNNs) have been successfully deployed across numerous practical domains, yet the precise mechanisms underlying their operations remain largely opaque. In response to this lack of clarity and to encourage the deployment of reliable AI systems, a variety of GNN explanation tools have been developed. Nonetheless, current interpretation techniques frequently encounter severe bottlenecks, including: (1) explanations that are restricted to localized individual instances; (2) poor generalizability when applied to unfamiliar graph structures; (3) the risk of creating structurally invalid graphs; and (4) confinement to specific task domains (such as only node or only graph classification). To resolve these deficiencies, we introduce a novel interpretation framework, GAN-GNNExplainer, which integrates a generator to construct explanations alongside a discriminator to supervise the production stage, thereby enhancing output dependability. Although advantageous, GAN-GNNExplainer still exhibits limitations in producing highly faithful explanations and shows suboptimal performance when tested on empirical datasets. To rectify these weaknesses, we present ACGAN-GNNExplainer, an advanced iteration that enhances GAN-GNNExplainer by utilizing a more robust discriminator to continuously monitor the generation phase, which yields explanatory outputs that are both dependable and faithful. Comprehensive empirical evaluations conducted on both artificial and empirical graph datasets confirm that our proposed methodologies outperform conventional GNN explanation frameworks.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 47

A Novel Dynamic Programming Approach for Generating Sparse Adversarial Examples in Deep Neural Networks
Recent investigations have demonstrated that deep neural networks are susceptible to subtly modified input data. This paper introduces a novel untargeted white-box adversarial attack termed the dynamic programming-based sub-pixel score method (SPSM) attack (DPSPSM). This technique modifies conventional gradient-based white-box adversarial methods, which are typically restricted by a predetermined Hamming distance, by incorporating a dynamic programming framework. The method is driven by a pixel score metric approach, the SPSM, which is developed herein. Unlike traditional gradient-based adversarial attacks that introduce nearly imperceptible modifications across the entire image, DPSPSM operates rapidly and maintains robustness by altering only a restricted number of input pixels. The introduced algorithm quantizes the gradient updates using a score computed for every pixel, which aggregates the contributions across all channels. Experimental outcomes indicate that the DPSPSM successfully misleads the model, achieving a success rate of 30.45% on the CIFAR-10 test set and 29.30% on the CIFAR-100 test set.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 50

Interpreting Convolutional Neural Network Classifications of Raman Spectra: Insights from Fungal Spores and Carotenoid-Pigmented Microorganisms
A multitude of studies demonstrate that robust predictive models for identifying microorganisms can be successfully developed by combining Raman micro-spectroscopy with chemometric techniques, frequently yielding highly accurate outcomes. Progress in machine learning alongside increased software availability has simplified the process for researchers to construct predictive frameworks from intricate data arrays. Conversely, the underlying reasons for the high accuracy of these classifications remain under-investigated. This study analyzes Raman spectroscopic datasets from fungal spores and carotenoid-producing microorganisms to demonstrate that successful differentiation is frequently independent of peak positions or subtle variations in spectral band ratios caused by minor biochemical differences among the organisms. Instead, distinctive baseline variations in biochemically related microorganisms, which may be amplified by specific data preprocessing routines, play a critical role; furthermore, seemingly neutral spectral intervals can significantly influence a convolutional neural network. By applying a technique known as Gradient-weighted Class Activation Mapping, we investigate the internal mechanics of convolutional neural networks within microbiological domains to determine the precise Raman spectral bands driving successful classification.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 57