Transparent Intelligence Publishing Transparent Intelligence Publishing

Search

Search results:
A Decade of Explainable and Interpretable Artificial Intelligence in Healthcare: A Bibliometric Perspective
This review investigates the expanding influence of machine learning and deep learning methodologies within the healthcare sector, concentrating specifically on the vital challenges of interpretability and explainability inherent in black-box models. Although machine learning frameworks are increasingly deployed to assist in medical analytics and diagnostics, their intricate nature emphasizes the necessity of comprehending how these systems process and interpret data to facilitate well-informed clinical choices. This paper offers a thorough examination of the obstacles and potential solutions documented in existing scholarship, delivering a synthesis of contemporary techniques in this domain. Furthermore, it establishes precise conceptual boundaries for interpretability and explainability, seeking to untangle the differences between these notions and outline their consequences for clinical decision-making. Utilizing a dataset of 448 publications to address seven distinct research questions, our investigation demonstrates that this research area has experienced exponential growth over the past ten years. Additionally, the psychological factors governing public opinion highlight the vital need for transparent communication regarding the capabilities and constraints of artificial intelligence. To mitigate opacity, investigators are actively engineering strategies to lower model complexity and implement visual explanation tools. Nevertheless, maintaining a delicate equilibrium between high predictive performance and adequate model interpretability remains an ongoing challenge. Given the expanding role of artificial intelligence in supporting clinical diagnostics and therapeutics, the development of inherently interpretable artificial intelligence systems is critical. Within this rapidly changing paradigm, a steadfast dedication to transparency, ethical standards, and cross-disciplinary partnerships is essential to guarantee the conscientious deployment of artificial intelligence. This shared dedication is foundational to cementing long-term trust between medical professionals and patients, resolving novel problems, and promoting the educated integration of these sophisticated technologies into medical practices.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 41

A Systematic Survey of Automated Visual Inspection Techniques for Printed Circuit Boards
The printed circuit board (PCB) serves as a fundamental component across all electronic devices, and the assurance of its fabrication quality exerts a major influence on the final pricing of the product. Consequently, research and development (R&D) focused on optimizing inspection techniques has emerged as a expanding area of global interest. Among the various paradigms utilized for PCB evaluation, visual analysis constitutes the preponderance of contemporary methodologies. To that end, this survey aggregates the most influential investigations conducted over the past quarter-century, outlining their operational frameworks and historical progression up to the present day, with a particular emphasis on the integration of artificial intelligence (AI) methodologies to enhance functional efficacy and minimize operational expenditures. Additionally, this study underscores the vital role that open-access PCB datasets play in facilitating the advancement of these techniques and establishing baseline metrics for benchmarking. Lastly, the current difficulties and obstacles confronting the discipline are delineated and analyzed.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 42

Artificial Intelligence Regulation in Financial Services: Mapping Principles, Priorities, and Compliance Requirements
The growing demand for automation within the financial sector creates an ideal environment for utilizing artificial intelligence. However, stringent and intricate regulatory mandates alongside rapid technical progress present substantial hurdles for the creation and deployment of AI-driven financial services. To foster comprehension, safeguard consumers, and maintain service quality in this sector, the regulatory guidelines established by European authorities must be organized into highly detailed components. This categorization will offer deeper insights into regulatory priorities and provide a structured approach to engineering AI-driven solutions. This study introduces a classification framework based on a taxonomy designed to demystify current European regulatory tenets for academics, supervisory bodies, and financial enterprises. By highlighting the core thematic focal points of these regulations, our research aims to facilitate the engineering of compliant AI services.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 43

Bridging AI Innovation and Regulatory Compliance in Financial Services: A Taxonomy of Regulatory Principles
The critical nature of explainable artificial intelligence and machine learning (XAI/XML) is gaining widespread recognition. This field focuses on uncovering how input variables influence automated choices, identifying systemic algorithmic bias, and detecting vulnerability to anomalies in data. Much of the existing literature centers on creating post hoc explanations tailored for opaque, black box architectures. These secondary techniques introduce novel vulnerabilities to error without correcting the fundamental flaws of the core model. Conversely, the development of inherently transparent systems receives far less attention. This paper introduces an inherently interpretable strategy rooted in the principles of ensemble learning, designated as the League of Experts (LoE) framework. We construct the initial theoretical foundations and subsequently derive a flexible, modular meta-algorithm. Although our presentation focuses primarily on classification tasks, the LoE framework adapts identically to regression scenarios. For classification specifically, we deploy traditional decision trees to form classifier ensembles as a concrete implementation. This paradigm simplifies the extraction of human-comprehensible decision rules for the underlying classification task, yielding an independent rule-induction system called RuleLoE. Beyond evaluating 12 KEEL classification datasets, we validate the LoE framework using two benchmarks from highly critical domains—healthcare and finance—to demonstrate its utility. The predictive capability of LoE matches contemporary state-of-the-art classification algorithms regarding both accuracy and rule coverage. Furthermore, LoE outputs a highly transparent group of decision rules featuring customizable complexity to map out the target classification problem. LoE provides a dependable approach for handling classification and regression problems. Its predictive precision is highly suited for environments where discovering root causal mechanisms is the primary objective, rather than merely maximizing predictive or classification performance.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 44

Towards Trustworthy Brain Tumor Diagnosis: An Explainable Deep Learning Framework for MRI Analysis
Among the most critical and fatal medical conditions are brain malignancies, making early identification vital for enhancing survival rates. Currently, magnetic resonance imaging (MRI) stands as the premier diagnostic tool for early detection because of its exceptional capability to capture soft tissue details. Nevertheless, relying solely on human interpretation of brain MRI scans introduces risks of oversight, which are frequently exacerbated by the fatigue and varying expertise of radiologists. To mitigate these limitations, computer-aided diagnosis (CAD) systems have taken on an increasingly vital role. Leveraging state-of-the-art computer vision methodologies like deep learning, these systems generate highly precise diagnoses from clinical imagery, thereby boosting both precision and consistency in clinical settings. This study introduces an innovative CAD architecture designed for multi-class classification of brain malignancies. The system evaluates six pre-trained deep learning networks as its foundational architecture, integrating robust data preprocessing and augmentation protocols to optimize computational throughput. To counter the opaque "black box" nature of deep learning architectures and improve interpretability, Gradient-weighted Class Activation Mapping (Grad-CAM) is integrated to illuminate the network's deductive steps when identifying anomalies in MRI data. Furthermore, an accessible, web-based Brain Tumor Detection System was constructed using the Streamlit framework, highlighting its utility for clinical environments and offering meaningful support to medical professionals. All experimental simulations were carried out using an open benchmark repository, proving that the introduced system delivers state-of-the-art diagnostic capabilities, achieving accuracy levels near 99% with the ResNet-50, Xception, and InceptionV3 architectures.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 45

The Anyon Hypothesis: A Topological Quantum Framework for Interpreting Large Language Models
While large language models (LLMs) demonstrate exceptional predictive capabilities, their internal reasoning mechanisms remain obscure, creating an urgent demand for enhanced interpretability in artificial intelligence. In this paper, we suggest addressing this explanatory deficit by incorporating principles from topological quantum computing (TQC), specifically the anyonic structures derived from S(2)k theories. Anyons serve as an interpolation between fermions and bosons, offering a formal mathematical paradigm that could clarify the latent configurations and decision-making pathways within LLMs. By investigating the relationships between these topological models, token interactions, and contextual dependencies within neural systems, we seek to offer an alternative viewpoint on the emergence of meaning and coherence. Drawing upon insights obtained from ChatGPT alongside an examination of low-level instances of S(2)k models, we propose that the formal machinery of modular tensor categories and topological phases could guide the development of clearer, more stable, and more resilient AI frameworks. This cross-disciplinary approach indicates that quantum-theoretic concepts may provide the foundation for a novel paradigm in explainable AI.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 46

Enhancing Trustworthy Graph Neural Network Explanations through Adversarial Generation
Graph neural networks (GNNs) have been successfully deployed across numerous practical domains, yet the precise mechanisms underlying their operations remain largely opaque. In response to this lack of clarity and to encourage the deployment of reliable AI systems, a variety of GNN explanation tools have been developed. Nonetheless, current interpretation techniques frequently encounter severe bottlenecks, including: (1) explanations that are restricted to localized individual instances; (2) poor generalizability when applied to unfamiliar graph structures; (3) the risk of creating structurally invalid graphs; and (4) confinement to specific task domains (such as only node or only graph classification). To resolve these deficiencies, we introduce a novel interpretation framework, GAN-GNNExplainer, which integrates a generator to construct explanations alongside a discriminator to supervise the production stage, thereby enhancing output dependability. Although advantageous, GAN-GNNExplainer still exhibits limitations in producing highly faithful explanations and shows suboptimal performance when tested on empirical datasets. To rectify these weaknesses, we present ACGAN-GNNExplainer, an advanced iteration that enhances GAN-GNNExplainer by utilizing a more robust discriminator to continuously monitor the generation phase, which yields explanatory outputs that are both dependable and faithful. Comprehensive empirical evaluations conducted on both artificial and empirical graph datasets confirm that our proposed methodologies outperform conventional GNN explanation frameworks.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 47

From Data to Arguments: A Unified Paradigm for Explainable Artificial Intelligence
Explainable Artificial Intelligence (XAI) represents an active discipline dedicated to clarifying the internal mechanisms of AI systems, thereby fostering stakeholder confidence and advancing the deployment of accountable technology. A primary theoretical hurdle within XAI involves engineering techniques capable of producing clear, accessible interpretations without sacrificing scalability or operational performance in intricate domains. While rule-based methodologies offer a framework for decoding AI-driven conclusions, they are prone to becoming overly complex in dense environments, which diminishes both their comprehensibility and computational scalability. Furthermore, these approaches frequently overlook contrastive logic, which leaves observers unable to discern why a specific outcome was prioritized over alternatives. To resolve this underlying scientific problem, this research investigates the fusion of computational argumentation—a specialized domain of AI that replicates cognitive processes through defeasible reasoning—with rule-based XAI architectures. Computational argumentation provides a mechanism whereby arguments derived from structured rules can be invalidated or withdrawn when confronted with conflicting evidence. Consequently, this framework offers a compelling avenue for refining rule-based systems to yield more transparent AI models. Despite the synergistic benefits of combining rule-based structures with computational argumentation, academic inquiry into their synthesis remains sparse. Accordingly, this paper also targets the practical engineering challenge of realizing this conceptual integration within functional AI software. This investigation utilizes the Logic Learning Machine (LLM)—a specialized rule-extraction methodology—and introduces a modular framework designed to map input rules directly into a structured argumentation network via state-of-the-art argumentation techniques. Empirical evaluations performed on various binary classification problems from the UCI Machine Learning Repository validate the utility of this combined model. The LLM strategy proved highly proficient at extracting a compact, interpretable collection of if-then rules characterized by minimal antecedent premises, all while sustaining strong predictive accuracy across all evaluated datasets. Concurrently, the argumentation-based models delivered classification performance comparable to the native if-then rule configurations, achieving this through a streamlined rule set while offering vastly superior explainability. Ultimately, this work delivers an innovative framework for the automated and streamlined generation of arguments and their internal conflicts directly from raw data, simultaneously addressing fundamental theoretical and practical barriers to propagate argumentation frameworks within the XAI landscape.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 48

Explainable Machine Learning for Early Prediction of Unscheduled Hospital Readmissions: Integrating Global and Local Interpretability
At present, the majority of healthcare spending is driven by chronic patients who experience multiple readmissions due to their underlying conditions. Implementing individualized prevention frameworks could optimize the clinical management of this demographic. This study aimed to generate localized predictive models utilizing interpretable machine learning approaches to provide early identification of individual unscheduled hospital readmissions. To achieve this, a retrospective case-control investigation was performed, utilizing patient readmission data spanning 2018–2019. Following the cleaning and curation of the initial data repository (n = 76,210), the final sample size comprised n = 29,026 participants. A machine learning analysis evaluated multiple algorithms, employing unscheduled hospital readmission as the dependent variable, alongside local model-agnostic interpretability techniques.The results demonstrated a 13% rate of unscheduled hospital readmissions. Statistically significant variations were identified regarding patient age and length of hospital stay (p < 0.001 in both instances). A logistic regression framework highlighted chronic therapy (odds ratio: 3.75), a history of diabetes mellitus (odds ratio: 1.14), and length of stay (odds ratio: 1.02) as statistically meaningful predictors. Machine learning algorithms demonstrated superior performance concerning sensitivity and complementary evaluation metrics. Within this analytical framework, length of stay and patient age emerged as the primary determinants for forecasting unscheduled readmissions. Notably, other clinical indicators, such as documented allergies and histories of adverse drug reactions, also demonstrated predictive relevance. Individualized prognostic models exhibited high sensitivity. In conclusion, this investigation identified critical determinants governing unscheduled hospital readmissions, highlighting the prominence of age and hospitalization duration. A personalized risk assessment framework capable of predicting readmissions with high accuracy was established. Future investigations should incorporate a broader array of clinical parameters to further optimize the predictive capacity of this model.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 49

A Novel Dynamic Programming Approach for Generating Sparse Adversarial Examples in Deep Neural Networks
Recent investigations have demonstrated that deep neural networks are susceptible to subtly modified input data. This paper introduces a novel untargeted white-box adversarial attack termed the dynamic programming-based sub-pixel score method (SPSM) attack (DPSPSM). This technique modifies conventional gradient-based white-box adversarial methods, which are typically restricted by a predetermined Hamming distance, by incorporating a dynamic programming framework. The method is driven by a pixel score metric approach, the SPSM, which is developed herein. Unlike traditional gradient-based adversarial attacks that introduce nearly imperceptible modifications across the entire image, DPSPSM operates rapidly and maintains robustness by altering only a restricted number of input pixels. The introduced algorithm quantizes the gradient updates using a score computed for every pixel, which aggregates the contributions across all channels. Experimental outcomes indicate that the DPSPSM successfully misleads the model, achieving a success rate of 30.45% on the CIFAR-10 test set and 29.30% on the CIFAR-100 test set.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 50

Leveraging Chain-of-Thought Prompt Engineering to Enhance Knowledge Creation through Improvable Student Ideas
Within a student learning community, the generation of new knowledge represents a vital mechanism for expanding collective understanding and driving original synthesis. Students assume critical responsibilities when they target knowledge deficiencies and engage in cooperative efforts to refine communal concepts emerging from dialogue. Nonetheless, the standard of these concepts can frequently be constrained by resource deficits or a narrow spectrum of viewpoints. The introduction of generative Artificial Intelligence and large language models (LLMs) into educational settings has unlocked new opportunities for analyzing and enriching idea-focused debates that were formerly unachievable. Maximizing the utility of LLMs necessitates specialized competencies in prompt engineering, particularly regarding the in-context methodology termed Chain-of-Thought (CoT) to formulate and scaffold progressive concepts within student conversations. This investigation gathered a dataset of 721 conversational entries—encompassing 272 applicable question–answer dynamics and 149 distinct discussion tracks—produced by 31 participants during a 2-day student Knowledge Building Design Studio (sKBDS). The researchers augmented student inputs utilizing the CoT technique and subsequently evaluated the AI-generated outputs against the original student assertions. The outcomes are demonstrated through two explicit conversational tracks, confirming that CoT-configured prompts enable LLMs to output responses that actively advance malleable concepts during knowledge construction. This inquiry examines data from genuine classroom discourse and offers actionable insights for both educational research and instructional implementation.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 51

Visual Cryptography Meets Remote Sensing Intelligence: A Secure and Efficient Recognition Framework
With the emergence of novel data collection platforms driven by the Internet of Things (IoT), this study expands beyond the conventional boundaries of traditional remote sensing technologies. The deep integration of remote sensing and computer vision has transitioned into industrial applications, enabling the utilization of Artificial Intelligence to address challenges like automated information extraction and image analysis. Nonetheless, due to the intricate architecture of IoT and the absence of standardized security protocols, remote sensing instruments are highly susceptible to privacy breaches during data transmission. Because traditional cryptographic approaches depend heavily on computational complexity, it is critical to develop security frameworks tailored for resource-constrained IoT devices. Visual Cryptography (VC) serves as an image-based threshold technique that allows direct decoding by the human visual system simply by overlaying encrypted shares. This "stacking-to-see" attribute, combined with basic Boolean decryption processes, positions VC as an optimal candidate for privacy-preserving identification within extensive IoT-based remote sensing imagery. In this investigation, the secure and productive dissemination of high-resolution remote sensing imagery is accomplished via meaningful VC. By distributing the error between the encryption block and the source block to neighboring blocks, the fidelity degradation in the reconstructed images is minimized. Furthermore, by adjusting a pre-trained model using large-scale datasets, the recognition accuracy of small, encrypted remote sensing datasets is enhanced. The empirical findings demonstrate that the introduced lightweight, privacy-preserving recognition system maintains superior identification performance while strengthening data security.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 52

Explainable Machine Learning for Long-Term Forecasting of Patient Demand in Urgent Care Clinics
Global emergency departments and urgent care facilities regularly experience prolonged waiting periods that surpass patient expectations, primarily driven by unexpected spikes in patient volumes. The resulting delays caused by insufficient staffing during these high-demand intervals have been associated with detrimental clinical consequences. Prior investigations into predicting patient volumes have predominantly relied on traditional statistical approaches. Furthermore, these efforts have mainly focused on short-range predictions, which offer minimal utility for the long-term scheduling of medical staff. This paper contributes to a growing domain of research aimed at investigating the capacity of machine learning frameworks to produce precise patient volume projections. Utilizing a decade's worth of data from two major urgent care centers, this study constructs long-range patient flow predictions extending up to one quarter in advance by evaluating a selection of modern, high-performing algorithms. A unique aspect of this research is the integration of eXplainable Artificial Intelligence (XAI) frameworks, specifically LIME and Shapley, to conduct a comprehensive assessment of internal model dynamics that are typically obscured. These diagnostic instruments allowed us to examine how effectively the predictive frameworks adjusted to volatile demand shifts induced by COVID-19 pandemic lockdowns, while also determining the most significant predictor variables to gain deep operational insights. The empirical findings indicate that an innovative ensemble combining gradient boosting techniques with advanced univariate frameworks like Prophet yielded the most precise and dependable performance overall. On average, this methodology produced a 16%–30% improvement over the current internal forecasting techniques utilized for predicting daily patient volumes 90 days into the future.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 53

Assessing the Fitness of Deep Learning for Combined Sewer Overflow Prediction Using a Problem Frames Approach
Unchecked discharges of untreated water from combined sewer systems pose critical hazards to public health and ecosystems. To address this, regional sanitation authorities like the Metropolitan Sewer District of Greater Cincinnati (MSDGC) have begun collecting massive streams of hydrological telemetry, looking toward deep learning (DL) methodologies—specifically recurrent neural networks (RNN)—to forecast these spill events. Crucially, determining whether a DL model is truly suitable for this domain requires a structured analysis of the environment in which it operates. This paper introduces a requirements engineering methodology grounded in problem frames to map out and organize stakeholder concerns. This approach systematically evaluates real-world conditions where standard assumptions of high-quality data break down, translating these insights into actionable software testing benchmarks known as metamorphic relations, which govern both data input adjustments and output validations. Deploying this methodology within MSDGC’s operational context provides a rigorous, standardized baseline to assess how effectively competing RNN designs fulfill core operational mandates.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 54

Explainable CNN-Based Cavitation Detection in Axial Piston Pumps Using Grad-CAM Visualization
The reliability and safety of an entire hydraulic system are compromised by the occurrence of cavitation in axial piston pumps. Because vibration signals can indicate cavitation states within these pumps, they have been increasingly coupled with machine learning techniques for detection purposes. Nevertheless, raw vibration signals captured under actual operating conditions are frequently corrupted by noise, complicating the precise identification of cavitation within noisy environments. To address this issue, this study introduces an intelligent methodology designed to identify cavitation in axial piston pumps under high-noise conditions. Initially, a convolutional neural network (CNN) is trained utilizing spectrogram images that are converted from raw, unstructured vibration datasets across varying cavitation states. Subsequently, the gradient-weighted class activation mapping (Grad-CAM) technique is applied to highlight and visualize the class-discriminative zones within these spectrogram images. Ultimately, a novel image processing approach leveraging the Grad-CAM heatmap is introduced to automatically filter out background noise and amplify the key class features in the spectrograms. The empirical outcomes demonstrate that this strategy substantially boosts the diagnostic efficacy of the CNN framework under noisy conditions. Specifically, the accuracy of cavitation condition classification is elevated from 0.50 to 0.89 at a signal-to-noise ratio of 4 dB, and from 0.80 to 0.92 at 6 dB.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 55

Toward Fair AI Development: How Developers Can Address Algorithmic Bias
Artificial intelligence (AI) has swiftly emerged as a core technology for securing a competitive advantage. Nonetheless, there are escalating anxieties regarding bias within AI models, given that AI developers are susceptible to introducing bias both inadvertently and intentionally. Utilizing a qualitative methodology, this study explored how AI developers can actively foster the creation of equitable AI models. The primary results indicate that the danger of bias stems predominantly from a deficiency in social and gender diversity within AI development teams, alongside pressure from AI managers to rapidly deliver highly anticipated outcomes. Furthermore, the integrity of AI developers is critical, as they possess the ability to hide biases from management and other relevant AI stakeholders. The testing phase preceding model deployment is also prone to bias because it seldom reflects the diverse social demographics that could be impacted.This research offers actionable recommendations across four core dimensions: governance, social, technical, and training and development procedures. Responsible organizations must implement deliberate measures to guarantee that their AI developers follow equitable protocols during AI creation; AI developers need to put ethical considerations first and evaluate the societal impacts of their models; collaborative partnerships ought to be formed among AI developers, AI stakeholders, and the communities likely to be affected by these models; and AI developers must emphasize transparency and explainability within their models while ensuring robust bias testing and remediation prior to deployment. Lastly, providing emotional intelligence training to AI developers is recommended to equip them for constructive dialogue with stakeholders outside the immediate development team.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 56

Interpreting Convolutional Neural Network Classifications of Raman Spectra: Insights from Fungal Spores and Carotenoid-Pigmented Microorganisms
A multitude of studies demonstrate that robust predictive models for identifying microorganisms can be successfully developed by combining Raman micro-spectroscopy with chemometric techniques, frequently yielding highly accurate outcomes. Progress in machine learning alongside increased software availability has simplified the process for researchers to construct predictive frameworks from intricate data arrays. Conversely, the underlying reasons for the high accuracy of these classifications remain under-investigated. This study analyzes Raman spectroscopic datasets from fungal spores and carotenoid-producing microorganisms to demonstrate that successful differentiation is frequently independent of peak positions or subtle variations in spectral band ratios caused by minor biochemical differences among the organisms. Instead, distinctive baseline variations in biochemically related microorganisms, which may be amplified by specific data preprocessing routines, play a critical role; furthermore, seemingly neutral spectral intervals can significantly influence a convolutional neural network. By applying a technique known as Gradient-weighted Class Activation Mapping, we investigate the internal mechanics of convolutional neural networks within microbiological domains to determine the precise Raman spectral bands driving successful classification.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 57

Comparing Intrinsic Interpretability and Post-Hoc Explanations: An Empirical Study across Multiple Datasets
In recent years, considerable attention has been directed toward clarifying complex, opaque systems, commonly referred to as black-box models, including deep neural networks and random forests. So-called model-agnostic techniques generally approximate the outputs of these non-transparent systems using an interpretable architecture, ignoring the internal mechanisms of the primary black-box system itself. Consequently, it remains an important inquiry whether the straightforward training of transparent, white-box architectures ought to be favored over the post-hoc approximation of these opaque models. This study presents the findings of an empirical investigation comparing post-hoc explanatory methods against inherently interpretable architectures across multiple datasets, focusing on both rule-based and feature-based frameworks. The empirical outcomes indicate that directly trained transparent models frequently approximate black-box models with an accuracy that matches or exceeds that of post-hoc surrogate models, despite lacking direct access to the underlying black-box structure.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 58

Transfer Learning with Simulated LiDAR Data for Robust Vehicle Detection in Parking Environments
Identifying vehicles within parking zones yields valuable insights into the spatial and temporal distribution of parking spot utilization. Because parking monitoring is conventionally conducted through manual observation, its temporal granularity remains restricted by substantial personnel expenditures. This study leverages simulated datasets alongside transfer learning to construct a dependable, real-world model tasked with detecting and categorizing vehicles via single-beam LiDAR within a roadside parking context. This work introduces a synthetically augmented transfer learning methodology designed for LiDAR-centric vehicle detection, alongside the deployment framework for synthetic LiDAR datasets. A synthetically augmented transfer learning technique was implemented to expand the constrained real-world dataset and facilitate the formulation of data management workflows. Furthermore, incorporating this synthetically augmented transfer learning methodology enhances both the resilience and the overall precision of the network. Empirical tests demonstrate that this approach enables the rapid deployment of models tailored for vehicle detection using a LiDAR sensor.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 59

Privacy-Aware Analysis of Encrypted Medical Image Sequences Using Convolutional Bi-LSTM Networks
The integration of deep learning (DL) algorithms offers significant potential to enhance healthcare applications. Specifically, DL has advanced medical imaging diagnostics, therapeutic interventions, and disease management. However, deploying deep learning algorithms on sensitive medical imagery introduces substantial challenges regarding data security and patient privacy. Optimizing medical imaging performance while simultaneously ensuring patient anonymity remains a complex task. Consequently, privacy-preserving techniques designed for deep learning model training and inference are increasingly gaining traction. Currently, these medical image sequences are interpreted using state-of-the-art computer-aided detection/diagnosis (CAD) methodologies. Yet, frameworks that require uploading medical photographs to external servers inherently create privacy vulnerabilities.To address this, this article introduces a convolutional Bi-LSTM network designed to evaluate fully homomorphic-encrypted (HE) time-series medical images. Within this framework, convolutional blocks extract selective spatial features from the secure image sequences, while analytical sequence layers based on Bi-LSTM encode the temporal data. To optimize efficiency and minimize misdiagnoses, a weighted unit and sequence voting layer incorporates spatial data using varying weights. The effectiveness of this framework is demonstrated using two rigorous public benchmarks: the CheXpert and BreaKHis datasets. The proposed technique surpasses several competing methods, achieving an accuracy exceeding 0.99 across both datasets. These findings indicate that the introduced methodology can successfully extract visual representations and sequential dynamics from encrypted medical image series, thereby safeguarding patient privacy while maintaining high performance in medical image analysis.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 60

Enhancing Transparency in Spatial PM₂.₅ Prediction over Singapore through Explainable Machine Learning
The climate system is heavily influenced by aerosols through direct and indirect pathways, notably the absorption and scattering of radiant energy. Beyond climate impacts, these particles compromise atmospheric visibility and jeopardize public health. Specifically, human populations face exposure to fine particulate matter PM2.5, a major driver of respiratory and cardiovascular pathologies. Long-term modifications in PM levels are generally governed by emission shifts and meteorological patterns, whereas short-term fluctuations are predominantly dictated by immediate weather dynamics. Surface variations in PM2.5 levels are further driven by environmental variables such as temperature, wind velocity, relative humidity, and vegetative coverage.Air quality forecasting has advanced significantly through the application of machine learning. This research explores PM2.5 estimation by utilizing these environmental and meteorological metrics as spatiotemporal inputs. This study evaluates seven years (2014–2021) of in situ PM2.5 measurements gathered from five distinct monitoring stations across Singapore, establishing a baseline dataset for spatial modeling.The primary contribution of this paper is a novel predictive architecture tailored for temporal analysis, utilizing Random Forest (RF), Gradient Boosting (GB) regression, and the Tree-based Pipeline Optimization Tool (TP) Auto ML framework, which optimizes via a meta-heuristic genetic algorithm. The TP model demonstrated strong predictive capabilities, achieving a peak Global Performance Index (GPI) of 7.4 in August 2016 and a minimum value of −0.6 in June 2019. These metrics highlight the robust nature of the TP framework; its negative values remained superior to competing models, indicating less pessimistic errors. Finally, the underlying logic of these machine learning systems is clarified using eXplainable Artificial Intelligence (XAI) methods, which validate the integrity of feature importance and offer deeper insights into the cyclical variations of PM2.5 concentrations.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 61

Understanding Non-Transparency in Digital Platforms: Ethical, Governance, and Accountability Perspectives
The integration of digital architectures powered by artificial intelligence (AI) into daily routines is expanding at an unprecedented rate. Within this paradigm, the twin dilemmas of accountability and openness endure as critical issues that have yet to be satisfactorily resolved. This study intends to delineate the specific domains of opacity within digital ecosystems and formulate actionable frameworks for mitigating these visibility challenges. Initially, by synthesizing existing scholarship, interrogating the dynamics of platformization, establishing a cross-disciplinary conceptualization of AI, and applying ethical AI frameworks, we map the dimensions of opacity inherent in digital environments. Subsequently, having mapped these obscure domains, we move beyond a standard literature synthesis to offer an independent perspective on mitigating these systemic vulnerabilities. Our evaluation of the literature indicates the existence of three primary spheres of opacity within digital networks: an inadequate understanding of platform content originators; a deficiency of visibility regarding the background labor force, their explicit operational outputs, and their occupational environments; and a lack of openness concerning the engineering and oversight of algorithmic models. Given the abundance of abstract, high-level ethical tenets in current scholarship that resist pragmatic implementation, this investigation represents an effort to reconcile theoretical concepts with operational execution.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 62

Interpretable Classification with Expressive Boolean Logic: A Stochastic and Quantum-Ready Optimization Framework
We introduce and execute a transparent machine learning classification model designed for Explainable AI (XAI) that leverages expressive Boolean formulas. Potential domains of use include credit risk evaluation and medical diagnostics. The system utilizes a Boolean formula to establish a rule with adjustable complexity (or interpretability) for categorizing input data. This formula accommodates any operator applicable to single or multiple Boolean variables, offering superior expressivity relative to more rigid rule- and tree-based methodologies. Native local optimization methods are used to train the classifier, allowing for an efficient search of the feasible formula space. Fast Integer Linear Programming (ILP) or Quadratic Unconstrained Binary Optimization (QUBO) solvers—potentially enhanced by custom hardware or quantum systems—can identify shallow rules. By executing non-local moves that optimize across subtrees of the complete Boolean formula, we merge the expressivity and efficiency of the native local optimizer with the rapid processing capabilities of these devices. We present comprehensive numerical benchmarking data against multiple baseline models using recognized public datasets. The findings indicate that the native local rule classifier performs competitively with alternative models. Incorporating non-local moves yields comparable outcomes with a reduced number of iterations. Consequently, deploying specialized or quantum hardware could provide substantial acceleration by rapidly generating non-local moves.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 63

Interpretable Deep Learning for Verbal Autopsy Analysis: A Stacked Ensemble Approach to Cause-of-Death Determination
Verbal autopsies (VA) serve as a frequent mechanism within Low- and Medium-Income Countries (LMIC) to ascertain the cause of death (CoD) for individuals who pass away outside of clinical environments, relative to the established international gold standard of physician medical certification. Field interviewers gather detailed testimonies from the families of the deceased concerning the events and circumstances preceding the fatality. This gathered information is documented in text format, referred to as VA narratives. These textual records contain granular details suitable for identifying the CoD. Nonetheless, this methodology continues to be a manual operation characterized by high costs, inconsistency, prolonged processing times, and subjectivity (making it susceptible to errors), among other limitations. Consequently, these disadvantages undermine the VA reporting infrastructure, which is otherwise crucial for defining public health agendas and populating civil registration frameworks.To address this operational gap, the present study introduces interpretable deep learning (DL) methodologies designed to analyze VA narratives and deliver CoD predictions in an efficient, transparent, economical, and precise manner. The verification of our DL models is conducted via optimization and performance accuracy machine learning (ML) curves mapped against the volume of training samples. Our models achieved training set accuracies of (LSTM = 76.11%, CNN = 76.35%, and SEDL = 82.1%), validation accuracies of (LSTM = 67.05%, CNN = 66.16%, and SEDL = 82%), and test set accuracies of (LSTM = 67%, CNN = 66.2%, and SEDL = 82%). Additionally, Local Interpretable Model-agnostic Explanations (LIME) are utilized to enhance the interpretability of the outcomes, thereby fostering confidence in automated healthcare solutions.This work offers robust deep learning strategies for CoD determination via VAs, where the stacked ensemble deep learning (SEDL) model achieved optimal performance, outperforming both Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN). Our empirical evidence suggests that ensemble DL paradigms can be successfully integrated into the CoD classification workflow to assist medical experts in reaching a diagnosis. Ultimately, this integration aims to compress the diagnostic turnaround time for physicians reviewing the narratives, while simultaneously reducing operational expenses and mitigating human error. The scope of this study was constrained by the volume of training data required for model optimization and the substantial lexical diversity present within the unstructured text.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 64

Towards Explainable Deep Reinforcement Learning: Interpreting Experience Replay through SHAP
Although Reinforcement Learning (RL) exhibits significant potential for refining intricate control and decision-making systems, Deep Reinforcement Learning (DRL) suffers from a distinct lack of explainability. This limitation restricts its deployment within highly regulated domains, including manufacturing, finance, and healthcare. The opacity of DRL decision-making introduces structural challenges that impede operational efficiency and resource management—an issue that intensifies alongside ongoing algorithmic advancements. While current trends favor transitioning from traditional Experience Replay configurations to A3C architectures, the latter imposes substantially greater resource demands. Furthermore, despite continuous efforts to refine selection mechanisms within Experience Replay, repository capacities generally remain elevated. In this study, we evaluate a Deep Convolutional Q-learning agent across 20 Atari environments while deliberately downsizing the Experience Replay capacity from 1×106 to 5×102. Our findings indicate that lowering the capacity from 1×104 to 5×103 does not meaningfully degrade performance rewards, thereby establishing a viable strategy for resource-efficient DRL frameworks. To clarify agent choices and correlate them with inherent game mechanics, we introduce a novel methodology: visualizing the Experience Replay mechanism utilizing the Deep SHAP Explainer. This framework enhances domain comprehension and delivers transparent, interpretable accounts, though capacity constraints must be managed carefully to mitigate overfitting. Ultimately, this research validates the practicality of minimizing Experience Replay storage and supports the integration of transparent explanations via the Deep SHAP Explainer to advance resource conservation.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 65

Explainable Machine Learning for COVID-19 Prognosis: Assessing Variant Influence and Clinical Risk Factors
Machine learning (ML) techniques have been applied in diverse manners to combat the COVID-19 pandemic. Predictive ML models have been constructed, for example, for both diagnostic and prognostic objectives, utilizing varying data modalities such as textual, visual, and structured formats. Because of the distinct characteristics of this disease and its temporal transformation, a comprehensive understanding of the central elements driving the clinical trajectory of COVID-19 in individual patients remains incomplete. Throughout all phases of this study, a medical specialist was actively engaged via a human-in-the-loop framework. This collaborative element represents a vital yet frequently overlooked component of the ML and knowledge extraction (KE) workflow. Our findings indicate that explainable artificial intelligence (XAI) can offer substantial reinforcement to this dimension of ML and KE. This investigation centered on employing ML for knowledge extraction across two distinct operational scenarios. In the initial scenario, we sought to determine if integrating data regarding the dominant COVID-19 variant affects the predictive performance of the ML models. In the subsequent scenario, we concentrated on prognostic classification models designed to predict a patient's requirement for intensive care unit admission, integrating various XAI methodologies. We implemented nine distinct ML techniques, specifically XGBoost, CatBoost, LightGBM, logistic regression, Naive Bayes, random forest, SGD, SVM-linear, and SVM-RBF. The predictive quality of these models was evaluated using precision, accuracy, and AUC metrics. Following this evaluation, knowledge extraction was performed on the highest-performing models through two discrete pathways: (a) automated feature extraction using forward stepwise selection (FSS); and (b) the identification of attributes and their interactions using model explainability tools. Both sets of features were evaluated against a baseline configuration of attributes selected a priori by medical professionals using domain expertise. Our empirical tests demonstrated that incorporating data on the prevailing COVID-19 variant did not alter the performance of the generated ML models. Furthermore, the results indicated that human medical experts displayed significantly higher precision in identifying meaningful attributes compared to the automated FSS approach. Explainability tools identified nearly identical sets of attributes as the medical expert, while additionally uncovering noteworthy attribute interactions that the specialist subsequently evaluated from a clinical perspective. The outcomes of this investigation and their broader implications are examined.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2025 | Article: 66

Machine Learning in Defense and Military Applications: Opportunities, Challenges, and Ethical Considerations
Machine learning (ML) has evolved into an indispensable technology within the defense sector, driving the creation of sophisticated mechanisms for identifying threats, supporting decision-making processes, and managing autonomous operations. Nevertheless, the expanding integration of ML into defense frameworks has generated significant ethical anxieties concerning accountability, transparency, and inherent bias. This paper offers an extensive evaluation of ML’s influence on the defense industry, detailing both the advantages and risks of implementing these methodologies across diverse domains, including surveillance operations, target pinpointing, and autonomous weapon platforms. Furthermore, we examine the ethical repercussions of deploying ML in military contexts, emphasizing problems tied to privacy preservation, responsibility attribution, and algorithmic bias. Ultimately, we outline strategic recommendations to alleviate these ethical complications, advocating for heightened operational transparency, clearer accountability structures, and the active engagement of diverse stakeholders in the architecture and deployment of ML systems within defense frameworks.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2025 | Article: 67

Interactive Explainable Fair Machine Learning: A Human-in-the-Loop Framework for Bias Detection and Mitigation
The expansion of machine-learning frameworks into sectors with significant end-user consequences has heightened concerns regarding the equity of trained models, particularly the necessity of preventing biases that disadvantage distinct demographic sub-groups. The majority of current bias-mitigation techniques adjust the relative weight of data observations during the pre-processing phase. Because fairness depends heavily on context, we support an interactive machine-learning strategy that empowers human supervisors to offer iterative guidance for model refinement. To this end, we adapt the explanatory interactive machine-learning framework Caipi for equitable machine learning. This modified approach, FairCaipi, integrates human-in-the-loop interventions regarding both model predictions and underlying explanations to enhance model equity. Empirical outcomes indicate that FairCaipi outpaces a state-of-the-art pre-processing bias mitigation strategy in both the equity metrics and the predictive accuracy of the resulting machine-learning framework. We show that FairCaipi serves to expose and minimize algorithmic bias in machine-learning architectures while simultaneously identifying underlying human bias.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2025 | Article: 68

Neural Logic Networks for Interpretable Decision Rule Learning: Alternative Formulations and Architectures
This study develops an expanded framework for extracting classification rules from neural network architectures applied to tabular data. We introduce alternative designs for learning Boolean logic operators by configuring neurons with continuous weights, including the integration of trainable NAND gates. This unified formulation ensures that distinct logical operations are trained under a singular mathematical approach, allowing developers to directly deploy established sparsity-inducing optimization routines—such as reweighted L1 regularization—to simplify the resulting network into concise rule sets. Furthermore, by utilizing De Morgan’s law, we construct an alternative network topology based on these trainable NAND components to create a NAND-NAND structure rather than the traditional AND-OR setup, both of which easily convert into discrete decision rules. Empirical evaluations show that these revised methodologies produce highly competitive decision rule models that match state-of-the-art predictive accuracy on tabular datasets.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2025 | Article: 69

Generating Feasible Minimally Contrastive Counterfactual Explanations for Real-Time Decision Support
Machine learning-driven decision support tools ought to assist users in detecting prospective advantages and risks. Widespread model-independent explanation frameworks can pinpoint elements underpinning specific forecasts—addressing queries like "What factors affect sales?" or "Why did sales decline?"—yet fail to indicate the actionable steps an individual can or should take to achieve a more favorable result. Although counterfactual explanation methods target actionable interventions and occasionally account for feasibility, they generally overlook viability for real-time systems like interactive question-answering platforms. In this study, we bridge this divide by proposing a unique model-agnostic technique that details concrete, viable adjustments capable of altering the predictions of a sophisticated Black Box AI framework for a specific instance; furthermore, we evaluate its practical utility by evaluating its computational speed in real time alongside its capacity to discover executable modifications. The developed approach leverages the target instance to produce high-precision explanations, subsequently employing a secondary mechanism to identify achievable minimally-contrastive counterfactual explanations (AMCC) by restricting the exploration space exclusively to modifications that adhere to domain-specific restrictions. Utilizing a well-established dataset, we assessed the classification procedure to determine both the success rate and the computational duration necessary to pinpoint valid counterfactual alternatives. Within the context of a classifier maintaining 90% accuracy, our proposed algorithm successfully extracted AMCC explanations in 47% of instances (38 of 81), requiring an average generation time of 80 ms. These outcomes confirm the efficiency of the algorithm in rapidly generating AMCC explanations, establishing its appropriateness for real-time deployments. The AMCC approach improves the interpretability of obscure Black Box AI models, thereby assisting stakeholders in analyzing corrective techniques or evaluating prospective consequences.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2025 | Article: 70