Transparent Intelligence Publishing Transparent Intelligence Publishing

Search

Search results:
An Explainable EfficientNet-ResNet Framework for Automated Breast Ultrasound Lesion Classification
Breast cancer represents a leading cause of mortality among women. Consequently, early detection is crucial for effective screening and reducing the overall death rate. Therefore, the development of computer-aided diagnosis systems is essential for identifying breast cancer in its initial stages. Because of its affordability, ultrasound serves as a vital examination tool for breast cancer diagnosis. Although numerous learning-based methods have recently been introduced to categorize breast cancer using the breast ultrasound imaging (BUSI) dataset, manual processing remains a difficult and highly time-consuming task. This study presents an XAI-based framework utilizing an EfficientNet-integrated ResNet deep network to precisely categorize breast cancer as either benign or malignant. Initially, data augmentation is applied to expand the training sample size. To achieve this, three-pixel flip mathematical equations are implemented: horizontal, vertical, and 90°. Subsequently, two pre-trained deep learning models are utilized, selectively skipping specific layers before fine-tuning. These adjusted models are then trained via a deep transfer learning process to extract features from their deeper layers. The performance of these trained models is evaluated utilizing explainable artificial intelligence (XAI). Following this, a novel feature selection approach known as cuckoo search controlled standard error mean is introduced, which is grounded in the cuckoo search algorithm. This method isolates the optimal features, which are then combined using a novel parallel zero-padding maximum correlated coefficient fusion technique. Finally, the selection algorithm is re-applied to the integrated feature vector prior to classification via machine learning algorithms. Experimental evaluations of the proposed framework, performed on the publicly accessible BUSI dataset, achieved accuracies of 98.4% and 98% across two distinct experiments. Comparisons with contemporary techniques demonstrate that the proposed framework offers enhanced accuracy. Furthermore, it requires less execution time compared to the standalone original deep learning models.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2022 | Article: 5

Machine Learning-Driven Obesity Classification and Tailored Treatment Recommendations
As a leading risk factor for chronic illnesses, obesity highlights the critical need for prompt diagnosis and active management. This research introduces an innovative expert system developed to precisely categorise obesity severity and offer individualised treatment advice. Utilising the SEMMA data mining framework alongside the tidymodels ecosystem, the study assessed five machine learning algorithms: decision tree, random forest, multinomial logistic regression (MLR), Naive Bayes, and support vector machine (SVM). Achieving the highest predictive accuracy (97.48%), MLR was chosen as the definitive model. The platform incorporates an intuitive R Shiny interface, promoting seamless real-time user engagement. Treatment protocols are formulated via if-then rule-based logic to guarantee customised advice for every obesity classification. A comparative evaluation emphasises the system's exceptional diagnostic precision and its functional utility in recommending treatments. By improving accessibility—especially for marginalised rural communities—this system bolsters public health by facilitating preliminary diagnosis, specific interventions, and forward-looking obesity care.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2022 | Article: 7

Explainable AI for Automatic Detection of Equatorial Plasma Bubbles Using 2DPCA and Recursive Feature Elimination
Equatorial plasma bubbles (EPBs) represent areas of reduced electron density that develop within Earth’s ionosphere as a result of Rayleigh–Taylor instability. These formations can trigger signal scintillation, which induces signal degradation and navigation positioning inaccuracies. EPBs can be identified via imagery gathered by All-Sky Imager (ASI) networks. The present research introduces an inexpensive, automated technique for identifying EPBs within ASI datasets, suitable for real-time monitoring as well as categorization tasks. This approach couples Two-Dimensional Principal Component Analysis (2DPCA) and Recursive Feature Elimination (RFE) alongside a Random Forest machine learning classifier to construct an Explainable Artificial Intelligence (XAI) framework. This framework extracts image characteristics to automate EPB detection using minimal dimensionality, yielding a compact and exceptionally rapid-to-train model designed to discern EPBs in recorded ASI frames. A collection of 2458 images, categorized into "Event" and "Empty" classes, was assembled to construct the database. This image repository was partitioned randomly into two distinct portions: an 80% training dataset and a 20% testing dataset. The developed XAI framework achieved a marginally better detection accuracy relative to the conventional 2DPCA approach while maintaining a notably smaller structural footprint. Additionally, the execution of the proposed framework was assessed and contrasted against alternative deep learning baselines (ResNet18, Inception-V3, VGG16, and VGG19) under identical environmental conditions.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2022 | Article: 10

Bridging Hand-Crafted Features and Deep Learning Through an Explainable Shallow CNN for Medical Image Segmentation
In clinical environments, the transparency of artificial intelligence models is paramount, as understanding the rationale behind automated choices is vital. Although black-box AI frameworks exhibit remarkable efficacy, their deployment in healthcare remains impractical unless their underlying mechanisms can be clarified for medical professionals. Deep neural networks (DNNs) achieve state-of-the-art results, yet the justifications they provide are frequently incomprehensible to human observers. Conversely, traditional machine learning approaches and manually designed features extracted to capture distinct data characteristics offer greater transparency, though human constraints in feature engineering often limit their overall effectiveness. To bridge this gap, we present ExShall-CNN, a novel and transparent shallow convolutional neural network tailored for medical image analytics. This framework builds upon hand-crafted feature methodologies to preserve human interpretability, guaranteeing clear and verifiable decision-making. We introduce the explainable shallow convolutional neural network (ExShall-CNN), which merges the high performance of sophisticated deep architectures like U-Net with the inherent clarity of hand-crafted features for segmenting medical images. Grounded in contemporary machine learning progress, ExShall-CNN integrates widely utilized kernels while maintaining structural openness, allowing medical practitioners to visually inspect and comprehend its outputs. This balanced strategy delivers the precision expected of deep learning paradigms alongside the explainability required for clinical deployment.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2022 | Article: 12

Brain-Wide Mental State Analysis Using Explainable Shallow Neural Networks on HCP fMRI Data
Artificial neural networks (ANNs) have frequently been deployed for classification objectives within functional magnetic resonance imaging (fMRI) research, though such analyses traditionally isolate only specific regional sub-volumes of the brain. Recent methodology has integrated shallow neural networks (SNNs) with explainable artificial intelligence (xAI) frameworks to uncover underlying mechanisms of neural processing. Although prior research validated this paradigm utilizing fMRI data from motor execution tasks, the current investigation implements this framework on Theory of Mind (ToM) cognitive tasks, leveraging data extracted from the Human Connectome Project (HCP) Young Adult repository. Cognitive paradigms introduce greater computational complexity due to the fundamentally non-linear dynamics of neural functions. The HCP multimodal parcellation brain atlas is utilized to partition the cortex, thereby guiding the systematic training, pruning, and subsequent retraining of an SNN. Shapley values are subsequently computed to decode the retrained architecture, with the resulting outputs benchmarked against a General Linear Model (GLM) analysis for empirical validation. The baseline network demonstrated a classification accuracy of 88.2%, which decreased to 80.0% following network pruning, and subsequently recovered to 84.7% after the retraining phase. The SHAP-derived explanations demonstrated strong convergence with both GLM observations and established neuroanatomical regions associated with ToM. This fMRI investigation effectively navigated a highly complex cognitive paradigm, illustrating the efficacy of explainability techniques in deciphering non-linear neurological dynamics. These outcomes indicate that xAI platforms, particularly knowledge extraction frameworks, offer substantial utility for progressing mental health research and neural state decoding applications.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2022 | Article: 13

Enhanced MINDWALC Node Classification for Explainable Knowledge Graph Analytics
This research investigates and refines the MINDWALC node classification algorithm, specifically targeting its capacity to extract human-interpretable decision trees from knowledge graph (KG) architectures. We propose techniques to tailor MINDWALC for environments where the knowledge graph is bifurcated into internal background knowledge (domain-specific data) and instance knowledge (specific case data). While the core logic of MINDWALC—utilizing discriminative graph walks as features—is preserved, we introduce two primary enhancements. First, we implement relation-tail merging to empower the algorithm to process relation-modified nodes. Second, we propose flexible walking depths that, when combined with the original MINDWALC strategy, improve the detection of commonalities between node instances. Our findings suggest that these flexible walks can markedly increase classification accuracy in hierarchical or incomplete tree-like graphs, though performance varies on more heterogeneous KG structures. Ultimately, this work demonstrates that our refinements significantly boost MINDWALC’s utility in tree-structured environments, allowing the system to use background knowledge to bridge gaps in instance data in a transparent way. Initial evaluations on medical toy datasets suggest these optimizations could refine clinical decision-making, particularly in sectors requiring explainable artificial intelligence.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 14

Pixel-Level Interpretability for Explainable Medical Imaging: A Comparative Evaluation against Grad-CAM
To overcome critical constraints regarding diagnostic accuracy and model transparency in medical imaging, this paper presents a novel framework termed the Pixel-Level Interpretability (PLI) model. The investigation primarily aims to benchmark PLI against Gradient-Weighted Class Activation Mapping (Grad-CAM) to attain superior localization correctness and highly detailed explainability. Operating on a VGG19 convolutional neural network infrastructure, the methodology incorporates three open-access COVID-19 chest X-ray datasets containing over 1000 categorized images; these underwent augmentation, normalization, and resizing steps to achieve generalizability and algorithmic stability. The experimental evaluation focused on vital performance metrics: structural similarity (SSIM), mean squared error (MSE), interpretability, computational speed, and diagnostic precision. The findings indicate that PLI drastically surpasses Grad-CAM across every evaluated metric. Specifically, PLI generated fine-grained, pixel-granular heatmaps that yielded reduced MSE, superior SSIM metrics, and expedited inference rates, demonstrating an capability to reveal minute diagnostic elements without sacrificing processing speed. Conversely, the explanations produced via Grad-CAM regularly lack the detailed resolution necessary for dependable clinical integration. By incorporating fuzzy logic to refine both numeric and visual explanations, PLI delivers transparent data that corresponds with clinical logic, thereby allowing healthcare professionals to execute decisions with greater certainty. Ultimately, this research validates PLI as a dependable framework for resolving deficiencies in AI explainability and clinical adoption. By simultaneously managing the dual demands of precision and transparency, PLI advances medical AI deployment and establishes a precedent for implementation within other high-stakes environments.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 15

Advancing Trustworthy AI in Indian Healthcare: Ethical, Legal, and Implementation Challenges
The medical industry in India has undergone substantial evolution driven by progress in technology and infrastructure. Yet, prominent hurdles remain when addressing critical problems like inadequate medical infrastructure for the nation's massive population, restricted accessibility, a deficit of qualified practitioners, and high-quality services. Artificial intelligence (AI)-based interventions offer the possibility to ease the burden on the Indian medical network; nevertheless, incorporating reliable AI into this domain is still obstructed by moral and legal barriers. The objective of this research is to critically analyze the contemporary state of AI system engineering in Indian medical care and its alignment with the moral and legal dimensions of AI, while also pinpointing the obstacles and prospects for implementing reliable AI within the Indian medical domain. This investigation assessed 15 papers chosen from an initial pool of 1136 papers collected across two digital databases, PubMed and Google Scholar, alongside project sites. This inquiry utilizes the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR). The findings indicate that current publications primarily deployed traditional machine learning (ML) models and artificial neural networks (ANNs) for diverse operations, including drug discovery, disease monitoring networks, early sickness identification and diagnostic precision, and medical resource administration in India. This analysis uncovers a deficiency in the integration of reliable AI within Indian medical care and various obstructions linked to it. It examines pathways for engineering reliable AI within Indian medical environments, prioritizing patient protection, information confidentiality, and adherence to moral and statutory criteria.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 16

Mapping Understanding through Biometric Signals: AI Approaches to Cognitive and Affective Phases
The connection between the cognitive and emotional components of comprehension has remained largely unexamined because of a shortage of dependable techniques for assessing feelings and emotions during educational activities. Concentrating on five distinct stages of comprehension—nascent understanding, misunderstanding, confusion, emergent understanding, and deep understanding—this investigation presents an AI-based method to assess subjective comprehension by tracking the physiological activity visible in facial expressions. To examine these stages, 103 individuals worked on 15 riddles in a remote setting while their facial expressions were video recorded. Action units (AUs) for each stage occurrence were quantified utilizing AFFDEX software. Recurring AU configurations linked to each stage were subsequently pinpointed by implementing six supervised machine learning algorithms. Unique AU configurations were identified for all five stages, with gradient boosting machine and random forest models delivering the greatest predictive precision. These outcomes indicate that physiological activity can be utilized to dependably assess comprehension. Furthermore, they propose an innovative methodology for assessing and encouraging comprehension in instructional environments, as well as designing adaptive educational technologies and tailored learning interventions. Future research should investigate how the physiological indicators of comprehension stages simultaneously mirror and impact their related cognitive workflows, alongside the generalizability of these outcomes across varied demographics and educational environments (A suite of AI tools was employed in the development of this paper: (1) ChatGPT4o (for writing clarity and reference checking), (2) Grammarly (for grammar and editorial corrections), and (3) ResearchRabbit (reference management)).
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 17

A Hybrid Fuzzy Multi-Criteria Decision-Making Framework for Evaluating AI-Driven Mental Health Interventions
Artificial intelligence (AI) applications in mental health possess immense potential to revolutionize clinical results, yet determining the most optimal strategies remains a persistent difficulty. This research fills this gap by assessing and establishing priorities among various AI-based mental health options. The evaluation is grounded in essential benchmarks: implementation feasibility, economic efficiency, scalability, adherence to ethical standards, user satisfaction, and overall clinical efficacy. A fuzzy multi-criteria decision-making (MCDM) framework, integrating fuzzy TOPSIS and fuzzy ARAS, was utilized to determine the ranking of the alternatives. A hybrid combination of these two approaches was subsequently implemented to reconcile variations in the results, as each methodology highlights distinct evaluative dimensions. Fuzzy TOPSIS, which privileges proximity to the ideal target, positioned the personalization of care (A5) as the primary alternative, yielding a closeness coefficient of 0.50, with user engagement (A2) following at 0.45. Similarly, fuzzy ARAS, which measures aggregate performance, ranked A5 as the leading option, demonstrating a total performance score of Si = 0.90 and a utility degree of Qi = 0.92. The integration of both frameworks offered a well-rounded analysis, wherein A5 maintained its premier ranking because of its superior performance in user satisfaction and clinical outcomes. These findings highlight the critical role that personalization and active engagement play in refining AI-enabled mental health platforms. This indicates that customized, patient-centric strategies are vital for maximizing therapeutic efficacy and sustaining user commitment.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 18

Evaluating Large Language Models for Symptom-Based Medical Diagnosis: A Comparative Performance Analysis
This research undertook an assessment of Large Language Models (LLMs) within the domain of clinical diagnostics, specifically investigating their capacity to interpret symptom-centered inputs and generate precise clinical assessments. The investigation focused on evaluating several prominent architectures—including GPT-4, GPT-4o, Gemini, o1 Preview, and GPT-3.5—to gauge their proficiency in identifying clinical conditions solely from presented symptomatology. To guarantee clinical validity and appropriateness, symptom profiles were compiled from verified medical literature. Each architecture underwent testing within standardized environments to quantify diagnostic accuracy, precision, recall, and underlying logical processing. Specialized clinical scenarios were structured to examine performance boundaries across both routine and critical diagnostic conditions. The findings indicated that GPT-4 delivered the highest overall diagnostic accuracy, displaying a strong synergy with established medical logic. Conversely, Gemini demonstrated superior performance in high-stakes environments where exact decision-making is vital. Both GPT-4o and o1 Preview presented equitable metrics, maintaining a stable balance between precision and recall while managing real-time diagnostic scenarios. While possessing less advanced capabilities, GPT-3.5 remained functional for standard diagnostic duties. Ultimately, this investigation outlines the current capabilities and constraints of utilizing LLMs for clinical determination. Even though architectures like GPT-4 and Gemini show significant capability, critical barriers regarding data confidentiality, ethical protocols, and training bias correction must still be overcome. These insights offer strategic directions for the ethical incorporation of LLMs into diagnostic frameworks to optimize patient care outcomes.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 19

A SHAP-Guided Bi-GRU Framework for Explainable Short-Term Load Forecasting in Smart Energy Systems
This research introduces an innovative prediction framework utilizing a parallel multi-input Bi-GRU structure coupled with a sliding window-based Multi-Input Multi-Output (MIMO) forecasting approach. The primary novelty centers on feature-specific Bi-GRU encoding, which independently processes historical demand, meteorological variables, and calendar indicators prior to data fusion, thereby empowering the network to more effectively capture temporal correlations across disparate data streams. In contrast to preceding models, our methodology distinctively merges a structured feature-specific Bi-GRU encoding mechanism with interpretability via SHAP analysis, providing both superior predictive fidelity and transparent algorithmic decision-making suitable for practical deployment within electrical grids. The introduced Bi-GRU MIMO architecture operates as the foundational model. To rigorously evaluate its performance, an attention-enhanced LSTM model is incorporated solely as a baseline for comparative analysis. Empirical validation using actual data from Singapore’s National Electricity Market System (NEMS) demonstrates that the Bi-GRU MIMO framework enhances forecasting accuracy by 1.64% and decreases MAE by 2.19% relative to traditional models, underscoring its capability to improve decision-making in smart grids.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 20

AI in Recruitment: How Trust and Procedural Justice Influence Organizational Attraction and Intent to Apply
The escalating value placed on organizational human capital underscores the necessity for firms to optimize their systems for sourcing, evaluating, acquiring, onboarding, deploying, and retaining premium talent. Although embedding artificial intelligence (AI) within recruitment infrastructure promises unmatched workflow acceleration, operational scalability, minimized human subjectivity, and enhanced candidate sourcing, its deployment introduces distinct friction points that can degrade candidate attitudes and behavioral trajectories. Grounded in signaling and organizational justice frameworks, the deficit of interpersonal relational dynamics paired with the opaque nature of automated screening algorithms can prompt prospective candidates to question the integrity and structural fairness of the employment screening process. These institutional challenges risk modifying not only how job seekers deduce corporate principles, but also whether they choose to abandon the submission process altogether. Consequently, this investigation explores the comparative impact of AI-mediated versus human-mediated selection pathways on applicant trust, perceived procedural equity, corporate appeal, and subsequent intent to apply. Furthermore, we map the precise pathways through which trust and structural fairness mediate the link between the initial screening modality and early-stage candidate attraction and submission intentions. Data from an experimental student questionnaire indicate that job candidates view automated hiring protocols as significantly less credible, less equitable, and less appealing. Similarly, respondents exhibited depressed motivation to pursue open roles within enterprises leveraging automated recruitment architectures. Cultivating higher levels of institutional trust during the selection pathway and designing touchpoints that allow candidates to authentically showcase their professional capabilities can constructively reshape applicant attitudes and behavioral outcomes. These insights challenge the reductive assumption that operational velocity inherently yields strategic talent management success, emphasizing instead the vital role of interpersonal markers in securing top-tier talent within an automated recruitment environment.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2022 | Article: 21

Structured Assessment of AI-Assisted Systematic Reviews: Workflow and Rubric Insights
The application of Artificial Intelligence (AI) tools to assist with systematic literature reviews (SLRs) is expanding rapidly, yet researchers still lack structured frameworks for assessing and incorporating these tools into their research practices. This paper introduces and implements a rubric-based framework designed to evaluate AI toolchains across essential SLR stages, such as literature searching, screening, and data synthesis. A mixed-methods study was executed with 63 postgraduate student researchers, who represent the early-stage investigators that serve as primary adopters of these digital workflows. The results demonstrate that workflows structured around the rubric enhanced the participants' perceived utility of AI utilities, provided organized criteria for comparative analysis, and promoted greater transparency during tool selection. Although formal indicators of inter-rater reliability (such as Cohen’s or Fleiss’ Kappa) were not calculated due to the fact that participants did not evaluate identical benchmark sets concurrently, the consistency of the rubric was qualitatively validated through pilot evaluations, iterative updates, and structured focus group discussions. These efforts confirm interpretative alignment among users while recognizing that future validation via independent assessors is required. This research contributes a validated assessment rubric, an operational example of an AI-augmented SLR workflow, and a discussion regarding the consequences for research training, study reproducibility, and cross-disciplinary translation.
Journal of Trustworthy and Explainable Intelligent Systems
Review | Open access | 15 September 2022 | Article: 22

Digital Transformation of Ideological Education: AI-Driven Role Reconstruction and Competency Growth
The rise of the digital era presents mounting difficulties for ideological and political instruction, as the swift dissemination of internet-based public sentiment and the expanding reach of social media frequently outpace conventional pedagogical approaches. Present methodologies suffer from poor precision in detecting risks and lack explicit metrics for assessing the fundamental competencies of instructors. To surmount these challenges, this paper presents a dual-core system framework. Chiefly, the Multi-modal Decision Support System (MADF-SE) merges classroom behavioral identification, semantic evaluation, and blockchain-enabled traceability to enhance both the precision and punctuality of hazard detection. Secondary to this, the Dynamic Capability Evolution Model (DCEM) quantitatively monitors and upgrades the cognitive, practical, and inventive skills of teachers over time. Furthermore, a hierarchical federated learning architecture is deployed to strike a balance between individual privacy safeguarding and inter-collegiate cooperation. Empirical testing and case analyses across more than 30 higher education institutions indicate that the introduced framework cuts the public opinion management timeframe by 50%, elevates instructor engagement in strategic choices from 20% to 60%, and boosts their collective core competencies by greater than 25% on an annual basis. These outcomes highlight a viable and scalable methodology for driving the digital evolution of ideological and political schooling.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2023 | Article: 23

Advancing AI-Augmented Business Process Management through Knowledge-Driven Explainability and Agile Knowledge Sharing
Merging Artificial Intelligence (AI) with Business Process Management Systems (BPMSs) has given rise to AI-Augmented Business Process Management Systems (ABPMSs). These modern setups deliver flexible adaptability, instantaneous process refinement, and superior knowledge management workflows. Yet, notable hurdles persist, especially regarding contextual clarity, user buy-in, and the incorporation of behavioral dynamics. This research introduces a novel conceptual model that unifies the Socialization, Externalization, Combination, and Internalization (SECI) knowledge paradigm with Agile workflows (specifically Scrum) and advanced AI capabilities like explainable AI (XAI), process mining, and Robotic Process Automation (RPA). This setup streamlines the formalization, validation, and dissemination of insights through structured, intuitive software and collaborative frameworks, particularly Communities of Practice (CoPs). The system highlights context-sensitive transparency, step-by-step deployment, and iterative refinement to foster productive human–AI partnerships. It delivers both conceptual and practical blueprints for linking AI strengths with corporate knowledge assets. The introduced model eases the upgrade path from legacy BPMSs to advanced ABPMSs by utilizing systematic practices and technologies. This strategy accelerates knowledge distribution and operational refinement, mapped out via exhaustive BPMN 2.0 diagrams.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2023 | Article: 27

Offline Reinforcement Learning with Implicit Policy Constraints and Energy-Based Modeling
Offline reinforcement learning (RL) focuses on training policies exclusively from passively gathered datasets, establishing it as a data-driven decision-making framework. A major obstacle in offline RL is the phenomenon of distribution shift, which leads the algorithm to encounter out-of-distribution (OOD) samples. While constraining the divergence between the target policy and the behavior policy can alleviate distribution shift, this strategy can unduly restrict the target policy and degrade algorithmic performance, because it fails to explicitly differentiate between in-distribution and OOD samples. Furthermore, accurately learning and representing a multi-modal behavior policy is highly challenging when datasets are compiled from multiple distinct behavior policies. To resolve these limitations, the authors tackle the distribution shift problem through implicit policy constraints utilizing energy-based models (EBMs), bypassing the explicit modeling of the behavior policy. EBMs offer a powerful framework for characterizing complex multi-modal distributions while effectively distinguishing between in-distribution and OOD samples. Experimental outcomes demonstrate that this approach substantially surpasses explicit policy constraint methods and other baseline techniques. Furthermore, the trained energy model serves as an effective indicator of OOD encounters, providing alerts for potential failures.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2023 | Article: 34

Toward Robust Offline Reinforcement Learning Through Energy-Based Implicit Policy Constraints
Laminectomy represents one of the most frequently performed posterior spinal procedures. Because the lamina is situated in close proximity to critical neural structures, any operative damage can induce profound complications, potentially culminating in paralysis. To mitigate these risks and associated adverse events, surgical robotics and ultrasonic bone scalpels have been integrated into spinal laminectomy procedures, prompting extensive academic investigation into methods for detecting bone tissue states. At present, virtually all state-recognition methodologies rely on sensor signals acquired via high-precision instrumentation integrated into the end-effectors of surgical robots. Nevertheless, previous approaches have struggled to achieve highly accurate identification of spinal bone tissue states. In an innovative departure, this study frames the identification of bone tissue states as a time-series classification problem. The LSTM-FCN classification algorithm was deployed to analyze multi-sensor fusion signals combining force and cutting depth data, thereby enabling precise categorization of the laminal bone tissue state. Furthermore, experimental validation demonstrated that this approach achieves an accuracy of 98.85% in identifying bone tissue states during porcine spinal laminectomy. Crucially, the maximum penetration depth can be restricted to within 0.6 mm, confirming its safety and viability for practical clinical application.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2023 | Article: 35

A Decade of Explainable and Interpretable Artificial Intelligence in Healthcare: A Bibliometric Perspective
This review investigates the expanding influence of machine learning and deep learning methodologies within the healthcare sector, concentrating specifically on the vital challenges of interpretability and explainability inherent in black-box models. Although machine learning frameworks are increasingly deployed to assist in medical analytics and diagnostics, their intricate nature emphasizes the necessity of comprehending how these systems process and interpret data to facilitate well-informed clinical choices. This paper offers a thorough examination of the obstacles and potential solutions documented in existing scholarship, delivering a synthesis of contemporary techniques in this domain. Furthermore, it establishes precise conceptual boundaries for interpretability and explainability, seeking to untangle the differences between these notions and outline their consequences for clinical decision-making. Utilizing a dataset of 448 publications to address seven distinct research questions, our investigation demonstrates that this research area has experienced exponential growth over the past ten years. Additionally, the psychological factors governing public opinion highlight the vital need for transparent communication regarding the capabilities and constraints of artificial intelligence. To mitigate opacity, investigators are actively engineering strategies to lower model complexity and implement visual explanation tools. Nevertheless, maintaining a delicate equilibrium between high predictive performance and adequate model interpretability remains an ongoing challenge. Given the expanding role of artificial intelligence in supporting clinical diagnostics and therapeutics, the development of inherently interpretable artificial intelligence systems is critical. Within this rapidly changing paradigm, a steadfast dedication to transparency, ethical standards, and cross-disciplinary partnerships is essential to guarantee the conscientious deployment of artificial intelligence. This shared dedication is foundational to cementing long-term trust between medical professionals and patients, resolving novel problems, and promoting the educated integration of these sophisticated technologies into medical practices.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 41

Bridging AI Innovation and Regulatory Compliance in Financial Services: A Taxonomy of Regulatory Principles
The critical nature of explainable artificial intelligence and machine learning (XAI/XML) is gaining widespread recognition. This field focuses on uncovering how input variables influence automated choices, identifying systemic algorithmic bias, and detecting vulnerability to anomalies in data. Much of the existing literature centers on creating post hoc explanations tailored for opaque, black box architectures. These secondary techniques introduce novel vulnerabilities to error without correcting the fundamental flaws of the core model. Conversely, the development of inherently transparent systems receives far less attention. This paper introduces an inherently interpretable strategy rooted in the principles of ensemble learning, designated as the League of Experts (LoE) framework. We construct the initial theoretical foundations and subsequently derive a flexible, modular meta-algorithm. Although our presentation focuses primarily on classification tasks, the LoE framework adapts identically to regression scenarios. For classification specifically, we deploy traditional decision trees to form classifier ensembles as a concrete implementation. This paradigm simplifies the extraction of human-comprehensible decision rules for the underlying classification task, yielding an independent rule-induction system called RuleLoE. Beyond evaluating 12 KEEL classification datasets, we validate the LoE framework using two benchmarks from highly critical domains—healthcare and finance—to demonstrate its utility. The predictive capability of LoE matches contemporary state-of-the-art classification algorithms regarding both accuracy and rule coverage. Furthermore, LoE outputs a highly transparent group of decision rules featuring customizable complexity to map out the target classification problem. LoE provides a dependable approach for handling classification and regression problems. Its predictive precision is highly suited for environments where discovering root causal mechanisms is the primary objective, rather than merely maximizing predictive or classification performance.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 44

Towards Trustworthy Brain Tumor Diagnosis: An Explainable Deep Learning Framework for MRI Analysis
Among the most critical and fatal medical conditions are brain malignancies, making early identification vital for enhancing survival rates. Currently, magnetic resonance imaging (MRI) stands as the premier diagnostic tool for early detection because of its exceptional capability to capture soft tissue details. Nevertheless, relying solely on human interpretation of brain MRI scans introduces risks of oversight, which are frequently exacerbated by the fatigue and varying expertise of radiologists. To mitigate these limitations, computer-aided diagnosis (CAD) systems have taken on an increasingly vital role. Leveraging state-of-the-art computer vision methodologies like deep learning, these systems generate highly precise diagnoses from clinical imagery, thereby boosting both precision and consistency in clinical settings. This study introduces an innovative CAD architecture designed for multi-class classification of brain malignancies. The system evaluates six pre-trained deep learning networks as its foundational architecture, integrating robust data preprocessing and augmentation protocols to optimize computational throughput. To counter the opaque "black box" nature of deep learning architectures and improve interpretability, Gradient-weighted Class Activation Mapping (Grad-CAM) is integrated to illuminate the network's deductive steps when identifying anomalies in MRI data. Furthermore, an accessible, web-based Brain Tumor Detection System was constructed using the Streamlit framework, highlighting its utility for clinical environments and offering meaningful support to medical professionals. All experimental simulations were carried out using an open benchmark repository, proving that the introduced system delivers state-of-the-art diagnostic capabilities, achieving accuracy levels near 99% with the ResNet-50, Xception, and InceptionV3 architectures.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 45

From Data to Arguments: A Unified Paradigm for Explainable Artificial Intelligence
Explainable Artificial Intelligence (XAI) represents an active discipline dedicated to clarifying the internal mechanisms of AI systems, thereby fostering stakeholder confidence and advancing the deployment of accountable technology. A primary theoretical hurdle within XAI involves engineering techniques capable of producing clear, accessible interpretations without sacrificing scalability or operational performance in intricate domains. While rule-based methodologies offer a framework for decoding AI-driven conclusions, they are prone to becoming overly complex in dense environments, which diminishes both their comprehensibility and computational scalability. Furthermore, these approaches frequently overlook contrastive logic, which leaves observers unable to discern why a specific outcome was prioritized over alternatives. To resolve this underlying scientific problem, this research investigates the fusion of computational argumentation—a specialized domain of AI that replicates cognitive processes through defeasible reasoning—with rule-based XAI architectures. Computational argumentation provides a mechanism whereby arguments derived from structured rules can be invalidated or withdrawn when confronted with conflicting evidence. Consequently, this framework offers a compelling avenue for refining rule-based systems to yield more transparent AI models. Despite the synergistic benefits of combining rule-based structures with computational argumentation, academic inquiry into their synthesis remains sparse. Accordingly, this paper also targets the practical engineering challenge of realizing this conceptual integration within functional AI software. This investigation utilizes the Logic Learning Machine (LLM)—a specialized rule-extraction methodology—and introduces a modular framework designed to map input rules directly into a structured argumentation network via state-of-the-art argumentation techniques. Empirical evaluations performed on various binary classification problems from the UCI Machine Learning Repository validate the utility of this combined model. The LLM strategy proved highly proficient at extracting a compact, interpretable collection of if-then rules characterized by minimal antecedent premises, all while sustaining strong predictive accuracy across all evaluated datasets. Concurrently, the argumentation-based models delivered classification performance comparable to the native if-then rule configurations, achieving this through a streamlined rule set while offering vastly superior explainability. Ultimately, this work delivers an innovative framework for the automated and streamlined generation of arguments and their internal conflicts directly from raw data, simultaneously addressing fundamental theoretical and practical barriers to propagate argumentation frameworks within the XAI landscape.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 48

Explainable Machine Learning for Early Prediction of Unscheduled Hospital Readmissions: Integrating Global and Local Interpretability
At present, the majority of healthcare spending is driven by chronic patients who experience multiple readmissions due to their underlying conditions. Implementing individualized prevention frameworks could optimize the clinical management of this demographic. This study aimed to generate localized predictive models utilizing interpretable machine learning approaches to provide early identification of individual unscheduled hospital readmissions. To achieve this, a retrospective case-control investigation was performed, utilizing patient readmission data spanning 2018–2019. Following the cleaning and curation of the initial data repository (n = 76,210), the final sample size comprised n = 29,026 participants. A machine learning analysis evaluated multiple algorithms, employing unscheduled hospital readmission as the dependent variable, alongside local model-agnostic interpretability techniques.The results demonstrated a 13% rate of unscheduled hospital readmissions. Statistically significant variations were identified regarding patient age and length of hospital stay (p < 0.001 in both instances). A logistic regression framework highlighted chronic therapy (odds ratio: 3.75), a history of diabetes mellitus (odds ratio: 1.14), and length of stay (odds ratio: 1.02) as statistically meaningful predictors. Machine learning algorithms demonstrated superior performance concerning sensitivity and complementary evaluation metrics. Within this analytical framework, length of stay and patient age emerged as the primary determinants for forecasting unscheduled readmissions. Notably, other clinical indicators, such as documented allergies and histories of adverse drug reactions, also demonstrated predictive relevance. Individualized prognostic models exhibited high sensitivity. In conclusion, this investigation identified critical determinants governing unscheduled hospital readmissions, highlighting the prominence of age and hospitalization duration. A personalized risk assessment framework capable of predicting readmissions with high accuracy was established. Future investigations should incorporate a broader array of clinical parameters to further optimize the predictive capacity of this model.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 49

Leveraging Chain-of-Thought Prompt Engineering to Enhance Knowledge Creation through Improvable Student Ideas
Within a student learning community, the generation of new knowledge represents a vital mechanism for expanding collective understanding and driving original synthesis. Students assume critical responsibilities when they target knowledge deficiencies and engage in cooperative efforts to refine communal concepts emerging from dialogue. Nonetheless, the standard of these concepts can frequently be constrained by resource deficits or a narrow spectrum of viewpoints. The introduction of generative Artificial Intelligence and large language models (LLMs) into educational settings has unlocked new opportunities for analyzing and enriching idea-focused debates that were formerly unachievable. Maximizing the utility of LLMs necessitates specialized competencies in prompt engineering, particularly regarding the in-context methodology termed Chain-of-Thought (CoT) to formulate and scaffold progressive concepts within student conversations. This investigation gathered a dataset of 721 conversational entries—encompassing 272 applicable question–answer dynamics and 149 distinct discussion tracks—produced by 31 participants during a 2-day student Knowledge Building Design Studio (sKBDS). The researchers augmented student inputs utilizing the CoT technique and subsequently evaluated the AI-generated outputs against the original student assertions. The outcomes are demonstrated through two explicit conversational tracks, confirming that CoT-configured prompts enable LLMs to output responses that actively advance malleable concepts during knowledge construction. This inquiry examines data from genuine classroom discourse and offers actionable insights for both educational research and instructional implementation.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 51

Explainable Machine Learning for Long-Term Forecasting of Patient Demand in Urgent Care Clinics
Global emergency departments and urgent care facilities regularly experience prolonged waiting periods that surpass patient expectations, primarily driven by unexpected spikes in patient volumes. The resulting delays caused by insufficient staffing during these high-demand intervals have been associated with detrimental clinical consequences. Prior investigations into predicting patient volumes have predominantly relied on traditional statistical approaches. Furthermore, these efforts have mainly focused on short-range predictions, which offer minimal utility for the long-term scheduling of medical staff. This paper contributes to a growing domain of research aimed at investigating the capacity of machine learning frameworks to produce precise patient volume projections. Utilizing a decade's worth of data from two major urgent care centers, this study constructs long-range patient flow predictions extending up to one quarter in advance by evaluating a selection of modern, high-performing algorithms. A unique aspect of this research is the integration of eXplainable Artificial Intelligence (XAI) frameworks, specifically LIME and Shapley, to conduct a comprehensive assessment of internal model dynamics that are typically obscured. These diagnostic instruments allowed us to examine how effectively the predictive frameworks adjusted to volatile demand shifts induced by COVID-19 pandemic lockdowns, while also determining the most significant predictor variables to gain deep operational insights. The empirical findings indicate that an innovative ensemble combining gradient boosting techniques with advanced univariate frameworks like Prophet yielded the most precise and dependable performance overall. On average, this methodology produced a 16%–30% improvement over the current internal forecasting techniques utilized for predicting daily patient volumes 90 days into the future.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 53

Explainable CNN-Based Cavitation Detection in Axial Piston Pumps Using Grad-CAM Visualization
The reliability and safety of an entire hydraulic system are compromised by the occurrence of cavitation in axial piston pumps. Because vibration signals can indicate cavitation states within these pumps, they have been increasingly coupled with machine learning techniques for detection purposes. Nevertheless, raw vibration signals captured under actual operating conditions are frequently corrupted by noise, complicating the precise identification of cavitation within noisy environments. To address this issue, this study introduces an intelligent methodology designed to identify cavitation in axial piston pumps under high-noise conditions. Initially, a convolutional neural network (CNN) is trained utilizing spectrogram images that are converted from raw, unstructured vibration datasets across varying cavitation states. Subsequently, the gradient-weighted class activation mapping (Grad-CAM) technique is applied to highlight and visualize the class-discriminative zones within these spectrogram images. Ultimately, a novel image processing approach leveraging the Grad-CAM heatmap is introduced to automatically filter out background noise and amplify the key class features in the spectrograms. The empirical outcomes demonstrate that this strategy substantially boosts the diagnostic efficacy of the CNN framework under noisy conditions. Specifically, the accuracy of cavitation condition classification is elevated from 0.50 to 0.89 at a signal-to-noise ratio of 4 dB, and from 0.80 to 0.92 at 6 dB.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 55

Toward Fair AI Development: How Developers Can Address Algorithmic Bias
Artificial intelligence (AI) has swiftly emerged as a core technology for securing a competitive advantage. Nonetheless, there are escalating anxieties regarding bias within AI models, given that AI developers are susceptible to introducing bias both inadvertently and intentionally. Utilizing a qualitative methodology, this study explored how AI developers can actively foster the creation of equitable AI models. The primary results indicate that the danger of bias stems predominantly from a deficiency in social and gender diversity within AI development teams, alongside pressure from AI managers to rapidly deliver highly anticipated outcomes. Furthermore, the integrity of AI developers is critical, as they possess the ability to hide biases from management and other relevant AI stakeholders. The testing phase preceding model deployment is also prone to bias because it seldom reflects the diverse social demographics that could be impacted.This research offers actionable recommendations across four core dimensions: governance, social, technical, and training and development procedures. Responsible organizations must implement deliberate measures to guarantee that their AI developers follow equitable protocols during AI creation; AI developers need to put ethical considerations first and evaluate the societal impacts of their models; collaborative partnerships ought to be formed among AI developers, AI stakeholders, and the communities likely to be affected by these models; and AI developers must emphasize transparency and explainability within their models while ensuring robust bias testing and remediation prior to deployment. Lastly, providing emotional intelligence training to AI developers is recommended to equip them for constructive dialogue with stakeholders outside the immediate development team.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2024 | Article: 56

Enhancing Transparency in Spatial PM₂.₅ Prediction over Singapore through Explainable Machine Learning
The climate system is heavily influenced by aerosols through direct and indirect pathways, notably the absorption and scattering of radiant energy. Beyond climate impacts, these particles compromise atmospheric visibility and jeopardize public health. Specifically, human populations face exposure to fine particulate matter PM2.5, a major driver of respiratory and cardiovascular pathologies. Long-term modifications in PM levels are generally governed by emission shifts and meteorological patterns, whereas short-term fluctuations are predominantly dictated by immediate weather dynamics. Surface variations in PM2.5 levels are further driven by environmental variables such as temperature, wind velocity, relative humidity, and vegetative coverage.Air quality forecasting has advanced significantly through the application of machine learning. This research explores PM2.5 estimation by utilizing these environmental and meteorological metrics as spatiotemporal inputs. This study evaluates seven years (2014–2021) of in situ PM2.5 measurements gathered from five distinct monitoring stations across Singapore, establishing a baseline dataset for spatial modeling.The primary contribution of this paper is a novel predictive architecture tailored for temporal analysis, utilizing Random Forest (RF), Gradient Boosting (GB) regression, and the Tree-based Pipeline Optimization Tool (TP) Auto ML framework, which optimizes via a meta-heuristic genetic algorithm. The TP model demonstrated strong predictive capabilities, achieving a peak Global Performance Index (GPI) of 7.4 in August 2016 and a minimum value of −0.6 in June 2019. These metrics highlight the robust nature of the TP framework; its negative values remained superior to competing models, indicating less pessimistic errors. Finally, the underlying logic of these machine learning systems is clarified using eXplainable Artificial Intelligence (XAI) methods, which validate the integrity of feature importance and offer deeper insights into the cyclical variations of PM2.5 concentrations.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 61

Towards Explainable Deep Reinforcement Learning: Interpreting Experience Replay through SHAP
Although Reinforcement Learning (RL) exhibits significant potential for refining intricate control and decision-making systems, Deep Reinforcement Learning (DRL) suffers from a distinct lack of explainability. This limitation restricts its deployment within highly regulated domains, including manufacturing, finance, and healthcare. The opacity of DRL decision-making introduces structural challenges that impede operational efficiency and resource management—an issue that intensifies alongside ongoing algorithmic advancements. While current trends favor transitioning from traditional Experience Replay configurations to A3C architectures, the latter imposes substantially greater resource demands. Furthermore, despite continuous efforts to refine selection mechanisms within Experience Replay, repository capacities generally remain elevated. In this study, we evaluate a Deep Convolutional Q-learning agent across 20 Atari environments while deliberately downsizing the Experience Replay capacity from 1×106 to 5×102. Our findings indicate that lowering the capacity from 1×104 to 5×103 does not meaningfully degrade performance rewards, thereby establishing a viable strategy for resource-efficient DRL frameworks. To clarify agent choices and correlate them with inherent game mechanics, we introduce a novel methodology: visualizing the Experience Replay mechanism utilizing the Deep SHAP Explainer. This framework enhances domain comprehension and delivers transparent, interpretable accounts, though capacity constraints must be managed carefully to mitigate overfitting. Ultimately, this research validates the practicality of minimizing Experience Replay storage and supports the integration of transparent explanations via the Deep SHAP Explainer to advance resource conservation.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2025 | Article: 65

Explainable Machine Learning for COVID-19 Prognosis: Assessing Variant Influence and Clinical Risk Factors
Machine learning (ML) techniques have been applied in diverse manners to combat the COVID-19 pandemic. Predictive ML models have been constructed, for example, for both diagnostic and prognostic objectives, utilizing varying data modalities such as textual, visual, and structured formats. Because of the distinct characteristics of this disease and its temporal transformation, a comprehensive understanding of the central elements driving the clinical trajectory of COVID-19 in individual patients remains incomplete. Throughout all phases of this study, a medical specialist was actively engaged via a human-in-the-loop framework. This collaborative element represents a vital yet frequently overlooked component of the ML and knowledge extraction (KE) workflow. Our findings indicate that explainable artificial intelligence (XAI) can offer substantial reinforcement to this dimension of ML and KE. This investigation centered on employing ML for knowledge extraction across two distinct operational scenarios. In the initial scenario, we sought to determine if integrating data regarding the dominant COVID-19 variant affects the predictive performance of the ML models. In the subsequent scenario, we concentrated on prognostic classification models designed to predict a patient's requirement for intensive care unit admission, integrating various XAI methodologies. We implemented nine distinct ML techniques, specifically XGBoost, CatBoost, LightGBM, logistic regression, Naive Bayes, random forest, SGD, SVM-linear, and SVM-RBF. The predictive quality of these models was evaluated using precision, accuracy, and AUC metrics. Following this evaluation, knowledge extraction was performed on the highest-performing models through two discrete pathways: (a) automated feature extraction using forward stepwise selection (FSS); and (b) the identification of attributes and their interactions using model explainability tools. Both sets of features were evaluated against a baseline configuration of attributes selected a priori by medical professionals using domain expertise. Our empirical tests demonstrated that incorporating data on the prevailing COVID-19 variant did not alter the performance of the generated ML models. Furthermore, the results indicated that human medical experts displayed significantly higher precision in identifying meaningful attributes compared to the automated FSS approach. Explainability tools identified nearly identical sets of attributes as the medical expert, while additionally uncovering noteworthy attribute interactions that the specialist subsequently evaluated from a clinical perspective. The outcomes of this investigation and their broader implications are examined.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2025 | Article: 66