Transparent Intelligence Publishing Transparent Intelligence Publishing

Search

Search results:
Bridging Hand-Crafted Features and Deep Learning Through an Explainable Shallow CNN for Medical Image Segmentation
In clinical environments, the transparency of artificial intelligence models is paramount, as understanding the rationale behind automated choices is vital. Although black-box AI frameworks exhibit remarkable efficacy, their deployment in healthcare remains impractical unless their underlying mechanisms can be clarified for medical professionals. Deep neural networks (DNNs) achieve state-of-the-art results, yet the justifications they provide are frequently incomprehensible to human observers. Conversely, traditional machine learning approaches and manually designed features extracted to capture distinct data characteristics offer greater transparency, though human constraints in feature engineering often limit their overall effectiveness. To bridge this gap, we present ExShall-CNN, a novel and transparent shallow convolutional neural network tailored for medical image analytics. This framework builds upon hand-crafted feature methodologies to preserve human interpretability, guaranteeing clear and verifiable decision-making. We introduce the explainable shallow convolutional neural network (ExShall-CNN), which merges the high performance of sophisticated deep architectures like U-Net with the inherent clarity of hand-crafted features for segmenting medical images. Grounded in contemporary machine learning progress, ExShall-CNN integrates widely utilized kernels while maintaining structural openness, allowing medical practitioners to visually inspect and comprehend its outputs. This balanced strategy delivers the precision expected of deep learning paradigms alongside the explainability required for clinical deployment.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2022 | Article: 12

Advancing AI-Augmented Business Process Management through Knowledge-Driven Explainability and Agile Knowledge Sharing
Merging Artificial Intelligence (AI) with Business Process Management Systems (BPMSs) has given rise to AI-Augmented Business Process Management Systems (ABPMSs). These modern setups deliver flexible adaptability, instantaneous process refinement, and superior knowledge management workflows. Yet, notable hurdles persist, especially regarding contextual clarity, user buy-in, and the incorporation of behavioral dynamics. This research introduces a novel conceptual model that unifies the Socialization, Externalization, Combination, and Internalization (SECI) knowledge paradigm with Agile workflows (specifically Scrum) and advanced AI capabilities like explainable AI (XAI), process mining, and Robotic Process Automation (RPA). This setup streamlines the formalization, validation, and dissemination of insights through structured, intuitive software and collaborative frameworks, particularly Communities of Practice (CoPs). The system highlights context-sensitive transparency, step-by-step deployment, and iterative refinement to foster productive human–AI partnerships. It delivers both conceptual and practical blueprints for linking AI strengths with corporate knowledge assets. The introduced model eases the upgrade path from legacy BPMSs to advanced ABPMSs by utilizing systematic practices and technologies. This strategy accelerates knowledge distribution and operational refinement, mapped out via exhaustive BPMN 2.0 diagrams.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2023 | Article: 27

Bridging AI Innovation and Regulatory Compliance in Financial Services: A Taxonomy of Regulatory Principles
The critical nature of explainable artificial intelligence and machine learning (XAI/XML) is gaining widespread recognition. This field focuses on uncovering how input variables influence automated choices, identifying systemic algorithmic bias, and detecting vulnerability to anomalies in data. Much of the existing literature centers on creating post hoc explanations tailored for opaque, black box architectures. These secondary techniques introduce novel vulnerabilities to error without correcting the fundamental flaws of the core model. Conversely, the development of inherently transparent systems receives far less attention. This paper introduces an inherently interpretable strategy rooted in the principles of ensemble learning, designated as the League of Experts (LoE) framework. We construct the initial theoretical foundations and subsequently derive a flexible, modular meta-algorithm. Although our presentation focuses primarily on classification tasks, the LoE framework adapts identically to regression scenarios. For classification specifically, we deploy traditional decision trees to form classifier ensembles as a concrete implementation. This paradigm simplifies the extraction of human-comprehensible decision rules for the underlying classification task, yielding an independent rule-induction system called RuleLoE. Beyond evaluating 12 KEEL classification datasets, we validate the LoE framework using two benchmarks from highly critical domains—healthcare and finance—to demonstrate its utility. The predictive capability of LoE matches contemporary state-of-the-art classification algorithms regarding both accuracy and rule coverage. Furthermore, LoE outputs a highly transparent group of decision rules featuring customizable complexity to map out the target classification problem. LoE provides a dependable approach for handling classification and regression problems. Its predictive precision is highly suited for environments where discovering root causal mechanisms is the primary objective, rather than merely maximizing predictive or classification performance.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2024 | Article: 44

Beyond Opening the Black Box—A Critical Review of Explainability for Human Understanding, Decision Usefulness, Trust Calibration, and Accountability
Explainability has become central to discussions of trustworthy and responsible intelligent systems, yet it is frequently reduced to the technical production of information about model structure, features, rules, or predictions. This critical review examines why opening a computational black box is not equivalent to producing an explanation that people can understand or use responsibly. The review synthesises conceptual, technical, human-centred, decisional, and accountability-oriented scholarship to evaluate the shift from model interpretability toward meaningful explanation. It distinguishes technical properties such as transparency, fidelity, plausibility, completeness, stability, and local or global scope from human outcomes such as comprehension, cognitive accessibility, mental-model development, decision usefulness, calibrated trust, and appropriate reliance. It further examines how explanations may produce overreliance, underreliance, automation bias, false reassurance, cognitive overload, or explanation mismatch when their content and presentation are not aligned with users, tasks, uncertainty, and decision consequences. From an accountability perspective, explanations may contribute to auditability, traceability, contestability, and recourse, but they cannot independently assign responsibility, provide institutional authority, or guarantee effective remedies. The main contribution is a human-centred critical synthesis that treats explainability as a socio-technical achievement rather than a model output. Meaningful explanation requires technical credibility, cognitive accessibility, contextual relevance, decision-sensitive evaluation, uncertainty communication, opportunities for challenge, and governance mechanisms capable of acting on explanatory evidence. The review therefore rejects universal explanation-sufficiency claims and identifies conditions under which explanations may assist understanding and responsible decision use without being treated as proof of trustworthiness, accountability, fairness, safety, or deployment readiness.
Journal of Trustworthy and Explainable Intelligent Systems
Review | Open access | 15 March 2025 | Article: 75

Building Trustworthiness into Intelligent Systems: From Data Provenance and Model Reliability to Human Control and Institutional Accountability
Intelligent systems increasingly influence decisions, allocate attention, generate recommendations, and mediate interactions across technical, organizational, and social environments. Yet trustworthiness is often approached through isolated interventions, such as improving model accuracy, adding post-hoc explanations, documenting datasets, or assigning nominal human oversight. This article proposes a trustworthy-by-design conceptual framework that treats trustworthiness as a system-level property requiring coordinated design and evaluation across the full intelligent-system lifecycle. The framework begins with system-purpose and risk-context definition, followed by stakeholder and responsibility mapping, data-integrity assessment, provenance and lineage review, data documentation and governance, model-reliability evaluation, robustness and uncertainty assessment, and bias and fairness review where relevant. It then connects explanation quality with human interpretability and decision usefulness, rather than assuming that technically generated explanations will necessarily support understanding. Human oversight, meaningful control, contestability, recourse, institutional accountability, traceability, auditability, monitoring, and feedback updating form additional interdependent layers. The framework’s principal contribution is to clarify how technical evidence must be connected to human decision conditions and institutional responsibility without treating any individual safeguard as sufficient. Data provenance does not establish data appropriateness, robustness does not prove safety, explanations do not guarantee interpretability, human presence does not ensure meaningful control, and governance arrangements do not independently establish accountability. The framework may support conceptual analysis, assurance-oriented evaluation, and lifecycle governance planning, but requires empirical validation, human-centered assessment, domain-specific interpretation, and continuing institutional review.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 September 2025 | Article: 76

Making Machine Reasoning Open to Question: An Integrative Review of Causal, Neuro-Symbolic, Knowledge-Grounded, and Uncertainty-Aware Explainability
Explainable artificial intelligence increasingly seeks to provide more than visualisations of feature importance or simplified descriptions of model outputs. For intelligent systems used in consequential settings, machine reasoning should be open to questioning, inspection, contextualisation, correction, and evaluation under uncertainty. This integrative review examines emerging approaches that may support this broader objective. It connects causal explainability and counterfactual reasoning with neuro-symbolic intelligence, knowledge-grounded inference, uncertainty-aware explanation, reliability evaluation, and human-centred interpretability. The review uses a structured integrative approach to compare how these paradigms represent causes, alternatives, rules, concepts, knowledge sources, inferential steps, confidence, and limitations. Causal and counterfactual approaches can support questions about mechanisms, contrasts, interventions, actionability, and recourse, but their conclusions depend on explicit assumptions, feasible alternatives, and appropriate causal evidence. Neuro-symbolic and knowledge-grounded approaches can make concepts, relations, rules, provenance, and intermediate reasoning more traceable, although symbolic structure or retrieved knowledge does not automatically ensure faithful, valid, or truthful explanation. Uncertainty-aware approaches can qualify predictions and explanations through uncertainty characterisation, calibration, confidence communication, robustness assessment, and human interpretation, but uncertainty estimates alone do not establish reliability. The synthesis indicates that no individual paradigm is sufficient. More defensible machine reasoning requires complementary evaluation of explanation fidelity, causal plausibility, knowledge validity, reasoning traceability, uncertainty communication, cognitive accessibility, contestability, and lifecycle accountability. Important research opportunities concern causal validation, feasible recourse, provenance governance, explanation uncertainty, hybrid-system evaluation, human-centred testing, and cross-paradigm assessment. The review contributes an integrated account of how machine reasoning may become not merely visible, but meaningfully open to question.
Journal of Trustworthy and Explainable Intelligent Systems
Review | Open access | 15 March 2026 | Article: 83

Responsible Foundation Models Need More Than Alignment: A Governance and Technical Safeguard Framework for Explainability, Reliability, Human Control, and Institutional Accountability
Foundation models are increasingly embedded in information access, content production, analysis, communication, and decision-support processes. Their broad capabilities have intensified interest in alignment, yet alignment alone cannot establish responsible use. A model may appear responsive to instructions or preferences while remaining opaque, weakly grounded, unreliable under changing conditions, overconfident, difficult to contest, or insufficiently governed. This article proposes a conceptual governance and technical safeguard framework for responsible foundation models. The framework treats foundation models as components of decision infrastructure situated within socio-technical and institutional environments rather than as isolated predictive tools. It integrates eight interdependent areas: model and alignment safeguards; explainability and interpretability; knowledge grounding and provenance; reliability, uncertainty, and hallucination containment; human control and escalation; institutional accountability; auditability and documentation; and lifecycle monitoring and governance updating. Explainability requirements are differentiated from interpretability, transparency, explanation usefulness, and human understanding, while knowledge grounding is separated from factuality, truthfulness, source quality, and provenance. Reliability is addressed through calibration, robustness assessment, boundary testing, uncertainty communication, failure containment, and continuing monitoring. Human oversight is defined through authority, competence, review capacity, escalation pathways, contestability, and recourse rather than nominal human presence. Institutional responsibility is connected to role allocation, evidence trails, documentation, review authority, and consequences. The principal contribution is an integrated framework that clarifies how technical and governance safeguards may support responsible foundation-model assessment without treating alignment, explanation, grounding, testing, oversight, or auditing as independently sufficient. The framework requires empirical validation, domain-specific evaluation, human-centred assessment, institutional governance, and residual-risk management before it can inform consequential operational decisions.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2026 | Article: 84

Following the Evidence Across Modalities: A Conceptual Architecture for Explainability in Text, Vision, Audio, Sensor, and Structured-Data Intelligent Systems
Multimodal intelligent systems combine evidence from text, vision, audio, sensors, and structured data, but their explanations often remain fragmented into separate saliency maps, feature scores, textual rationales, attention visualisations, or confidence indicators. Such outputs may describe isolated aspects of a model while obscuring how modality-specific evidence was represented, transformed, fused, and communicated. This article develops a conceptual multimodal explainability architecture organised around the principle of following evidence across modalities. The approach begins by distinguishing the evidential and representational characteristics of textual, visual, acoustic, temporal-sensor, and structured inputs. It then identifies attribution difficulties created by early, intermediate, late, and hybrid fusion, including modality dominance, missing evidence, conflicting inputs, representation alignment, and uncertainty about joint contributions. Evidence traceability is treated as an end-to-end requirement connecting input provenance, modality-specific representation, fusion operations, attribution processes, consistency assessment, uncertainty communication, and decision explanation. The architecture further incorporates human interpretation because technically detailed multimodal explanations may remain cognitively inaccessible, visually persuasive, incomplete, or poorly calibrated to user expertise and decision context. Its central contribution is a layered framework linking modality inventory, evidence classification, representation mapping, fusion attribution, provenance, consistency review, human-centred presentation, evaluation, and feedback updating. The architecture is intended to structure research and design reasoning rather than prescribe a particular model, explanation technique, evaluation metric, or deployment procedure. It does not establish that an explanation is faithful, that users understand it, that reliance is appropriate, or that the underlying intelligent system is trustworthy or ready for operational use.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2026 | Article: 87

From Prediction to Intervention: A Causal Explainability Framework for Counterfactual Reasoning, Mechanism Discovery, Confounding Control, and Decision Validity
Explainable artificial intelligence increasingly supports decisions whose consequences extend beyond prediction, yet many explanations remain grounded in associations, feature contributions, saliency, or local perturbations. Such outputs may clarify model behaviour without establishing what would happen under intervention, why an outcome arose through a particular mechanism, or whether a proposed change is feasible and decision-valid. This article develops a conceptual causal explainability framework for moving from prediction-oriented explanation toward intervention-aware reasoning. The framework begins by separating predictive, associative, and model-behaviour claims from causal questions. It then connects explicit causal assumptions with causal structures, mechanistic reasoning, interventional explanation, counterfactual reasoning, confounding assessment, identifiability, sensitivity, external validity, uncertainty communication, and human interpretation. Particular attention is given to the distinction between model counterfactuals and real-world recourse, the risks created by unmeasured confounding, selection and collider bias, and the need to define decision boundaries before explanations inform action. Human users are treated as active interpreters whose causal literacy, expertise, cognitive constraints, and susceptibility to false causal confidence affect explanation usefulness. The main contribution is an integrated framework that organizes the evidential transition from prediction to intervention while preserving explicit no-guarantee boundaries concerning causal truth, intervention validity, recourse, identifiability, human understanding, decision validity, and deployment readiness. The framework is intended to support disciplined conceptual development and evaluation planning, not to prescribe a universal causal model, intervention procedure, or operational decision system.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2026 | Article: 88

The Hidden Cost of Unexplained AI: A Conceptual Model of Explainability Debt, Documentation Gaps, Interface Misalignment, Stakeholder Exclusion, and Accumulating Governance Risk
Artificial intelligence systems may remain operationally usable while carrying unresolved explanation-related deficiencies that are difficult to observe during initial development. These deficiencies can emerge later as opaque model behavior, uncertain data provenance, incomplete documentation, broken traceability, unsuitable explanation interfaces, excluded stakeholder perspectives, weak contestability, and ambiguous accountability. This article introduces explainability debt as a conceptual lifecycle risk arising when explanation design, documentation, traceability, stakeholder engagement, interface alignment, and governance review are deferred or inadequately maintained. The proposed approach adapts technical-debt reasoning without treating explainability debt as a measurable financial liability or validated risk score. It distinguishes model and data opacity from the organizational deficiencies that prevent opacity from being investigated, communicated, challenged, and corrected. Documentation and traceability deficits are interpreted as missing or deteriorating evidence relationships across datasets, model versions, explanations, decisions, and responsible actors. Interface misalignment and stakeholder exclusion are treated as mechanisms through which technical opacity can become misunderstanding, inappropriate reliance, limited participation, and weak reviewability. Unresolved deficiencies may subsequently contribute to accountability gaps, operational uncertainty, decision risk, and broader governance risk. The article’s main contribution is a conceptual model connecting deferred explainability work with accumulating lifecycle costs and feedback failures across development, deployment, monitoring, incident learning, correction, and model updating. The model supports structured reasoning about hidden explainability obligations but does not measure debt, prove trustworthiness, guarantee human understanding, establish legal or ethical sufficiency, certify governance, or determine deployment readiness.
Journal of Trustworthy and Explainable Intelligent Systems
Original Research | Open access | 15 March 2026 | Article: 94