TY - JOUR
T1 - Following the Evidence Across Modalities: A Conceptual Architecture for Explainability in Text, Vision, Audio, Sensor, and Structured-Data Intelligent Systems
AU - Amelia Clarke
AU - Carlos Mendoza
JF - Journal of Trustworthy and Explainable Intelligent Systems
JO - J Trustworthy Explain Intell Syst
Y1 - 2026
VL - 5
IS - 1
SP - 87
N2 - Multimodal intelligent systems combine evidence from text, vision, audio, sensors, and structured data, but their explanations often remain fragmented into separate saliency maps, feature scores, textual rationales, attention visualisations, or confidence indicators. Such outputs may describe isolated aspects of a model while obscuring how modality-specific evidence was represented, transformed, fused, and communicated. This article develops a conceptual multimodal explainability architecture organised around the principle of following evidence across modalities. The approach begins by distinguishing the evidential and representational characteristics of textual, visual, acoustic, temporal-sensor, and structured inputs. It then identifies attribution difficulties created by early, intermediate, late, and hybrid fusion, including modality dominance, missing evidence, conflicting inputs, representation alignment, and uncertainty about joint contributions. Evidence traceability is treated as an end-to-end requirement connecting input provenance, modality-specific representation, fusion operations, attribution processes, consistency assessment, uncertainty communication, and decision explanation. The architecture further incorporates human interpretation because technically detailed multimodal explanations may remain cognitively inaccessible, visually persuasive, incomplete, or poorly calibrated to user expertise and decision context. Its central contribution is a layered framework linking modality inventory, evidence classification, representation mapping, fusion attribution, provenance, consistency review, human-centred presentation, evaluation, and feedback updating. The architecture is intended to structure research and design reasoning rather than prescribe a particular model, explanation technique, evaluation metric, or deployment procedure. It does not establish that an explanation is faithful, that users understand it, that reliance is appropriate, or that the underlying intelligent system is trustworthy or ready for operational use.
UR - https://tipublications.com/a289871048
ER -