Research Article
A Graph-Enhanced Multimodal Transformer Model for
Fine-Grained Document Parsing
Gabriel Ihuoma Lilian*,
Laud Charles Ochei
,
Martha Ozohu Musa
Issue:
Volume 15, Issue 1, February 2026
Pages:
1-18
Received:
31 July 2026
Accepted:
17 August 2026
Published:
9 September 2026
DOI:
10.11648/j.ijiis.20261501.11
Downloads:
Views:
Abstract: The digitization of documents across industries has created an urgent need for intelligent systems capable of extracting structured information from unstructured layouts. While transformer-based models like LayoutLMv3 have advanced document understanding, they struggle to capture relational dependencies between spatially separated but semantically related elements—a fundamental challenge in form understanding and layout analysis. This paper introduces IntelliDocFormer, an enhanced multimodal transformer that explicitly models relationships between document elements through graph-based relational reasoning. Building on LayoutLMv3, our architecture incorporates a Graph Attention Network encoder that captures both sequential and spatial dependencies between text tokens, complemented by document-type classification for adaptive processing. We evaluate IntelliDocFormer on DocBank (4,000 samples) and FUNSD (119 samples), achieving Macro-F1 scores of 0.7435 (DocBank) and 0.7450 (FUNSD); performance improved substantially when scaling from 50 to 4,000 DocBank training samples (a 6,045% relative gain over a near-zero low-data baseline). Ablation studies confirm that graph-based reasoning contributes the most significant performance gain (+0.04 Macro-F1). Our results demonstrate that explicit relationship modeling is critical for fine-grained document parsing, particularly for structurally complex elements like tables (F1=0.94) and paragraphs (F1=0.75). On the validated element types (tables, paragraphs), inference runs at 1.67 sec/sample on a single Tesla T4 GPU. These findings establish graph-enhanced multimodal transformers as a promising direction for document intelligence, with implications for automated data extraction across finance, healthcare, and legal domains, particularly for well-represented structural elements such as tables and paragraphs; performance on low-frequency structural classes (e.g., footers, section headers) currently limits applicability to compliance-sensitive extraction tasks. To probe generalization beyond these two domains, we additionally repurposed IntelliDocFormer's document-type classification head for 16-way real-world document classification on RVL-CDIP using a stratified 3,200-image subsample. This yielded a Macro-F1 of 0.22 (Accuracy 0.29) — well below both our DocBank/FUNSD results and literature-reported RVL-CDIP figures for other architectures — revealing that the document-type module, as configured here, does not yet generalize to fine-grained, small-sample, multi-class real-world classification; we report this result in full, together with its diagnostic analysis, as an explicit boundary condition on our generalization claims.
Abstract: The digitization of documents across industries has created an urgent need for intelligent systems capable of extracting structured information from unstructured layouts. While transformer-based models like LayoutLMv3 have advanced document understanding, they struggle to capture relational dependencies between spatially separated but semantically ...
Show More
