Description
Identifying the layout semantics of historical Sino-Nom documents is a challenging problem at the intersection of document image analysis and cultural heritage preservation. Sino-Nom manuscripts exhibit severe degradation, heterogeneous writing styles, and highly varied structural conventions across genres, making automated layout understanding difficult. I propose a multi-modal pipeline that classifies each page into one of six semantic layout categories. Our system integrates three modalities: a specialized OCR module for historical Sino-Nom script; a PaddleLayout-based component identifying 27 structural region types; and a Vision-Language Model that jointly reasons over page appearance, OCR output, and layout summaries to produce the final label. I construct a domain-specific dataset of 450 annotated pages across six categories, built on the SCUT-CAB corpus. Experimental results show that multi-modal fusion outperforms unimodal baselines, achieving a macro F1-score of 66% across six classes.
Từ khóa
Sino-Nom documents, layout analysis, document classification, Vision-Language Model, OCR, multi-modal fusion, historical document understanding, cultural heritage preservation
Thông tin các tác giả
1/Nguyễn Quốc Bảo: Cử nhân, đang công tác tại Trường Đại học Khoa học Tự nhiên TP. HCM, 227 Nguyễn Văn Cừ, Chợ Quán, Hồ Chí Minh, email: 24C01003@student.hcmus.edu.vn
