Automating UML Code Generation from Images: Leveraging Multimodal Large Language Models

Abstract

In software engineering, Unified Modeling Language (UML) is a widely used tool for visually representing and analyzing complex systems. However, translating UML diagrams, particularly those stored in non-editable formats, into executable code remains challenging due to manual and error-prone workflows. This thesis introduces an automated approach for UML-to-code generation by leveraging multimodal large language models (MM-LLMs), specifically LLaVA and its enhanced variant LLaVA-1.5. These models utilize advanced visual and textual processing capabilities to convert fixed UML diagrams into editable, machine-readable code, thus bridging a critical gap in software design workflows. The methodology includes fine-tuning MM-LLMs with synthetic UML datasets to enhance their accuracy in handling activity and sequence diagrams, focusing on maintaining syntactic fidelity and structural coherence. The evaluation metrics, including BLEU and SSIM, reveal that the enhanced LLaVA-1.5 model exhibits high accuracy and efficiency in generating UML code from visual inputs, outperforming baseline models and conventional tools. This work contributes to the field by proposing a scalable, automated solution that simplifies UML code generation and enhances development efficiency, particularly for legacy systems and long-term projects where the original design artifacts are often inaccessible.