Bringing Systems Engineering Models to Large Language Models: An Integration of OPM with an LLM for Design Assistants
R. M. García Alarcia, P. Russo, A. Renga and A. Golkar
Proceedings of the 12th International Conference on Model-Based Software and Systems Engineering (MODELSWARD 2024), MBSE-AI Integration, pp. 334–345, SciTePress, 2024
Abstract
Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.
Keywords: Large Language Models, Systems Engineering, Model-Based Systems Engineering, Object-Process Methodology, Engineering Design, Design Assistant
Cite
@inproceedings{GarciaAlarcia2024BringingSystems,
author = {Garc{\'\i}a Alarcia, Ram{\'o}n Mar{\'\i}a and Russo, Pietro and Renga, Alfredo and Golkar, Alessandro},
title = {Bringing Systems Engineering Models to Large Language Models: An Integration of {OPM} with an {LLM} for Design Assistants},
booktitle = {Proceedings of the 12th International Conference on Model-Based Software and Systems Engineering (MODELSWARD 2024), MBSE-AI Integration},
pages = {334--345},
publisher = {SciTePress},
month = {feb},
year = {2024},
doi = {10.5220/0012621900003645},
url = {https://mediatum.ub.tum.de/1737156},
abstract = {Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.},
keywords = {Large Language Models, Systems Engineering, Model-Based Systems Engineering, Object-Process Methodology, Engineering Design, Design Assistant},
file = {2024/2024_GarciaAlarcia_BringingSystemsEngineeringModelsToLLMs.pdf}
}