Ferreira, JoséJoséFerreiraDarabi, RoyaRoyaDarabiSousa, ArmandoArmandoSousaBrückner, FrankFrankBrücknerReis, Luís PauloLuís PauloReisReis, AnaAnaReisTavares, João Manuel R.S.João Manuel R.S.TavaresMeireles De Sousa, Joao PauloJoao PauloMeireles De Sousa2025-07-152025-07-152025https://publica.fraunhofer.de/handle/publica/48952910.1007/s10845-025-02614-42-s2.0-105003559561This work introduces Gen-JEMA, a generative approach based on joint embedding with multimodal alignment (JEMA), to enhance feature extraction in the embedding space and improve the explainability of its predictions. Gen-JEMA addresses these challenges by leveraging multimodal data, including multi-view images and metadata such as process parameters, to learn transferable semantic representations. Gen-JEMA enables more explainable and enriched predictions by learning a decoder from the embedding. This novel co-learning framework, tailored for directed energy deposition (DED), integrates multiple data sources to learn a unified data representation and predict melt pool images from the primary sensor. The proposed approach enables real-time process monitoring using only the primary modality, simplifying hardware requirements and reducing computational overhead. The effectiveness of Gen-JEMA for DED process monitoring was evaluated, focusing on its generalization to downstream tasks such as melt pool geometry prediction and the generation of external melt pool representations using off-axis sensor data. To generate these external representations, autoencoder (AE) and variational autoencoder (VAE) architectures were optimized using Bayesian optimization. The AE outperformed other approaches achieving a 38% improvement in melt pool geometry prediction compared to the baseline and 88% in data generation compared with the VAE. The proposed framework establishes the foundation for integrating multisensor data with metadata through a generative approach, enabling various downstream tasks within the DED domain and achieving a small embedding, allowing efficient process control based on model predictions and embeddings.entrueArtificial intelligenceContrastive learningEmbedding representationMetal additive manufacturingTransference600 Technik, Medizin, angewandte Wissenschaften::620 Ingenieurwissenschaftenjournal article