1 - 10 of 2543
-
PublicationDeveloping trustworthy AI applications with foundation models( 2024-04)
-
PublicationMachine Learning Operations (MLOps): Grundlagen, Chancen und Herausforderungen beim MLOps-Einsatz in Unternehmen( 2024-04)
-
PublicationKreativität der generativen KI( 2024-03-28)In diesem Beitrag wird die Frage diskutiert, ob auch Systeme der generativen KI kreative Inhalte erzeugen können. Es wird zunächst beschrieben, wie solche Systeme intern funktionieren und wie sie potenziell neue Inhalte generieren können. Anschließend wird der kreative Prozess diskutiert und es wird überprüft, ob KI-Systeme kreative Leistungen für die unterschiedlichen Medien Text, Bild und Musik erbringen können. In standardisierten Tests konnte gezeigt werden, dass das Sprachmodell GPT-4 inzwischen kreativere Antworten produziert als Menschen. Ähnliche Tests haben ergeben, dass Bilder, die mit einer älteren Version von DALL-E erstellt wurden, nur schwer von Künstlerbildern zu unterscheiden sind. Aufgrund der stark verbesserten Detailgenauigkeit neuerer Systeme ist davon auszugehen, dass diese heute eine verbesserte Kreativität besitzen. Systeme zur Generierung von Musik können derzeit dagegen noch nicht mit der Kreativität menschlicher Komponist*innen mithalten.
-
PublicationIncorporating Query Recommendation for Improving In-Car Conversational Search( 2024-03-23)Retrieval-augmented generation has become an effective mechanism for conversational systems in domain-specific settings. Retrieval of a wrong document due to the lack of context from the user utterance may lead to wrong answer generation. Such an issue may reduce the user engagement and thereby the system reliability. In this paper, we propose a context-guided follow-up question recommendation to internally improve the document retrieval in an iterative approach for developing an in-car conversational system. Specifically, a user utterance is first reformulated, given the context of the conversation to facilitate improved understanding to the retriever. In the cases, where the documents retrieved by the retriever are not relevant enough for answering the user utterance, we employ a large language model (LLM) to generate question recommendation which is then utilized to perform a refined retrieval. An empirical evaluation confirms the effectiveness of our proposed approaches in in-car conversations, achieving 48% and 22% improvement in the retrieval and system generated responses, respectively, against baseline approaches.
-
PublicationControlled Randomness Improves the Performance of Transformer Models( 2024-03-19)
-
PublicationDeep Dynamic Language Models( 2024-03-15)This thesis investigates the domain of deep dynamic language models, focusing on the integration of temporal dynamics to enhance language modeling and its application in various tasks, such as text generation, recommendation systems, and predicting post popularity. Temporal content change, i.e., trends and themes that change with time featured in document collections such as academic journals, news articles and social media, make the traditional static language models (LMs) not an optimal solution. In order to address this limitation, several approaches to develop dynamic LMs are proposed and explored in this thesis. Initially, the impact of incorporating temporal information is explored, specifically in the context of modeling online communities. For the analysis of temporal content change in Yelp - a crowd-sourced review platform - an instantaneous language model is proposed. This model combines a temporal point process (TPP) for modeling review creation times and a LM to capture textual aspects. Empirical evaluations demonstrate that this model significantly improves the performance of LMs in terms of both language modeling and prediction of review creation time. Building upon the success of the instantaneous LM, the research in this thesis is extended to more application oriented task, such as recommender systems. Recognizing that user preferences and item reviews change over time, the proposed model here leverages users’ reviews to enhance rating predictions. By developing time-interval aware representations, the proposed model outperforms several state-of-the-art recommender systems models in real-world datasets. Additionally, the integration of dynamic topic models into LMs is explored. First, the problem of skewed topic distributions in topic modeling is addressed, which can cause models to learn more general topics present in the majority of documents, rather than rare topics present in only a few documents. A neural dynamic focused topic model is proposed as a solution, which decouples topic activities from topic proportions in documents using sequences of Bernoulli random variables. Experimental evaluations show that this model outperforms state-of-the-art topic models in generalization tasks, while employing a comparable number of parameters and converging two times faster. Furthermore, the performance of large pre-trained language models (LPLMs) in dynamic environments is explored. The empirical analysis on Reddit datasets reveals significant performance drops when predicting the popularity of future posts due to temporal distribution shifts in data. To mitigate this issue, a model is proposed that combines neural variational dynamic topic models and attention mechanisms to infer temporal LM representations. The proposed model exhibit improved performance while utilizing only a fraction of the parameters of LPLMs, and providing interpretable representations that offer insights into real-world events. In summary, this thesis emphasizes the significance of incorporating temporal dynamics into LMs and explores their application in various tasks.
-
PublicationWie Agenten und Foundation-Modelle bei der Versorgung Schwerverletzter helfen( 2024-03)
-
PublicationSuperkraft Sprachmodell?( 2024-03)Die deutsche Verwaltungslandschaft steht angesichts der notwendigen Digitalisierung und Automatisierung von bisher manuellen Prozessen vor einer grundlegenden Transformation. Der Anstieg an Anträgen für Wohngeld, BAföG oder Einbürgerungsverfahren setzt Behörden zusätzlich unter Druck. Der entstehende Rückstau trägt zu einem sinkenden Vertrauen in die Leistungsfähigkeit der öffentlichen Verwaltung bei. Gleichzeitig muss sie die rückläufigen Mitarbeitendenzahlen infolge des demografischen Wandels kompensieren. Generative Künstliche Intelligenz (GenAI) und insbesondere große Sprachmodelle (Large Language Models, LLMs) spielen hier eine wichtige Rolle, um die Mitarbeitenden zukünftig in ihren Aufgaben zu unterstützen, zu entlasten und hierdurch Freiräume zu schaffen, um sich verstärkt der direkten Interaktion mit Bürgerinnen und Bürgern zu widmen. In diesem Briefing präsentieren Fraunhofer IAIS und Deloitte drei Anwendungsbeispiele großer Sprachmodelle, von welchen die öffentliche Verwaltung schon heute profitieren kann. Bei der Betrachtung zu etablierender Rahmenbedingungen muss zwischen den behördeninternen Voraussetzungen und der staatlichen Infrastruktur unterschieden werden. Diese Publikation betrachtet die Voraussetzungen auf individueller Ebene der Behörden.
-
PublicationVertrauenswürdige KI-Anwendungen mit Foundation-Modellen entwickeln( 2024-01-22)
-
PublicationOn the effects of biased quantum random numbers on the initialization of artificial neural networks( 2024-01-16)Recent advances in practical quantum computing have led to a variety of cloud-based quantum computing platforms that allow researchers to evaluate their algorithms on noisy intermediate-scale quantum devices. A common property of quantum computers is that they can exhibit instances of true randomness as opposed to pseudo-randomness obtained from classical systems. Investigating the effects of such true quantum randomness in the context of machine learning is appealing, and recent results vaguely suggest that benefits can indeed be achieved from the use of quantum random numbers. To shed some more light on this topic, we empirically study the effects of hardware-biased quantum random numbers on the initialization of artificial neural network weights in numerical experiments. We find no statistically significant difference in comparison with unbiased quantum random numbers as well as biased and unbiased random numbers from a classical pseudo-random number generator. The quantum random numbers for our experiments are obtained from real quantum hardware.
