1 - 10 of 2537
-
PublicationIncorporating Query Recommendation for Improving In-Car Conversational Search( 2024-03-23)Retrieval-augmented generation has become an effective mechanism for conversational systems in domain-specific settings. Retrieval of a wrong document due to the lack of context from the user utterance may lead to wrong answer generation. Such an issue may reduce the user engagement and thereby the system reliability. In this paper, we propose a context-guided follow-up question recommendation to internally improve the document retrieval in an iterative approach for developing an in-car conversational system. Specifically, a user utterance is first reformulated, given the context of the conversation to facilitate improved understanding to the retriever. In the cases, where the documents retrieved by the retriever are not relevant enough for answering the user utterance, we employ a large language model (LLM) to generate question recommendation which is then utilized to perform a refined retrieval. An empirical evaluation confirms the effectiveness of our proposed approaches in in-car conversations, achieving 48% and 22% improvement in the retrieval and system generated responses, respectively, against baseline approaches.
-
PublicationDeep Dynamic Language Models( 2024-03-15)This thesis investigates the domain of deep dynamic language models, focusing on the integration of temporal dynamics to enhance language modeling and its application in various tasks, such as text generation, recommendation systems, and predicting post popularity. Temporal content change, i.e., trends and themes that change with time featured in document collections such as academic journals, news articles and social media, make the traditional static language models (LMs) not an optimal solution. In order to address this limitation, several approaches to develop dynamic LMs are proposed and explored in this thesis. Initially, the impact of incorporating temporal information is explored, specifically in the context of modeling online communities. For the analysis of temporal content change in Yelp - a crowd-sourced review platform - an instantaneous language model is proposed. This model combines a temporal point process (TPP) for modeling review creation times and a LM to capture textual aspects. Empirical evaluations demonstrate that this model significantly improves the performance of LMs in terms of both language modeling and prediction of review creation time. Building upon the success of the instantaneous LM, the research in this thesis is extended to more application oriented task, such as recommender systems. Recognizing that user preferences and item reviews change over time, the proposed model here leverages users’ reviews to enhance rating predictions. By developing time-interval aware representations, the proposed model outperforms several state-of-the-art recommender systems models in real-world datasets. Additionally, the integration of dynamic topic models into LMs is explored. First, the problem of skewed topic distributions in topic modeling is addressed, which can cause models to learn more general topics present in the majority of documents, rather than rare topics present in only a few documents. A neural dynamic focused topic model is proposed as a solution, which decouples topic activities from topic proportions in documents using sequences of Bernoulli random variables. Experimental evaluations show that this model outperforms state-of-the-art topic models in generalization tasks, while employing a comparable number of parameters and converging two times faster. Furthermore, the performance of large pre-trained language models (LPLMs) in dynamic environments is explored. The empirical analysis on Reddit datasets reveals significant performance drops when predicting the popularity of future posts due to temporal distribution shifts in data. To mitigate this issue, a model is proposed that combines neural variational dynamic topic models and attention mechanisms to infer temporal LM representations. The proposed model exhibit improved performance while utilizing only a fraction of the parameters of LPLMs, and providing interpretable representations that offer insights into real-world events. In summary, this thesis emphasizes the significance of incorporating temporal dynamics into LMs and explores their application in various tasks.
-
PublicationWie Agenten und Foundation-Modelle bei der Versorgung Schwerverletzter helfen( 2024-03)
-
PublicationSuperkraft Sprachmodell?( 2024-03)Die deutsche Verwaltungslandschaft steht angesichts der notwendigen Digitalisierung und Automatisierung von bisher manuellen Prozessen vor einer grundlegenden Transformation. Der Anstieg an Anträgen für Wohngeld, BAföG oder Einbürgerungsverfahren setzt Behörden zusätzlich unter Druck. Der entstehende Rückstau trägt zu einem sinkenden Vertrauen in die Leistungsfähigkeit der öffentlichen Verwaltung bei. Gleichzeitig muss sie die rückläufigen Mitarbeitendenzahlen infolge des demografischen Wandels kompensieren. Generative Künstliche Intelligenz (GenAI) und insbesondere große Sprachmodelle (Large Language Models, LLMs) spielen hier eine wichtige Rolle, um die Mitarbeitenden zukünftig in ihren Aufgaben zu unterstützen, zu entlasten und hierdurch Freiräume zu schaffen, um sich verstärkt der direkten Interaktion mit Bürgerinnen und Bürgern zu widmen. In diesem Briefing präsentieren Fraunhofer IAIS und Deloitte drei Anwendungsbeispiele großer Sprachmodelle, von welchen die öffentliche Verwaltung schon heute profitieren kann. Bei der Betrachtung zu etablierender Rahmenbedingungen muss zwischen den behördeninternen Voraussetzungen und der staatlichen Infrastruktur unterschieden werden. Diese Publikation betrachtet die Voraussetzungen auf individueller Ebene der Behörden.
-
PublicationVertrauenswürdige KI-Anwendungen mit Foundation-Modellen entwickeln( 2024-01-22)
-
PublicationOn the effects of biased quantum random numbers on the initialization of artificial neural networks( 2024-01-16)Recent advances in practical quantum computing have led to a variety of cloud-based quantum computing platforms that allow researchers to evaluate their algorithms on noisy intermediate-scale quantum devices. A common property of quantum computers is that they can exhibit instances of true randomness as opposed to pseudo-randomness obtained from classical systems. Investigating the effects of such true quantum randomness in the context of machine learning is appealing, and recent results vaguely suggest that benefits can indeed be achieved from the use of quantum random numbers. To shed some more light on this topic, we empirically study the effects of hardware-biased quantum random numbers on the initialization of artificial neural network weights in numerical experiments. We find no statistically significant difference in comparison with unbiased quantum random numbers as well as biased and unbiased random numbers from a classical pseudo-random number generator. The quantum random numbers for our experiments are obtained from real quantum hardware.
-
-
PublicationShort-term predictor for COVID-19 severity from a longitudinal multi-omics study for practical application in intensive care units( 2024)
-
PublicationEinsatz von KI-basierten Anwendungen durch Krankenhauspersonal: Aufgabenprofile und Qualifizierungsbedarfe( 2024)
-
PublicationDeep reinforcement learning in service of air traffic controllers to resolve tactical conflicts( 2024)Dense and complex air traffic requires higher levels of automation than those exhibited by tactical conflict detection and resolution (CD&R) tools that air traffic controllers (ATCOs) use today: AI tools can act on their own initiative, increasing the capacity of ATCOs to control higher volumes of traffic. However, given that the air traffic control (ATC) domain is safety critical, requires AI systems to which ATCOs are comfortable to relinquishing control, guaranteeing operational integrity and automation adoption. Two major factors towards this goal are quality of solutions and operational transparency. ResoLver, the system that this article presents, addresses these challenges using an enhanced graph convolutional reinforcement learning method operating in a multiagent setting where each agent – representing a flight – performs a CD&R task, jointly with other agents. We show that ResoLver can provide high-quality solutions with respect to stakeholders interests (air traffic controllers and airspace users), addressing also operational transparency issues, which have been validated by ATCOs in simulated real-world settings.
