Now showing 1 - 10 of 88
  • Publication
    Implementation and evaluation of an additional GPT-4-based reviewer in PRISMA-based medical systematic literature reviews
    ( 2024-09-01) ; ;
    Mackay, Sina
    ;
    ; ; ;
    Höres, Timm
    ;
    Allende-Cid, Héctor
    Background: PRISMA-based literature reviews require meticulous scrutiny of extensive textual data by multiple reviewers, which is associated with considerable human effort. Objective: To evaluate feasibility and reliability of using GPT-4 API as a complementary reviewer in systematic literature reviews based on the PRISMA framework. Methodology: A systematic literature review on the role of natural language processing and Large Language Models (LLMs) in automatic patient-trial matching was conducted using human reviewers and an AI-based reviewer (GPT-4 API). A RAG methodology with LangChain integration was used to process full-text articles. Agreement levels between two human reviewers and GPT-4 API for abstract screening and between a single reviewer and GPT-4 API for full-text parameter extraction were evaluated. Results: An almost perfect GPT–human reviewer agreement in the abstract screening process (Cohen's kappa > 0.9) and a lower agreement in the full-text parameter extraction were observed. Conclusion: As GPT-4 has performed on a par with human reviewers in abstract screening, we conclude that GPT-4 has an exciting potential of being used as a main screening tool for systematic literature reviews, replacing at least one of the human reviewers.
  • Publication
    Addressing a new Paradigm Shift: An Empirical Study on Novel Project Characteristics for Foundation Model Projects
    In recent years, data science and machine learning (ML) has become common across sectors and industries. Project methodologies are aimed at supporting projects and try catching up with ML trends and paradigm shifts. However, they are hardly successful, since still 80% of data science projects never reach deployment. The latest paradigm shift in the area of ML - the trend of generative AI and foundation models - changes the nature of data science projects and is not yet addressed by existing project methodologies. In this work, we present novel requirements that arise from real-world projects incorporating foundation models based on 29 case studies from the NLU domain. Furthermore, we assess existing data science methodologies and identify their shortcomings. Finally, we provide guidance on adapting projects to address the new challenges in the development and operation of foundation model based solutions.
  • Publication
    POS0881 Specific AI-Generated Pattern of Tender Joints and Tenderness at Enthesial Sites are Predictive for Objective Detection of Musculoskeletal Inflammation in Psoriasis Patients
    ( 2023-05-30)
    Köhm, Michaela
    ;
    ; ;
    Mackay, Sina
    ;
    ;
    Kratz, Hannah
    ;
    Zerweck, Lukas
    ;
    ;
    Behrens, Frank
    Psoriasis (Pso) is one of the most common chronic inflammatory skin diseases in Europe. Psoriatic arthritis (PsA) is closely associated to Pso. Up to 30% of the Pso patients will develop PsA during skin disease course. Defined and validated approaches for early detection are still missing. Beside biomarkers from blood or imaging, clinical characteristics of the patients may be of value to detect PsA patients in the transition state early. To perform an AI-based cluster analysis in a cohort of Pso patients at-risk for development of PsA to assess clinical characteristics as markers for early PsA.
  • Publication
    A machine learning method for the identification and characterization of novel COVID-19 drug targets
    ( 2023-05-03) ;
    Delong, Lauren Nicole
    ;
    Masny, Aliaksandr
    ;
    Lentzen, Manuel
    ;
    ;
    Dijk, David van
    ;
    ;
    Hansen, Anne Funck
    ;
    ; ; ; ; ; ;
    Kannt, Aimo
    ;
    Foldenauer, Ann Christina
    ;
    ;
    Resch, Eduard
    ;
    Frank, Kevin
    ;
    ; ; ;
    Laue, Hendrik
    ;
    ;
    Hirsch, Jochen
    ;
    Wischnewski, Marco
    ;
    ; ;
    Tom Kodamullil, Alpha
    ;
    Gemünd, Andre
    ;
    Fluck, Juliane
    ;
    Steinborn, Carina
    ;
    ; ;
    Hermanowski, Helena
    ;
    ;
    Klein, Jürgen
    ;
    ; ; ;
    Knieps, Meike
    ;
    ;
    Wendland, Philipp Johannes
    ;
    Wegner, Philipp
    ;
    ; ; ;
    Lentzen, Manuel
    ;
    In addition to vaccines, the World Health Organization sees novel medications as an urgent matter to fight the ongoing COVID-19 pandemic. One possible strategy is to identify target proteins, for which a perturbation by an existing compound is likely to benefit COVID-19 patients. In order to contribute to this effort, we present GuiltyTargets-COVID-19 (https://guiltytargets-covid.eu/), a machine learning supported web tool to identify novel candidate drug targets. Using six bulk and three single cell RNA-Seq datasets, together with a lung tissue specific protein-protein interaction network, we demonstrate that GuiltyTargets-COVID-19 is capable of (i) prioritizing meaningful target candidates and assessing their druggability, (ii) unraveling their linkage to known disease mechanisms, (iii) mapping ligands from the ChEMBL database to the identified targets, and (iv) pointing out potential side effects in the case that the mapped ligands correspond to approved drugs. Our example analyses identified 4 potential drug targets from the datasets: AKT3 from both the bulk and single cell RNA-Seq data as well as AKT2, MLKL, and MAPK11 in the single cell experiments. Altogether, we believe that our web tool will facilitate future target identification and drug development for COVID-19, notably in a cell type and tissue specific manner.
  • Publication
    Natural Language Processing in der Medizin. Whitepaper
    Künstliche Intelligenz (KI) ist in der Medizin angekommen und bereits jetzt schon unverzichtbar. Zusammen mit der Digitalisierung beschleunigt KI die Verbreitung einer datengetriebenen und personalisierten Behandlung von Patient*innen. Gerade im Krankenhaus kann KI dabei helfen, Mitarbeitende zu unterstützen, Behandlungsergebnisse zu verbessern und Kosten einzusparen. KI-Anwendungen sind mittlerweile dazu fähig, radiologische Bildgebungen auszuwerten, herapieentscheidungen zu unterstützen und Sprachdiktate zu transkribieren. Im Besonderen wurde die Textverarbeitung durch Algorithmen des Natural Language Processing (NLP) revolutioniert, die auf einer KI basieren, die sich mit natürlicher Sprache beschäftigt. Gemeint ist damit das Lesen, Verstehen und Schreiben von Texten wie beispielsweise medizinischer Befunde, Dokumentationen oder Leitlinien.
  • Publication
    Symptom diaries as a digital tool to detect SARS-CoV-2 infections and differentiate between prevalent variants
    ( 2022-11-14)
    Grüne, Barbara
    ;
    ; ;
    Wolff, Anna
    ;
    Buess, Michael
    ;
    Kossow, Annelene
    ;
    Küfer-Weiß, Annika
    ;
    ;
    Neuhann, Florian
    The COVID-19 pandemic and the high numbers of infected individuals pose major challenges for public health departments. To overcome these challenges, the health department in Cologne has developed a software called DiKoMa. This software offers the possibility to track contact and index persons, but also provides a digital symptom diary. In this work, the question of whether these can also be used for diagnostic purposes will be investigated. Machine learning makes it possible to identify infections based on early symptom profiles and to distinguish between the predominant dominant variants. Focusing on the occurrence of the symptoms in the first week, a decision tree is trained for the differentiation between contact and index persons and the prevailing dominant variants (Wildtype, Alpha, Delta, and Omicron). The model is evaluated, using sex- and age-stratified cross-validation and validated by symptom profiles of the first 6 days. The variants achieve an AUC-ROC from 0.89 for Omicron and 0.6 for Alpha. No significant differences are observed for the results of the validation set (Alpha 0.63 and Omicron 0.87). The evaluation of symptom combinations using artificial intelligence can determine the individual risk for the presence of a COVID-19 infection, allows assignment to virus variants, and can contribute to the management of epidemics and pandemics on a national and international level. It can help to reduce the number of specific tests in times of low labor capacity and could help to early identify new virus variants.
  • Publication
    Omics and Multi-Omics Analysis for the Early Identification and Improved Outcome of Patients with Psoriatic Arthritis
    ( 2022-09-24)
    Gurke, Robert
    ;
    Bendes, Annika
    ;
    Bowes, John
    ;
    Köhm, Michaela
    ;
    Twyman, Richard M.
    ;
    Barton, Anne
    ;
    Elewaut, Dirk
    ;
    Goodyear, Phil
    ;
    Hahnefeld, Lisa
    ;
    Hillenbrand, Rainer
    ;
    Hunter, Ewan
    ;
    Ibberson, Mark
    ;
    Ioannidis, Vassilios
    ;
    ;
    Lories, Rik J.
    ;
    Resch, Eduard
    ;
    ;
    Scholich, Klaus
    ;
    Schwenk, Jochen M.
    ;
    Waddington, James C.
    ;
    Whitfield, Phil
    ;
    ;
    FitzGerald, Oliver
    ;
    Behrens, Frank
    ;
    Pennington, Stephen R.
    The definitive diagnosis and early treatment of many immune-mediated inflammatory diseases (IMIDs) is hindered by variable and overlapping clinical manifestations. Psoriatic arthritis (PsA), which develops in ~30% of people with psoriasis, is a key example. This mixed-pattern IMID is apparent in entheseal and synovial musculoskeletal structures, but a definitive diagnosis often can only be made by clinical experts or when an extensive progressive disease state is apparent. As with other IMIDs, the detection of multimodal molecular biomarkers offers some hope for the early diagnosis of PsA and the initiation of effective management and treatment strategies. However, specific biomarkers are not yet available for PsA. The assessment of new markers by genomic and epigenomic profiling, or the analysis of blood and synovial fluid/tissue samples using proteomics, metabolomics and lipidomics, provides hope that complex molecular biomarker profiles could be developed to diagnose PsA. Importantly, the integration of these markers with high-throughput histology, imaging and standardized clinical assessment data provides an important opportunity to develop molecular profiles that could improve the diagnosis of PsA, predict its occurrence in cohorts of individuals with psoriasis, differentiate PsA from other IMIDs, and improve therapeutic responses. In this review, we consider the technologies that are currently deployed in the EU IMI2 project HIPPOCRATES to define biomarker profiles specific for PsA and discuss the advantages of combining multi-omics data to improve the outcome of PsA patients.
  • Publication
    Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
    ( 2022-06-18)
    Houben, Sebastian
    ;
    Albrecht, Stefanie
    ;
    ;
    Bär, Andreas
    ;
    Brockherde, Felix
    ;
    Feifel, Patrick
    ;
    Fingscheidt, Tim
    ;
    ;
    Ghobadi, Seyed Eghbal
    ;
    Hammam, Ahmed
    ;
    Haselhoff, Anselm
    ;
    Hauser, Felix
    ;
    Heinzemann, Christian
    ;
    Hoffmann, Marco
    ;
    Kapoor, Nikhil
    ;
    Kappel, Falk
    ;
    Klingner, Marvin
    ;
    Kronenberger, Jan
    ;
    Küppers, Fabian
    ;
    Löhdefink, Jonas
    ;
    Mlynarski, Michael
    ;
    ;
    Mualla, Firas
    ;
    Pavlitskaya, Svetlana
    ;
    ;
    Pohl, Alexander
    ;
    Ravi-Kumar, Varun
    ;
    ;
    Rottmann, Matthias
    ;
    ;
    Sämann, Timo
    ;
    Schneider, Jan David
    ;
    Schulz, Elena
    ;
    Schwalbe, Gesina
    ;
    Sicking, Joachim
    ;
    Srivastava, Toshika
    ;
    Varghese, Serin
    ;
    Weber, Michael
    ;
    Wirkert, Sebastian
    ;
    ;
    Woehrle, Matthias
    Deployment of modern data-driven machine learning methods, most often realized by deep neural networks (DNNs), in safety-critical applications such as health care, industrial plant control, or autonomous driving is highly challenging due to numerous model-inherent shortcomings. These shortcomings are diverse and range from a lack of generalization over insufficient interpretability and implausible predictions to directed attacks by means of malicious inputs. Cyber-physical systems employing DNNs are therefore likely to suffer from so-called safety concerns, properties that preclude their deployment as no argument or experimental setup can help to assess the remaining risk. In recent years, an abundance of state-of-the-art techniques aiming to address these safety concerns has emerged. This chapter provides a structured and broad overview of them. We first identify categories of insufficiencies to then describe research activities aiming at their detection, quantification, or mitigation. Our work addresses machine learning experts and safety engineers alike: The former ones might profit from the broad range of machine learning topics covered and discussions on limitations of recent methods. The latter ones might gain insights into the specifics of modern machine learning methods. We hope that this contribution fuels discussions on desiderata for machine learning systems and strategies on how to help to advance existing approaches accordingly.
  • Publication
    A Quantitative Human-Grounded Evaluation Process for Explainable Machine Learning
    ( 2022) ;
    Müller, Sebastian
    ;
    Methods from explainable machine learning are increasingly applied. However, evaluation of these methods is often anecdotal and not systematic. Prior work has identified properties of explanation quality and we argue that evaluation should be based on them. In this position paper, we provide an evaluation process that follows the idea of property testing. The process acknowledges the central role of the human, yet argues for a quantitative approach for the evaluation. We find that properties can be divided into two groups, one to ensure trustworthiness, the other to assess comprehensibility. Options for quantitative property tests are discussed. Future research should focus on the standardization of testing procedures.
  • Publication
    Die zeitlich-räumliche Verteilung von COVID-19 in Köln und beeinflussende soziale Faktoren im Zeitraum Februar 2020 bis Oktober 2021
    ( 2022)
    Neuhann, F.
    ;
    ;
    Buess, M.
    ;
    Wolff, A.
    ;
    ;
    Schlanstedt, G.
    ;
    Kossow, A.
    ;
    Nießen, J.
    ;
    Hintergrund und Ziele: Schon in der frühen Phase der global sehr verschieden verlaufenden COVID-19-Pandemie zeigten sich Hinweise auf den Einfluss sozioökonomischer Faktoren auf die Ausbreitungsdynamik der Erkrankung, die vor allem ab der zweiten Phase (September 2020) Menschen mit geringerem sozioökonomischen Status stärker betraf. Solche Effekte können sich auch innerhalb einer Großstadt zeigen. Die vorliegende Studie visualisiert und untersucht die zeitlich-räumliche Verbreitung aller in Köln gemeldeten COVID-19-Fälle (Februar 2020–Oktober 2021) auf Stadtteilebene und deren mögliche Assoziation mit sozioökonomischen Faktoren. Methoden: Pseudonymisierte Daten aller in Köln gemeldeten COVID-19-Fälle wurden geocodiert, deren Verteilung altersstandardisiert auf Stadtteilebene über 4 Zeiträume kartiert und mit der Verteilung von sozialen Faktoren verglichen. Der mögliche Einfluss der ausgewählten Faktoren wird zudem in einer Regressionsanalyse in einem Modell mit Fallzuwachsraten betrachtet. Ergebnisse: Das kleinräumige lokale Infektionsgeschehen ändert sich im Pandemieverlauf. Stadtteile mit schwächeren sozioökonomischen Indizes weisen über einen großen Teil des pandemischen Verlaufs höhere Inzidenzzahlen auf, wobei eine positive Korrelation zwischen den Armutsrisikofaktoren und der altersstandardisierten Inzidenz besteht. Die Stärke dieser Korrelation ändert sich im zeitlichen Verlauf. Schlussfolgerung: Die zeitnahe Beobachtung und Analyse der lokalen Ausbreitungsdynamik lassen auch auf der Ebene einer Großstadt die positive Korrelation von nachteiligen sozioökonomischen Faktoren auf die Inzidenzrate von COVID-19 erkennen und können dazu beitragen, lokale Eindämmungsmaßnahmen zielgerecht zu steuern.