Advances in Interpretable Language Models

In recent years, Natural Language Processing (NLP) has experienced a paradigm shift from fine-tuning pre-trained large language models (LLMs) on task-specific data to in-context learning, where task descriptions are embedded directly into the LLM input, allowing the same model to perform multiple tasks. While both approaches have demonstrated impressive performance across  various NLP tasks, their opaque nature poses challenges in understanding their inner workings and decision-making processes. In this talk, I will discuss the research my team has conducted to address interpretability concerns surrounding neural models for language understanding. This includes interpreting uncertainty in text classifiers built on LLMs, developing explainable methods for scoring student answers in science exams, and exploring monosemanticity in LLMs through feature decorrelation. I will conclude my talk by sharing perspectives on potential future directions for interpretable language understanding.

About the speaker:
Yulan He is a Professor in Natural Language Processing at the Department of Informatics in King’s College London. She is currently holding a prestigious 5-year UKRI Turing AI Fellowship. Yulan’s research interests lie in the integration of machine learning and natural language processing for text understanding. Recently, she has focused on addressing the limitations of LLMs, aiming to enhance their reasoning capabilities, robustness, and explainability. She has received several prizes and awards for her research, including a SWSA Ten-Year Award, a CIKM Test-of-Time Award, and AI 2020 Most Influential Scholar Honourable Mention.

---

Model-based Control in Construction Robotics  

Efficiency in the building sector is worldwide rather low. There are intensive research activities in various fields to increase this. These include the use of robotic machines, which are to be used in semi-automatic or fully automatic or even autonomous operation. Various projects in this field will be presented in the talk. An overhead crane with trajectory tracking control of the load will be presented for transporting large loads and supporting assembly tasks. A special load handling device at the crane hook enables orientation in six degrees of freedom and thus support for assembly tasks. In a second system, a hydraulic manipulator with 7 axes and a working area of around 15 meters was converted so that it can also support the operator in semi-automatic mode, as well as be operated in fully automatic mode to carry out assembly tasks. Other research activities of automated excavators, mobile concrete pumps, or the use of autonomous robot systems on wheels with robot arms to carry out work in existing buildings are discussed. For the systems presented, the approaches for deriving dynamic models, control, trajectory generation and path planning are presented and discussed, particularly with regard to cooperation between the machines.

About the speaker:
Professor Sawodny received his Dipl.-Ing. degree in electrical engineering from the University of Karlsruhe, Germany, in1991 and his Ph.D. degree from the University of Ulm,  Germany, in 1996. In 2002, he became a Full Professor at the Technical University of Ilmenau, Germany. Since 2005, he has been the Director of the Institute for System Dynamics, University of Stuttgart, Stuttgart, Germany. His current research interests include methods of differential geometry, trajectory generation, and applications to mechatronic systems. He received important paper awards in major control application journals such as IEEE Transaction on Control System Technology Outstanding Paper Award (2013). He is a senior member of IEEE and Co-EiC of IFAC Journal Mechatronics.

---

Solving the gravity-quantum dilemma in experiments

Modern physics is facing a philosophical dilemma: its two main pillars, quantum theory and the theory of gravity, are rooted in world views that mutually exclude each other. If quantum physics is correct, we need to radically rethink our notions of space and time. If gravity theory is correct, quantum physics requires a dramatic revision. This is an experimental problem. However, up to this date there is no evidence that gravity requires a quantum description. Quantum experiments with increasingly massive particles may change that situation – by directly probing how gravity reacts to a quantum object.

About the speaker:
Markus Aspelmeyer is Professor of Physics at the University of Vienna and Scientific Director at the Institute for Quantum Optics and Quantum Information (IQOQI) of the Austrian Academy of Sciences in Vienna. His research combines the development of new quantum technologies with fundamental quantum experiments. His current interest is focused on the intriguing puzzles around quantum physics and gravity.

---

Modeling national supply chains with data driven 1:1 agent based models – and why it is important

Supply chains are less robust than we generally believed in the past decades. How robust is the economy, how could we quantify that? For the first time we can now observe national supply chain networks on the firm-level, involving hundred thousands firms and millions of supply relations that constantly change over time. By assigning production functions to these firms and by estimating the re-linking dynamics of the economy, we are able to represent national economies as a 1:1 agent based model. On the basis of this model, we can rethink the big questions of economics and in particular make predictive statements about the robustness and resilience of the present economy. We discuss new ways to optimally transform production networks towards a green and—at the same time—socially acceptable economy.

About the speaker:
Stefan Thurner is a professor of Science of Complex Systems at the Medical University of Vienna. He is co-founder and president of the Complexity Science Hub since 2015. He is external professor at the Santa Fe Institute in New Mexico. He earned a PhD in theoretical physics from the TU Wien and a PhD in economics from the University of Vienna. He held PostDoc positions at the Humboldt University Berlin and Boston University. His habilitation is in theoretical physics. Stefan Thurner started his career with contributions to theoretical particle physics and gradually shifted his focus to complex systems understanding. He published more than 290 scientific articles ranging from fundamental physics (topological excitations in quantum field theories, statistics and entropy of complex systems), applied mathematics (wavelet statistics, fractal harmonic analysis, anomalous diffusion), network theory, evolutionary systems, life sciences (network medicine, gene regulatory networks, bioinformatics, heartbeat dynamics, cell motility), economics and finance (supply chain dynamics, price formation, regulation, systemic risk) and in social sciences (opinion formation, bureaucratic inefficiency, collective human behavior, efficiency of healthcare systems).His work has been covered extensively in more than 1,000 newspaper, radio and television reports in Austrian and international media such as the New York Times, BBC World, Nature, New Scientist and Physics World. In 2017, he was named Austrian Physicist of the Year and he received the Paul Watzlawick Ring of Honor in 2021.