Teaching

University of Twente, in collaboration with ING Analytics

LLM-based retrieval-augmented systems are increasingly used at ING to extract information from corporate and financial documents, but stakeholders need to know not just the answer, also how likely it is to be correct. This project investigates whether the evidence and citations returned by RAG systems can be leveraged to estimate the probability that an answer is correct, moving beyond model-internal confidence signals.

RQ: How can evidence provided by retrieval-augmented language models be leveraged to estimate the probability that an answer is correct?

• Solid Python programming skills
• Experience with data analysis (Pandas, NumPy, Scikit-learn)
• Background in Machine Learning, Statistics, Data Mining, or AI

University of Twente, in collaboration with ING Analytics

Conventional RAG pipelines can be sensitive to repeated runs of the same question or to small, meaning-preserving rephrasings, which undermines traceability and auditability in applications such as Know Your Customer (KYC). This project investigates how to measure and improve the stability of a RAG pipeline—at the retrieval, evidence, and answer level—under repeated and semantically equivalent queries.

RQ: How can the stability of retrieval-augmented generation systems be measured and improved under repeated and semantically equivalent queries?

• Solid Python programming skills
• Experience with data analysis and experimentation
• Interest in LLMs, RAG systems, and trustworthy AI