What can AI systems do? Answering this question requires us to model their capabilities, but this first demands a clear conception of what capabilities are and which tools we can use to measure them. I advance a dispositional account of capabilities, understanding them as a system’s propensity to behave in certain ways under certain conditions. I then survey the tools we have at our disposal to measure capabilities, and what the nascent field of AI Evaluation can learn from the broader cognitive sciences.

Science of Intelligence (SCIoI) is a Cluster of Excellence in cooperation with and part of the Berlin University Alliance.

The event is addressed to Researchers, Scientists, PhD Candidates, Master Students and Professors.

Further information: https://www.scienceofintelligence.de/event/konstantinos-voudouris-helmholtz-ai-university-of-cambridge-capability-measurement-in-llms/