Open science in the era of AI

MVA course

Miguel Colom

About this course

Modern artificial intelligence (AI) has absolutely changed many aspects in our lives, including also the way we perform scientific research. We have quickly changed the focus from classic and explainable methods to large models that exhibit a larger performance (generation of images, language models, reasoning model, pattern recognition, ...), but come with several problems that need to be taken into account. These include the difficulty of understanding the methods themselves, the interpretability of their results, the availability of data, or the need of a very large amount of resources to train the models (in terms of energy, water consumption, GPUs).

But there's no way back now, as now large AI models have been consistently deployed, and we therefore need to analyse their impact in scientific research from a scientific and critical point of view. What's the place of Open Science and Reproducible Research in this new era?

In this course we'll work in groups to study an existing scientific article or project that describes an AI-related method, and we'll discuss which are the challenges in terms of open science and reproducibility that the method presents. The students will propose solutions to these problems and propose an alternative method that is aligned with the principles of open science and reproducibility. This might include re-training the method with a smaller dataset obtained from finetuning or transfer learning, avoiding the use of codes and data which are not free, or documenting properly all missing pre/post processing steps, among others.

This course can be considered the practical follow up of the MVA course Fundamentals of reproducible research and free software, although both courses are independent. The students are free to pick any project or article if an open and reproducible version has not already been published. Regularly all groups will make a presentation of their work, to report on their progress, explain the difficulties they are facing, and to get feedback from the responsible of the course and all the students.

Evaluation

Each group must deliver for evaluation a reproducible article and a report in the form of a pre-print. This includes the article, the source code, any associated data, and optionally an online demo.

Material

↩ Back to the main page