ATENEA

Artificial Intelligence Applied to the Characterization and Comparison of Fabrics of Canvases in Art Masterpieces

ABSTRACT

This project addresses the application of signal processing and artificial intelligence to the analysis of x-rays of paintings from the 16th-18th centuries. The objective is to develop methods and algorithms that are useful when it comes to simplifying and automating the work of painting conservators of this period, so that they are reliable, repeatable and can be applied in a standardized way. The proposal focuses, on the one hand, on the analysis of a work to extract descriptors that characterize it. Although the analysis of simple fabrics such as taffetas has been extensively studied, in certain works, such as a significant part of the Velázquez collection, current methods fail. Here it is proposed as a novelty to analyze these fabrics using deep learning. In addition, more complex twill fabrics by painters such as Murillo or El Greco have rarely been studied, and they will be characterized within the framework of this project. On the other hand, the main objective of this proposal is to be able to compare the degree of similarity between canvases, which allows us to conclude if a set of works were painted by the same author, in the same period or in the same place. The analysis of requirements is carried out in collaboration with the Museo Nacional del Prado, as well as the evaluation of the results.

GOALS

• Study complex twill fabrics to help curators in the analysis of these fabrics
• Improve Aracne, a thread counting algorithm based on frequency analysis, to make it freely available.
• Apply deep learning to improve the frequency analysis, in particular to analyze the collection by Velázquez at El Museo Nacional del Prado.
• Explore new ways of finding matches between fabrics of canvases not based on the correspondence of the thread counting maps.

DEVELOPMENT

Over the years, we found that frequency analysis failed in a wide variety of situations. In fact, while studying the canvases used in the works attributed to Velázquez at the Museo del Prado, we discovered that Aracne’s results were unsatisfactory. The problems appear when patterns are not uniform because the space between threads varies significantly within a centimetre, or because the threads in one direction are so taut that it is hard to observe the threads in the other. The following figure contains examples of these two scenarios.

To address this issue, we developed tools based on artificial intelligence. In the frequency domain, with Aracne, we seek maximum values in the Fourier transform. In this case, we work in the spatial domain, focusing on the image itself. First, we developed a method (Bejarano, J. Murillo-Fuentes and L. Alba-Carcelén 2022a; Bejarano, J. Murillo-Fuentes and L. Alba-Carcelén 2022b; Delgado, Laura Alba-Carcelén and J. J. Murillo-Fuentes 2023) to segment intersections and later estimate the space between them. The figure below shows how, after preprocessing, a deep learning model is used to detect the intersection points. Finally, a simple approach is used to translate the distance between intersections into vertical densities.

In a second approach, we estimated thread count directly from the image. After preprocessing, we applied a deep-learning regression model to obtain the vertical thread density (Delgado, J. Murillo-Fuentes and L. Alba-Carcelén 2025,Thread Counting in Plain Weave for Old Paintings Using Regression Deep Learning Models).

The Atenea project has proved quite useful for analysing canvases with the aforementioned characteristics. Below are the radiographs of two canvases: An Artillery General, formerly attributed to Francisco Rizi, on the left; and on the right, Saint Peter as Pope, painted by Francisco Herrera the Younger.

The next figure contains the vertical thread count maps for both works rotated 90º: above, the results with frequency analysis (FT), and below, the results with deep-learning regression (Atenea project). The distribution of threads across the entire canvas is easier to observe in the results obtained with Atenea. The fact that the thread count maps of both canvases matched was one of the reasons why their authorship was revisited and both were finally attributed to Francisco Herrera the Younger.

FUNDING AND COLLABORATORS

This project was funded by Consejería de Economía y Conocimiento, Junta de Andaluc ía and European Union in the framework of the Program FEDER Andalucía 2014-2010 “Crecimiento inteligente: una economía basada en el conocimiento y la innovación” for funding this research under the ATENEA Project P20-01216.

El Museo Nacional del Prado has been a relevant partner of this project, providing the needed material to perform the investigation. We also thank The National Gallery and especially Catherine Higgitt for providing the canvases by Poussin to allow for comparison with SOTA approaches.

PUBLICATIONS