AI to Touch: when art invites touch

It all began about four years ago, during a conversation with the organisers of an exhibition at Palazzo delle Paure in Lecco. Every year, from December to March, the museum hosts an exhibition, and at one point we asked ourselves a very simple question: how can we involve blind people as well? How can we convey a painting—something entirely two-dimensional, rich in nuances and colours—to someone who cannot see it?

At the time, we did not have an answer, but we began experimenting. The first software tools only allowed us to read the colours in an image: light tones became a positive bas-relief, while dark tones sank into a negative one. From there the first tactile tablets were born: small surfaces on which the painting—while still flattened—finally acquired a geography to be explored with the fingers. Together, we tested many materials and techniques (CNC milling, filament 3D printing, resin printing, laser printing), because the same idea tells different stories on different surfaces.

Exactly. The first time, we brought about fifteen tablets to the exhibition. The guide was Sara, a blind young woman who at the time attended high school in Oggiono. She explained each painting to visitors while touching the tablet together with them, with an incredible level of detail.

To accompany the exploration, we applied the “DescriVedendo” method: the artwork is divided into six sectors—like the keys of a telephone (three on top and three below)—and what is present in each sector is described. It is a way of giving structure to the discovery. The tablet is extremely rich in details and, without a “compass”, it is difficult to understand what one is touching. The guide—or a recorded audio description—is what truly allows the imagination to take flight.

It changed quite a lot. When we managed to apply some generative AI algorithms, we made a huge leap forward: from the image of a self-portrait, the algorithm reconstructs a complete 3D model—whether it is a Van Gogh, a Picasso or a Caravaggio. It recognises the face, the hair, the beard, even what the subject is wearing, and it reproduces what is not visible in the image, such as the back of the head or the nape of the neck. It is as if the machine “imagines” the missing volume.

And it does so very quickly: from image to 3D model in about two minutes.

Then comes the more “hands-on” phase, which remains the most fascinating. The digital model is prepared for printing through another software program that generates a code of around ten thousand lines melting temperatures, speed settings, material parameters: everything must be calibrated so that the quality meets expectations.

We then start the printing process in the laboratory. For a prototype about twenty centimetres tall, it takes between ten and twelve hours. The printer deposits material layer by layer, from the bottom up, until the figure gradually emerges.

We mainly work with PLA, a biocompatible material derived from corn. It is a choice that combines print quality with environmental awareness.

On 19 December, we officially delivered around forty prototypes to the Italian Union of the Blind and Partially Sighted (UIC): artists, scientists and athletes. For many of the people present, it was the first time they could “see” the faces of historical figures with their hands at such a level of detail. Their emotion was tangible.

From that moment on, the collaboration with UIC became permanent. We work closely with the Lombardy regional president, Silvano Stefanoni, and with the president of UIC Lecco. They are the ones who “test” the sculptures with us, including their dimensions, because for someone exploring with their hands, size is just as important as fidelity to detail.

They are extremely happy and tell us so. In fact, the Triulza Foundation recently discovered the project on LinkedIn and selected it as an innovative best practice for inclusive tourism. Their team sent a professional film crew to document the experiences directly with blind people interacting with the prototypes. Seeing the work recognised by the very people for whom it was designed is the greatest satisfaction.

Yes, because we wanted the project to speak to everyone, not only to blind people.

By scanning a QR code or a reference image with a smartphone, the app recognises the character and displays it in 3D with all its textures and colours, while also starting the audio description. Users can rotate the model, zoom in and explore its details from every angle.

The audio description is one of the most beautiful parts of the project. It was recorded by our students, and in some cases also by middle school students from Collegio Volta, who learned how to use 3D printers and then lent their voices to the various characters. They may not be professional narrations, but that is not a flaw: you can feel the emotion of the speaker, and that is exactly what we want to convey.

Many, and from very different backgrounds. I involved students from the bachelor’s degree programmes in Industrial Production Engineering, Communication for Interaction Design and Industrial Product Design, as well as master’s students in Mechanical Engineering and Architecture and Building Engineering.

These are extracurricular activities: nobody is required to participate. But when you explain what their work will be used for, everyone is eager to contribute. In my opinion, that is one of the most beautiful aspects: seeing how motivation changes when a project has a strong human purpose.

In addition, there are local companies that support us financially, along with researchers and cultural mediators with whom we collaborate in museums.

We presented it at the Museum of the Future, in an extraordinary architectural setting. The characters travelled with us, and the response was wonderful. Projects like this are appreciated because they show a side of technology that is rarely seen: the side that addresses vulnerability, accessibility and cultural experiences for those who are normally excluded from them. A blind person touching the face of someone who lived centuries ago and listening to a description of their clothes and colours suddenly opens up an entire world.

The Triulza Foundation also chose to promote the project as an innovative best practice for inclusive tourism and sent a film crew to document it.

Several directions. The first is multicolour and multi-texture printing. Today, the prototypes are monochrome and made from a single material, but in our laboratories we are working with three-resin printers that make it possible to digitally “blend” materials and, for example, make hair softer than the rest of the face. It is an expensive direction, but a very promising one.

The second is a fully virtual exhibition, experienced through a headset. Once inside, users are “teleported” into a digital museum where they can grasp the sculptures with their hands (the headset tracks movements in real time), rotate them, enlarge them, shrink them and activate the audio description with a gesture. It is an experience designed also for partially sighted people, who in that context can access a view of cultural heritage that would otherwise be difficult to achieve.

That technology, when it works together with people, is capable of very beautiful things. Generative AI, 3D printing, augmented reality and virtual reality: none of these, on their own, would have brought us to where we are today. We brought them together for a specific purpose: making art and culture accessible to those who risk being left out.

We involved engineers, designers, architects, university students and middle school students, museum mediators, associations and local companies.

The message I like most is this: projects like this do not speak only to blind people — they speak to everyone. Because touching a Caravaggio, listening to the emotional voice of a student describing Leonardo da Vinci, or rotating Botticelli in the palm of your hand on a smartphone screen is an experience that also changes those who can see perfectly well.

And that is exactly what we would like to say every time we talk about technology: it is not a neutral tool. It always depends, and only depends, on what we choose to do with it.