Beyond true and false: recognising synthetic content in the age of AI

Stefano Tubaro: I studied at Politecnico di Milano and, after an initial period at the CNR, I spent my entire career here. I have always worked on the analysis and processing of audio and visual signals, initially for industrial applications and for image and video coding.

For about twenty years, we have also worked on reconstructing the history of multimedia content: whether it is original, what processing it has undergone and how it has been modified. Since metadata may have been deleted or altered, it is necessary to observe and analyse images, videos and audio tracks directly.

Over the past four or five years, the landscape has changed further. With the possibility of generating content from scratch based on text or examples, research has gradually shifted towards the recognition of synthetic content.

Paolo Bestagini: My path also developed entirely within Politecnico di Milano: a Bachelor's and a Master's degree in Telecommunications Engineering, a PhD in Information Technology, several years of postdoctoral research and then the various stages of an academic career, culminating in the role of associate professor.

I work on the same topics described by Stefano, with a particular focus on the software development and implementation of algorithms, which have become increasingly sophisticated over time.

Stefano Tubaro: We took part in two PNRR subprojects: FunMedia, part of RESTART, dedicated primarily to the recognition of synthetic audio; and FF4All, within SERICS, focused mainly on images and videos.

The results allow us to distinguish natural content from synthetic content with very high probabilities, even when the alteration affects only part of the file: for example, a few words in a recording or a portion of an image or video, modified to change its semantic meaning.

Paolo Bestagini: It is an increasingly complex question. In the first deepfakes, some inconsistencies were visible even to the naked eye, such as the absence of blinking or a lack of synchronization between lips and speech. Today, however, the results are much more realistic and these clues are no longer sufficient.

For this reason, researchers often try to respond to artificial intelligence with artificial intelligence, analysing large amounts of natural and synthetic content to identify imperceptible differences. 

In images and videos, for example, a generative system may leave particular noise patterns or, conversely, in a photograph taken with a camera we can look for the characteristic traces of the device, which are difficult to find in a synthetic image. In audio, frequencies and how they evolve over time are analysed instead.

In some studies, we have also addressed a more specific problem: cloned voices. 
In these cases, we built a kind of profile of the person, collecting the phonemes pronounced in their various speeches. The track to be analysed was then broken down and compared with this set of references, looking for deviations that might indicate artificial synthesis.

Stefano Tubaro: At first, the focus was primarily on semantic inconsistencies, such as a voice sounding too metallic or a shadow inconsistent with the light. Today, since models make fewer obvious errors, we focus on infinitesimal traces invisible to the human eye: small noise patterns, similar to those that emerge in a photograph taken in low light, which may be characteristic of the generative system used.

Stefano Tubaro: We proceed in stages: first we verify whether the content is natural, manipulated or synthetic; then we try to understand which technique or model was used to produce it. In the past, obvious errors could be enough: in a video of an elephant walking in Piazza Duomo, for example, it had five legs in some frames, an immediate sign that the content was synthetic.

Today, similar anomalies are much less frequent. For this reason, we separate the visible content of the image from imperceptible traces, such as small noise patterns characteristic of different generative systems. In some cases, these traces make it possible not only to recognize that content is synthetic, but also to identify the model that generated it.

Paolo Bestagini: Our work begins with research and the proposal of new algorithms. We then develop software based on these algorithms: we take the image, video or audio track, analyse it using the program and try to extract the features useful for the assessment.

Stefano Tubaro: There are companies that offer services to analyse a file and indicate, with a given level of probability, whether it is natural or synthetic.

In the audio and video fields, being able to do this in real time is particularly useful, for example during a suspicious phone call or a videoconference in which the voice and image might be artificially generated. Verifying whether there is a real person on the other end therefore has very concrete applications.

Paolo Bestagini: It depends on the technique used. Some systems work very well in limited domains: they recognize a particular artefact with great reliability, but they are not necessarily able to interpret everything that does not fall within that specific case.

The problem is that new generation technologies may appear continuously. An effective detector today therefore risks becoming obsolete. New examples must be collected, the data updated and the software retrained, or new versions developed to be used alongside the previous ones.

The most immediate comparison is with antivirus software. Antivirus software installed today protects against certain threats and may also detect some future anomalies, but if it is not updated for a year, it will probably be much less effective against entirely new viruses.

Stefano Tubaro: We take a conservative approach: we declare content authentic only when we find many traces attesting to its authenticity. Otherwise, we flag the uncertainty, even if we cannot identify the model that generated it or state with certainty that it is synthetic, and further checks can be initiated.

Stefano Tubaro: The distinction is much less clear-cut than it might seem. An artificial intelligence system can regenerate an image in an apparently identical way, and a smartphone also applies a great deal of processing before showing us a photograph. Establishing what is “true” or “false” is therefore not always straightforward.

It is nevertheless important to be aware that photographs, videos and audio recordings may be synthetic or manipulated. We can no longer automatically consider them a direct representation of reality: we must avoid accepting them uncritically.

Stefano Tubaro: We can look for other sources to verify whether the event depicted actually took place and check the consistency between the image and the accompanying text. These are simple checks, but they help us look more consciously at what is in front of us.

Paolo Bestagini: In the case of phone calls, one of the first things to pay attention to is urgency. Someone who wants to deceive a person often tries not to give them time to think and pushes them to make an immediate decision. When someone puts us under pressure, the first step should be to stop and take a step back.

Another possibility is to reverse the communication: end the call and call the person back directly using the number we already know. In this way, we can verify whether we were actually speaking to them.

We can also ask very personal questions about information that only the real person should know. These are simple precautions that do not require technological tools but can help thwart a scam.

Many attempts succeed not because they are perfect, but because they can be produced in large quantities and at very low cost. It is enough for a minimal percentage of the people contacted to fall for the deception. Stopping for a few seconds and paying attention to inconsistencies can therefore already be very helpful.

Stefano Tubaro: When the first systems capable of generating images and audio appeared, the quality was still low, but the pace of improvement suggested that they would soon produce content indistinguishable, at least to a person, from natural content.

Some US colleagues with whom we collaborated on projects funded by DARPA, the research agency of the United States Department of Defense, also steered us in this direction: as early as 2019–2020, they viewed synthetic content as a problem destined to become central.

A decisive factor was the increase in computing power. Principles formulated as early as the 1960s and 1970s became applicable thanks to more powerful devices and the use of graphics cards to perform large amounts of computation, although energy consumption remains high.

Paolo Bestagini: Fakes are not, in fact, a new phenomenon. Manipulated photographs have existed since the beginnings of photography: historical images in which one person's face was superimposed on another person's body, or in which subjects and backgrounds came from different shots.

Later came Photoshop, online services and smartphone applications. Today, artificial intelligence makes the creation of fakes accessible to a much wider number of people. The real novelty is not so much the existence of fakes as their proliferation.

Paolo Bestagini: Techniques already exist that embed a kind of signature in a photograph when it is taken. The signature can be verified by an authority and makes it possible to certify that the image comes from a particular device and has not subsequently been modified.

Similar solutions are also adopted by some tools that produce synthetic images for non-malicious purposes. The file may contain a certificate indicating that the image was generated by that system.

However, it is not a definitive solution. If I take a screenshot of the image or photograph it again, the certificate embedded in the original file may be lost. In this case too, the system can be circumvented, but certifying origin nevertheless remains one of the avenues being explored.

Paolo Bestagini: We need to accept that almost all content will, sooner or later, be processed in some way. Even a photograph produced by a smartphone is not a completely neutral representation of the scene: it contains a great deal of processing.

The question, therefore, may no longer be only whether content is true or false, but whether it has been modified to influence a decision or change the meaning of what it represents. We will have to distinguish between processing that does not alter the meaning of the scene and interventions that instead have a semantic impact.

It will also be necessary to analyse jointly all the information relating to an event: the images, the text accompanying them, the audio recordings and other associated content. Only by relating these elements to one another can we understand whether a news story is presented correctly or whether it is constructed to induce a particular interpretation.

This will require a broader perspective and probably new collaborations as well. Engineering expertise will need to be complemented by expertise in the humanities, for example in semiotics and the study of social impacts.

Stefano Tubaro: An authentic photograph may accompany a news story about something entirely different and thus contribute to false information: in this case, it is the context, not the individual piece of content, that changes its meaning.

If most images are processed, limiting ourselves to determining whether they are true or false risks becoming of little use. Initially, we might have thought of stopping fakes before publication; today, it is difficult to prevent the entire phenomenon in advance. We must therefore analyse what circulates and reconstruct its context, knowing that a false image can have an effect even after it has been proven false.

Research will therefore have to continue developing technical tools, but also examine the cultural and social transformations produced by these technologies.

Recognizing a fake no longer means pursuing a fixed boundary between what is natural and what is synthetic. It means continuously updating analysis systems, cross-referencing sources and languages, and understanding the role that content takes on in the context in which it is used. Because an image can be modified without any intent to deceive, while a technically authentic photograph can be used to tell of something that never happened. The next frontier of research will be to understand not only how content was produced, but also how it can influence the way we interpret reality.