Detecting AI-Manipulated Content Is A Challenging Arms Race

3 minutes, 38 seconds Read

Declarations are no guarantee

A study by DR (In Danish) found that one in three children aged 9-14 never considers the possibility that photos and videos on social media could be manipulated. The many deepfakes have now led Meta, which owns Facebook and Instagram, to try to detect computer-generated images and videos. At the same time, the EU’s AI Act – the world’s first AI legislation – will make it mandatory to declare computer-generated content. However, even though regulation is being drafted in this area and tech companies will seek to detect deepfakes, Morten Mørup believes there is no guarantee that we can avoid seeing much more deepfake content in the future.

“Starting to declare deepfakes is definitely an important step, but there will still be people who can generate content without it being declared. Research is being done on developing AI-based deepfake detectors, but then we are back to the arms race. And it’s an arms race that I think will be very hard to win. We must therefore not turn a blind eye and just assume that anything that is not declared as deepfake is real,” says Morten Mørup.

There is currently another method for detecting deepfakes that completely bypasses AI and is based on thorough research.

“You can try to check a deepfake against other information. Suppose a video clip shows something that happened in Ukraine, for example. In that case, you can compare it with satellite photos and weather information at the time to see if everything matches the video clip. For example, was it raining that day, yet the video clip shows a cloudless sky? The only problem is that the AI models can potentially also have access to the information we are checking the video against. So a good deepfake will ensure that it is raining in the video,” says Morten Mørup.

A world of misinformation

In 2019, the CEO of a British energy company received a call from what he believed to be his superior in the parent company in Germany, telling him to transfer EUR 220,000 to a bank account. In reality, the CEO had been tricked by a deepfake. A con man had used AI to generate his superior’s voice so convincingly that the director transferred the money immediately. In February 2024, a large company in Hong Kong experienced a similar incident and was defrauded USD 25.6 million.

In Denmark, the Ministry of Foreign Affairs will more closely monitor diplomatic video conversations after the Foreign Minister, Lars Løkke Rasmussen (M), experienced a deepfake call last year from a group of Russian comedians who had faked the face and voice of Moussa Faki, commission chairman of the African Union.

While preventing similar scams and spreading deepfake-generated misinformation can be difficult, Morten Mørup believes that greater awareness of the issues is key to limiting the problem.

“Declaration requirements will make it harder for regular users to make deepfakes without being detected, but there will continue to be major players out there who will defraud others or influence democratic processes. We, therefore, need to recognize that these technologies exist and act accordingly. We need to practise source criticism and understand that we live in a world of misinformation, where manipulation exists that can be very difficult to detect. Our common understanding of what is real can be threatened as a society. It will be a big problem if we start to reject truths as misinformation and fake because they don’t mesh with our worldview,” says Morten Mørup.

The images of Taylor Swift were subsequently deleted, and searches using the singer’s name were disabled for a period on X to prevent sharing new images. Since then, several US politicians, including Congresswoman Yvette Clarke (Democrat), have called for legislation to ban the creation and sharing of deepfakes as pornographic content on the internet.

Source: DTU

Source link

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *