ChatGPT's new image generator continues to generate buzz. Shortly after the feature's rollout, many internet users realized that it was possible to generate fake receipts and expense reports. With GPT-4o, users can design highly realistic restaurant receipts, without the slightest pricing error. It's also possible to tamper with an existing receipt, for example by asking it to change a date or the amount of a bill. All these operations take only a few seconds and do not require any special image editing skills.
A fake passport signed by ChatGPT
A Polish researcher has gone even further by creating a passport with the help of generative AI. On his X account, Borys Musielak explains that he asked ChatGPT to "create a replica" of his passport that would be able to fool "most automated KYC systems". The researcher is referring to technologies that automatically identify and verify the identity of customers, often when registering for a banking, financial, or online service.
This is the KYC (Know Your Customer) process, imposed to combat fraud or money laundering. As part of verification, automated tools analyze identity documents, take selfies for facial recognition, or cross-reference data with official databases.
Image verification is "obsolete"
As Borys Musielak points out, it took less than five minutes to design a replica that automated verification systems "would probably accept without a second thought." For the researcher, the possibilities offered by ChatGPT are a game-changer for many identity verification systems.
From now on, anyone can forge anything in record time. All processes that rely on photos are therefore a thing of the past, the researcher believes. Financial platforms that rely on sending a selfie and an identity document, such as Revolut, N26, Coinbase, or Binance, can be fooled by ChatGPT. On the other hand, the absence of a chip on the forged passport will not pass scrutiny. A human will see through the deception.
The expert believes that eID wallets (for electronic IDentification), which will allow citizens to securely store their identity documents, driving licenses, diplomas, or means of payment on their smartphones, are one of "the only viable paths" in a world that takes AI into account.
We tried to replicate Borys Musielak's experiment by making a copy of our own passport and identity card. ChatGPT firmly refused. to comply, assuring that they "cannot provide copies of official identity documents." This includes "any reproduction, even for personal use, due to data protection and fraud prevention legislation."
It is possible that the researcher used a well-calibrated query to bypass OpenAI's security mechanisms. The startup may also have patched a hole in the operation of its generator, which has remained highly random since it went live. In our findings, ChatGPT sometimes agrees to generate images that its restrictions prohibit... before refusing. In other cases, the AI refuses to produce images and access authorized queries.
0 Comments