OpenAI, the maker of Microsoft-backed consumer-facing artificial intelligence (AI) service ChatGPT, may have scored something of an own-goal with the unveiling of Voice Engine, billed as “a model for creating custom voices”.
While OpenAI’s blog on Friday highlights the legitimate use of voice cloning, sometimes referred to as ‘deepfake voice’, such as providing reading assistance to non-readers and children, its widespread availability could soon metamorphose into a cybersecurity nightmare.
Deepfake voice and video software are already being used by cybercriminals to mimic the voices of senior executives to commit financial fraud and other crimes. But the widespread availability and marketing of deepfake voice software is now set to make cybercrime a virtual cottage industry where any number can play. It will open the floodgates to a whole new generation of cybercriminals, terrorists, pranksters, and disgruntled employees.
All incoming voice calls, whether business or personal, will soon need to be treated with the same degree of caution and skepticism now applied to emails coming from an unknown source. Incoming phone calls from a superior requesting information or demanding action will have to be disregarded until further verification can be made. Similarly, calls from colleagues, business contacts, or even family members will have to be treated in a similar fashion.
Deepfake calls demanding ransom payments
In practice, this will not be easy. A call from a family member or friend in what sounds like a genuine emergency will be tough for most people to ignore, as there will always be a possibility that the call is genuine. There are already numerous examples from across the world of deepfake voice scams where the caller claims to have been kidnapped or imprisoned, pleading for a ransom or payment demand to be met.
The deepfake genie may, however, already be out of the bottle, as a simple search will bring up dozens of existing voice cloning services capable of creating deepfake scams that are already freely available on the open market. There is no suggestion that any of these services or OpenAI’s Voice Engine were created with such nefarious purposes in mind – any more than a manufacturer of hunting rifles could be accused of assisting in assassination attempts.
But, just as cybercriminals and terrorists have been quick to realize the criminal potential of AI offerings such as ChatGPT, the increasingly widespread availability of voice cloning services, some of which can create a deepfake in seconds, will mean this type of scam will become increasingly commonplace. Already fearing a possible backlash, OpenAI is already furiously back-peddling its new voice cloning offering by only reviewing Voice Engine and withholding its release for the time being.
“We recognize that generating speech that resembles people’s voices has serious risks, which are especially top of mind in an election year. We are engaging with US and international partners from across government, media, entertainment, education, civil society, and beyond to ensure we are incorporating their feedback as we build,” admitted OpenAI in its blog post on Friday.
It remains to be seen whether the currently proposed safeguards will soon be viewed as a classic case of ‘too little, too late.’
