The AI landscape has started to move very, very fast: consumer-facing tools like halfway and ChatGPT they can now produce amazing image and text results in seconds based on natural language prompts, and we’re seeing it implemented everywhere from web search to children’s books.
However, these AI applications are turning to more nefarious uses, including spread malware. Take traditional scam email for example: it’s usually full of obvious mistakes in its grammar and spelling, mistakes that the latest batch of AI models don’t make, as noted in a recent Europol advisory report.
Think about it: Many phishing attacks and other security threats rely on social engineering, tricking users into revealing passwords, financial information, or other sensitive data. The authentic-sounding, persuasive copy required for these scams can now be extracted quite easily, without the need for human effort, and endlessly tweaked and refined for specific audiences.
In the case of ChatGPT, it is important to note first that the OpenAI developer has built protections into it. Ask it to “write malware” or a “phishing email” and it will tell you that it is “programmed to follow strict ethical guidelines that prohibit me from engaging in any malicious activity, including writing or assisting in the creation of malware.”
However, these protections aren’t too hard to circumvent: ChatGPT can certainly encrypt, and it can certainly compose email. Even if you don’t know you are writing malware, you may be asked to enter producing something like that. There is already sign that cybercriminals are working to circumvent the security measures that have been put in place.
We’re not particularly referring to ChatGPT here, but pointing out what’s possible once Large Language Models (LLMs) like this are used for more sinister purposes. In fact, it is not too difficult to imagine criminal organizations developing their own LLMs and similar tools to make your scams sound more convincing. And it’s not just text – audio and video are harder to fake – but it’s happening too.
When it comes to your boss urgently asking for a report, or company tech support telling you to install a security patch, or your bank telling you there’s a problem you need to respond to, all these potential scams they are based on building trust and sounding genuine. , and that’s something that AI bots are doing very well in. They can produce natural-sounding text, audio, and video tailored to specific audiences, and they can do it quickly and consistently on demand.