Research conducted by Kaspersky experts with regard to the detection capability of ChatGPT for phishing links revealed that despite its ability to create phishing emails and write malware, the AI chatbot had limitations in effectively detecting malicious links. Based on the study, ChatGPT has knowledge about phishing and can speculate the target of a phishing attack; however, it had a high false positive rate of up to 64%. The same study also found that ChatGPT produced imaginary explanations and false evidence to justify its verdicts, further highlighting its limitations in detecting malicious links.
The potential of ChatGPT, an AI-powered language model, to create phishing emails has been a topic of discussion in the cybersecurity world, leading to concerns about its impact on the job security of cybersecurity experts. Despite the creators’ warnings that it is too early to apply this novel technology to high-risk domains. In order to reveal ChatGPT’s ability to detect phishing links and its cybersecurity knowledge acquired during training, Kaspersky experts decided to conduct an experiment. The experts tested gpt-3.5-turbo, which powers ChatGPT, on over 2,000 links that Kaspersky anti-phishing technologies had deemed phishing, and mixed them with thousands of safe URLs.
Detection rates based on the experiment vary depending on the prompt used. ChatGPT was asked two (02) questions: “Does this link lead to a phishing website?” and “Is this link safe to visit?”. The results showed that ChatGPT had a detection rate of 87.2% and a false positive rate of 23.2% for the first question. For the second question, it resulted in a higher detection rate of 93.8%, but a higher false positive rate of 64.3%. While the detection rate is very high, the false positive rate is too high for any kind of production application.
| Question Asked | Detection Rate | False Positive Rate |
| Does this link lead to a phishing website? | 87.2% | 23.2% |
| Is this link safe to visit? | 93.8% | 64.3% |
“ChatGPT certainly shows promise in assisting human analysts in detecting phishing attacks but let’s not get ahead of us – language models still have their limitations. While they might be on par with an intern-level phishing analyst when it comes to reasoning about phishing attacks and extracting potential targets, they tend to hallucinate and produce random output. So, while they might not revolutionize the cybersecurity landscape just yet, they could still be helpful tools for the community,” according to Vladislav Tushkanov, Lead Data Scientist at Kaspersky.
While the detection capabilities of ChatGPT in phishing prevention were unsatisfactory, the AI language model demonstrated potential in identifying potential phishing targets by successfully extracting a target from over half of the URLs tested, including major tech portals, marketplaces, and global banks. However, the experiment also revealed ChatGPT’s limitations in proving its decision on whether the link is malicious, as it produced misleading explanations and statements despite its confident tone. These limitations reflect known challenges with language models, including hallucinations and misstatements.
Kaspersky’s ML team is a leader in the application of machine learning technologies to cybersecurity tasks, consistently updating Kaspersky products with the latest tech and intel. To stay protected, the company’s experts recommend utilizing Kaspersky Managed Detection and Response for corporate cybersecurity, which utilizes advanced machine-learning models to filter out mundane events and send alarming ones to professional human analysts, enhancing a company’s ability to withstand cyber threats while optimizing the use of existing workforce resources. In addition, providing staff with basic cybersecurity hygiene training and conducting simulated phishing attacks can help ensure they know how to distinguish phishing emails. The cybersecurity firm also recommended using the latest Threat Intelligence information to stay aware of actual tactics, techniques, and procedures used by threat actors is also recommended to enhance cybersecurity.
Learn more about this experiment conducted by Kaspersky by visiting Securelist.com.