Researchers have successfully used AI to create malware kits, underscoring the real-world genAI risks that exist alongside potential business benefits.
“Hornetsecurity researchers tested a new popular large language model (LLM) and successfully created a phishing toolkit, highlighting the potential dangers of AI,” the vendor said.
The security lab team at Hornetsecurity reported that AI “continues to be a double-edged sword”. This confirms that the evolving technology represents risks as well as potential gains.
“With AI being such a large part of the security ecosystem, we commonly look at new models and techniques. We look at how those models may be used for either defensive security or by threat actors,” the vendor said.
As well as initial testing to reveal detail on such aspects such as the reasoning scheme, Hornetsecurity tried to make OpenAI‘s new o3-mini LLM produce a malicious tool or malware kit.
“The first prompt we posed to the system exploring this possibility ultimately requested a corrected version of a provided sentence,” the vendor said. “But we found that it interpreted a prompt in a way that led down the path for creating a ‘Netflix connection page’.”
GenAI risks present in prompt responses
The model created a login page, payment page and OTP bank page, all resembling Netflix’s design. Although sometimes the model pushed back, bypassing the restrictions proved simple, Hornetsecurity said.
The related HTML file included HTML, CSS, and JavaScript without any warnings about phishing, and it neither blocked nor ignored the researcher request. The team then asked for multiple pages to be created.
“Amazingly, the model provided the needed code for each page requested by the team,” the vendor said.
“At this stage the team asked the model to create what amounted to a whole toolkit. That’s complete with a full package including the back-end, front-end, and even a Telegram bot with multiple buttons.”
Attackers could deploy the first version in just a few clicks via a cloud server provider, the vendor said.
“Once the code was running, it became quickly apparent that this could quickly be put to malicious use in the wrong hands.”
Lab report on malware kits, ransomware and more
Hornetsecurity’s monthly threat report releases insights into M365 security trends. Consequently they also look at email-based threats, and commentary on current events in the cybersecurity space.
The vendor bases monthly reports on the previous calendar month.
In addition, the March 2025 update looked at a TD Bank leak and the Medusa cybercrime group attack on the HCRG Care Group. Hornetsecurity also talked about the latest Microsoft vulnerabilities, and the Apple iCloud backdoor.
In addition, Hornetsecurity provides cloud-based security, compliance, backup, and security awareness solutions. The flagship offering is 365 Total Protection, a comprehensive cloud security suite for Microsoft 365 (M365).
It has around 12,000 channel partners worldwide, including managed services providers (MSPs).
( Photo by Solen Feyissa on Unsplash )
