Veriti’s latest research dives into the double-edged sword of AI in cybersecurity, revealing both its transformative potential and the substantial risks it introduces. While AI is redefining how organizations approach security, it also presents significant challenges, including data misuse, unauthorized access, and its weaponization by advanced threat actors.
Data Misuse and Unauthorized Access
The integration of AI technologies into workplace tools is redefining operational efficiency, as exemplified by Microsoft’s Recall feature within the Copilot AI suite.
Microsoft’s Recall is designed to enhance productivity by automatically capturing and storing snapshots of desktop activities, enabling users to quickly retrieve past actions and documents.
While this functionality promotes a highly efficient workflow, it also introduces significant privacy and security risks that cannot be overlooked.
These risks primarily arise from how Recall handles the data it captures, snapshots of user activities stored locally on hard drives, potentially unencrypted. This storage method poses a dual threat: first, the accidental exposure of sensitive information through data breaches or mishandling, and second, the potential for misuse by unauthorized entities who could access these data repositories. The fact that these snapshots could contain sensitive financial documents, personal employee information, or proprietary business data exacerbates the risks, turning a utility tool into a potential liability.
Microsoft has taken commendable steps by delaying Recall’s launch to address these privacy and security concerns, highlighting a proactive stance towards responsible AI deployment. However, this action opens up a broader debate about the balance AI tools must strike between functionality and privacy. The core challenge here is developing AI solutions that do not compromise on either front, ensuring that enhancements in operational efficiency do not come at the cost of security or user privacy.
This balancing act is not just about enhancing existing security measures but also about rethinking the approach to AI design and deployment in enterprise environments. It involves a shift from a purely functional perspective to one that equally prioritizes ethical considerations, particularly data protection.
AI-Enhanced Cyber Attacks
Veriti’s research reveals that while AI enhances cybersecurity defenses, it is also being weaponized by advanced threat actors. Groups like SweetSpecter and CyberAv3ngers are leveraging AI to conduct highly targeted and adaptive attacks, from crafting undetectable spear-phishing campaigns to exploiting industrial control system vulnerabilities in real time.
- SweetSpecter: This cyber collective, believed to originate from China, has adeptly integrated AI into its operations, particularly enhancing their capabilities in reconnaissance and spear phishing. SweetSpecter’s modus operandi involves deploying AI to analyze vast datasets to identify vulnerabilities in target organizations, crafting highly personalized spear phishing emails that are difficult to detect by traditional anti-phishing tools.
The types of questions asked by SweetSpecter
| Activity | LLM ATT&CK Framework Category |
| Asking about vulnerabilities in various applications | LLM-informed reconnaissance |
| Asking how to search for specific versions of Log4j that are vulnerable to the critical RCE Log4Shell | |
| Asking about popular content management systems used abroad | |
| Asking for themes that government department employees would find interesting and what would be good names for attachments to avoid being blocked | LLM-supported social engineering |
| Asking for variations of an attacker-provided job recruitment message | |
| Asking how sqlmap would be used to upload a potential web shell to a target server | LLM-assisted vulnerability research |
For example, they have been reported to tailor phishing messages based on employees’ job functions and recent activities by extracting such information through AI-enhanced social engineering techniques.
In May 2024, UNK_SweetSpecter cleverly employed a free email account to send an intriguing AI-themed lure, encouraging the recipient to explore an attached zip archive.
CyberAv3ngers: Linked to Iran’s Revolutionary Guard, CyberAv3ngers primarily targets critical infrastructure, such as energy grids and water treatment facilities. Leveraging AI, they automate the discovery of vulnerabilities in industrial control systems (ICS).
The types of questions asked by CyberAv3ngers:
| Activity | LLM ATT&CK Framework Category |
| Asking to list commonly used industrial routers in Jordan | LLM-informed reconnaissance |
| Asking to list industrial protocols and ports that can connect to the Internet | |
| Asking for the default password for a Tridium Niagara device | |
| Asking for the default user and password of a Hirschmann RS Series Industrial Router | |
| Asking for recently disclosed vulnerabilities in CrushFTP and the Cisco Integrated Management Controller as well as older vulnerabilities in the Asterisk Voice over IP software | |
| Asking for lists of electricity companies, contractors, and common PLCs in Jordan | |
| Asking why a bash code snippet returns an error | LLM-enhanced scripting techniques |
One documented instance involved the use of AI to rapidly test and adapt attack methods in real-time during an intrusion, allowing them to exploit system weaknesses before administrators could detect and react to the initial breach.
The CyberAv3ngers targeted integrated PLCs/HMIs used in water treatment facilities in the United States and Israel. These attacks demonstrated the group’s ability to manipulate industrial control systems, specifically exploiting vulnerabilities in the Unitronics Vision and Samba series of products which, at the time, lacked PCOM password protection. This allowed the attackers to remotely control the devices and deface them, posing significant threats to the operational security of critical infrastructure.
Veriti research shows that these groups are a stark warning about the evolving threat landscape. The same intelligence and adaptability that make AI invaluable for defensive measures are being exploited by attackers to devastating effect. SweetSpecter and CyberAv3ngers highlight a troubling shift: cyber threats are becoming both highly automated and disturbingly intelligent. These actors are not only leveraging AI to refine their tactics but are also executing precision attacks that can penetrate specific targets with alarming efficiency and minimal detection.
THE INTELLIGENCE AND ADAPTABILITY PROVIDED BY AI THAT ARE INVALUABLE FOR DEFENSIVE SECURITY MEASURES ARE EQUALLY POTENT WHEN MISUSED FOR OFFENSIVE PURPOSES.
The Implications for Cybersecurity
For organizations and cybersecurity professionals, the rise of AI-powered threats necessitates a reevaluation of traditional security strategies. There is a critical need for proactive defense mechanisms that can anticipate and neutralize AI-driven attacks before they manifest.
In light of these challenges, Veriti’s Exposure Assessment and Remediation solution emerges as an essential advancement in managing security risks. It proactively identifies and corrects misconfigurations at the operating system level, which are critical for preventing unauthorized data access and potential misuse. For example, Veriti’s system can detect when Windows policy settings allow inadvertent data sharing with Microsoft Co-Pilot for AI training, involving sensitive files like presentations and Excel documents. By preemptively securing these vulnerabilities, Veriti enables organizations to speedily, and safely harden their defenses against the exploitation of AI technologies by advanced threat actors.
