The State of AI in Cybersecurity: How AI will impact the cyber threat landscape in 2024
22
Apr 2024
Part 2: This blog discusses the impact of AI on the cyber threat landscape based on data from Darktrace’s State of AI Cybersecurity Report. Get the latest insights into the evolving challenges faced by organizations, the growing demand for skilled professionals, and the need for integrated security solutions.
About the AI Cybersecurity Report
We surveyed 1,800 CISOs, security leaders, administrators, and practitioners from industries around the globe. Our research was conducted to understand how the adoption of new AI-powered offensive and defensive cybersecurity technologies are being managed by organizations.
Are organizations feeling the impact of AI-powered cyber threats?
Nearly three-quarters (74%) state AI-powered threats are now a significant issue. Almost nine in ten (89%) agree that AI-powered threats will remain a major challenge into the foreseeable future, not just for the next one to two years.
However, only a slight majority (56%) thought AI-powered threats were a separate issue from traditional/non AI-powered threats. This could be the case because there are few, if any, reliable methods to determine whether an attack is AI-powered.
Identifying exactly when and where AI is being applied may not ever be possible. However, it is possible for AI to affect every stage of the attack lifecycle. As such, defenders will likely need to focus on preparing for a world where threats are unique and are coming faster than ever before.
Are security stakeholders concerned about AI’s impact on cyber threats and risks?
The results from our survey showed that security practitioners are concerned that AI will impact organizations in a variety of ways. There was equal concern associated across the board – from volume and sophistication of malware to internal risks like leakage of proprietary information from employees using generative AI tools.
What this tells us is that defenders need to prepare for a greater volume of sophisticated attacks and balance this with a focus on cyber hygiene to manage internal risks.
One example of a growing internal risks is shadow AI. It takes little effort for employees to adopt publicly-available text-based generative AI systems to increase their productivity. This opens the door to “shadow AI”, which is the use of popular AI tools without organizational approval or oversight. Resulting security risks such as inadvertent exposure of sensitive information or intellectual property are an ever-growing concern.
Are organizations taking strides to reduce risks associated with adoption of AI in their application and computing environment?
71.2% of survey participants say their organization has taken steps specifically to reduce the risk of using AI within its application and computing environment.
16.3% of survey participants claim their organization has not taken these steps.
These findings are good news. Even as enterprises compete to get as much value from AI as they can, as quickly as possible, they’re tempering their eager embrace of new tools with sensible caution.
Still, responses varied across roles. Security analysts, operators, administrators, and incident responders are less likely to have said their organizations had taken AI risk mitigation steps than respondents in other roles. In fact, 79% of executives said steps had been taken, and only 54% of respondents in hands-on roles agreed. It seems that leaders believe their organizations are taking the needed steps, but practitioners are seeing a gap.
Do security professionals feel confident in their preparedness for the next generation of threats?
A majority of respondents (six out of every ten) believe their organizations are inadequately prepared to face the next generation of AI-powered threats.
The survey findings reveal contrasting perceptions of organizational preparedness for cybersecurity threats across different regions and job roles. Security administrators, due to their hands-on experience, express the highest level of skepticism, with 72% feeling their organizations are inadequately prepared. Notably, respondents in mid-sized organizations feel the least prepared, while those in the largest companies feel the most prepared.
Regionally, participants in Asia-Pacific are most likely to believe their organizations are unprepared, while those in Latin America feel the most prepared. This aligns with the observation that Asia-Pacific has been the most impacted region by cybersecurity threats in recent years, according to the IBM X-Force Threat Intelligence Index.
The optimism among Latin American respondents could be attributed to lower threat volumes experienced in the region, but it's cautioned that this could change suddenly (1).
What are biggest barriers to defending against AI-powered threats?
The top-ranked inhibitors center on knowledge and personnel. However, issues are alluded to almost equally across the board including concerns around budget, tool integration, lack of attention to AI-powered threats, and poor cyber hygiene.
The cybersecurity industry is facing a significant shortage of skilled professionals, with a global deficit of approximately 4 million experts (2). As organizations struggle to manage their security tools and alerts, the challenge intensifies with the increasing adoption of AI by attackers. This shift has altered the demands on security teams, requiring practitioners to possess broad and deep knowledge across rapidly evolving solution stacks.
Educating end users about AI-driven defenses becomes paramount as organizations grapple with the shortage of professionals proficient in managing AI-powered security tools. Operationalizing machine learning models for effectiveness and accuracy emerges as a crucial skill set in high demand. However, our survey highlights a concerning lack of understanding among cybersecurity professionals regarding AI-driven threats and the use of AI-driven countermeasures indicating a gap in keeping pace with evolving attacker tactics.
The integration of security solutions remains a notable problem, hindering effective defense strategies. While budget constraints are not a primary inhibitor, organizations must prioritize addressing these challenges to bolster their cybersecurity posture. It's imperative for stakeholders to recognize the importance of investing in skilled professionals and integrated security solutions to mitigate emerging threats effectively.
1. IBM, X-Force Threat Intelligence Index 2024, Available at: https://www.ibm.com/downloads/cas/L0GKXDWJ
2. ISC2, Cybersecurity Workforce Study 2023, Available at: https://media.isc2.org/-/media/Project/ISC2/Main/Media/ documents/research/ISC2_Cybersecurity_Workforce_Study_2023.pdf?rev=28b46de71ce24e6ab7705f6e3da8637e
Like this and want more?
Receive the latest blog in your inbox
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Newsletter
Stay ahead of threats with the Darktrace blog newsletter
Get the latest insights from the cybersecurity landscape, including threat trends, incident analysis, and the latest Darktrace product developments – delivered directly to your inbox, monthly.
Thanks, your request has been received
A member of our team will be in touch with you shortly.
Oops! Something went wrong while submitting the form.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
A Snake in the Net: Defending Against AiTM Phishing Threats and Mamba 2FA
What are Adversary-in-the-Middle (AiTM) phishing kits?
Phishing-as-a-Service (PhaaS) platforms have significantly lowered the barriers to entry for cybercriminals, enabling a new wave of sophisticated phishing attacks. Among the most concerning developments in this landscape is the emergence of Adversary-in-the-Middle (AiTM) phishing kits, which enhance traditional phishing tactics by allowing attackers to intercept and manipulate communications in real-time. The PhaaS marketplace offers a wide variety of innovative capabilities, with basic services starting around USD 120 and more advanced services costing around USD 250 monthly [1].
These AiTM kits are designed to create convincing decoy pages that mimic legitimate login interfaces, often pre-filling user information to increase credibility. By acting as a man-in-the-middle, attackers can harvest sensitive data such as usernames, passwords, and even multi-factor authentication (MFA) tokens without raising immediate suspicion. This capability not only makes AiTM attacks more effective but also poses a significant challenge for cybersecurity defenses [2].
Mamba 2FA is one such example of a PhaaS strain with AiTM capabilities that has emerged as a significant threat to users of Microsoft 365 and other enterprise systems. Discovered in May 2024, Mamba 2FA employs advanced AiTM tactics to bypass MFA, making it particularly dangerous for organizations relying on these security measures.
What is Mamba 2FA?
Phishing Mechanism
Mamba 2FA employs highly convincing phishing pages that closely mimic legitimate Microsoft services like OneDrive and SharePoint. These phishing URLs are crafted with a specific structure, incorporating Base64-encoded parameters. This technique allows attackers to tailor the phishing experience to the targeted organization, making the deception more effective. If an invalid parameter is detected, users are redirected to a benign error page, which helps evade automated detection systems [5].
Real-Time Communication
A standout feature of Mamba 2FA is its use of the Socket.IO JavaScript library. This library facilitates real-time communication between the phishing page and the attackers' backend servers. As users input sensitive information, such as usernames, passwords, and MFA tokens on the phishing site, this data is immediately relayed to the attackers, enabling swift unauthorized access [5].
Multi-Factor Authentication Bypass
Mamba 2FA specifically targets MFA methods that are not resistant to phishing, such as one-time passwords (OTPs) and push notifications. When a user enters their MFA token, it is captured in real-time by the attackers, who can then use it to access the victim's account immediately. This capability significantly undermines traditional security measures that rely on MFA for account protection.
Infrastructure and Distribution
The platform's infrastructure consists of two main components: link domains and relay servers. Link domains handle initial phishing attempts, while relay servers are responsible for stealing credentials and completing login processes on behalf of the attacker. The relay servers are designed to mask their IP addresses by using proxy services, making it more difficult for security systems to block them [3].
Evasion Techniques
To evade detection by security tools, Mamba 2FA employs several strategies:
Sandbox Detection: The platform can detect if it is being analyzed in a sandbox environment and will redirect users to harmless pages like Google’s 404 error page.
Dynamic URL Generation: The URLs used in phishing attempts are frequently rotated and often short-lived to avoid being blacklisted by security solutions.
HTML Attachments: Phishing emails often include HTML attachments that appear benign but contain hidden JavaScript that redirects users to the phishing page [5].
Darktrace’s Coverage of Mamba 2FA
Starting in July 2024, the Darktrace Threat Research team detected a sudden rise in Microsoft 365 customer accounts logging in from unusual external sources. These accounts were accessed from an anomalous endpoint, 2607:5500:3000:fea[::]2, and exhibited unusual behaviors upon logging into Software-as-a-Service (SaaS) accounts. This activity strongly correlates with a phishing campaign using Mamba 2FA, first documented in late June 2024 and tracked as Mamba 2FA by Sekoia [2][3].
Darktrace / IDENTITY was able to identify the initial stages of the Mamba 2FA campaign by correlating subtle anomalies, such as unusual SaaS login locations. Using AI based on peer group analysis, it detected unusual behavior associated with these attacks. By leveraging Autonomous Response actions, Darktrace was able to neutralize these threats in every instance of the campaign detected.
On July 23, a SaaS user was observed logging in from a rare ASN and IP address, 2607:5500:3000:fea::2, originating from the US and successfully passed through MFA authentication.
Almost an hour later, the SaaS user was observed logging in from another suspicious IP address, 45.133.172[.]86, linked to ASN AS174 COGENT-174. This IP, originating from the UK, successfully passed through MFA validation.
Following this unusual access, the SaaS user was notably observed reading emails and files that could contain sensitive payment and contract information. This behavior suggests that the attacker may have been leveraging contextual information about the target to craft further malicious phishing emails or fraudulent invoices. Subsequently, the user was detected creating a new mailbox rule titled 'fdsdf'. This rule was configured to redirect emails from a specific domain to the 'Deleted Items' folder and automatically mark them as read.
Implications of Unusual Email Rules
Such unusual email rule configurations are a common tactic employed by attackers. They often use these rules to automatically forward emails containing sensitive keywords—such as "invoice”, "payment", or "confidential"—to an external address. Additionally, these rules help conceal malicious activities, keeping them hidden from the target and allowing the attacker to operate undetected.
Blocking the action
A few minutes later, the SaaS user from the unusual IP address 45.133.172[.]86 was observed attempting to send an email with the subject “RE: Payments.” Subsequently, Darktrace detected the user engaging in activities that could potentially establish persistence in the compromised account, such as registering a new authenticator app. Recognizing this sequence of anomalous behaviors, Darktrace implemented an Autonomous Response inhibitor, disabling the SaaS user for two hours. This action effectively contained potential malicious activities, such as the distribution of phishing emails and fraudulent invoices, and gave the customer’s security team the necessary time to conduct a thorough investigation and implement appropriate security measures.
In another example from mid-July, similar activities related to the campaign were observed on another customer network. A SaaS user was initially detected logging in from the unusual external endpoint 2607:5500:3000:fea[::]2.
A few minutes later, in the same manner as demonstrated in the previous case, the actor was observed logging in from another rare endpoint, 102.68.111[.]240. However, this time it was from a source IP located in Lagos, Nigeria, which no other user on the network had been observed connecting from. Once logged in, the SaaS user updated the settings to "User registered Authenticator App with Notification and Code," a possible attempt to maintain persistence in the SaaS account.
Based on unusual patterns of user behavior, a Cyber AI Analyst Incident was also generated, detailing all potential account hijacking activities. Darktrace also applied an Autonomous Response action, disabling the user for over five hours. This swift action was crucial in preventing further unauthorized access, potential data breaches and further implications.
Since the customer had subscribed to Darktrace Security Operations Centre (SOC) services, Darktrace analysts conducted an additional human investigation confirming the account compromise.
How Darktrace Combats Phishing Threats
The initial entry point for Mamba 2FA account compromises primarily involves phishing campaigns using HTML attachments and deceptive links. These phishing attempts are designed to mimic legitimate Microsoft services, such as OneDrive and SharePoint, making them appear authentic to unsuspecting users. Darktrace / EMAIL leverages multiple capabilities to analyze email content for known indicators of phishing. This includes looking for suspicious URLs, unusual attachments (like HTML files with embedded JavaScript), and signs of social engineering tactics commonly used in phishing campaigns like Mamba 2FA. With these capabilities, Darktrace successfully detected Mamba 2FA phishing emails in networks where this tool is integrated into the security layers, consequently preventing further implications and account hijacks of their users.
Mamba 2FA URL Structure and Domain Names
The URL structure used in Mamba 2FA phishing attempts is specifically designed to facilitate the capture of user credentials and MFA tokens while evading detection. These phishing URLs typically follow a pattern that incorporates Base64-encoded parameters, which play a crucial role in the operation of the phishing kit.
The URLs associated with Mamba 2FA phishing pages generally follow this structure [6]:
https://{domain}/{m,n,o}/?{Base64 string}
Below are some potential Mamba 2FA phishing emails, with the Base64 strings already decoded, that were classified as certain threats by Darktrace / EMAIL. This classification was based on identifying multiple suspicious characteristics, such as HTML attachments containing JavaScript code, emails from senders with no previous association with the recipients, analysis of redirect links, among others. These emails were autonomously blocked from being delivered to users' inboxes.
Conclusion
The rise of PhaaS platforms and the advent of AiTM phishing kits represent a concerning evolution in cyber threats, pushing the boundaries of traditional phishing tactics and exposing significant vulnerabilities in current cybersecurity defenses. The ability of these attacks to effortlessly bypass traditional security measures like MFA underscores the need for more sophisticated, adaptive strategies to combat these evolving threats.
By identifying and responding to anomalous activities within Microsoft 365 accounts, Darktrace not only highlights the importance of comprehensive monitoring but also sets a new standard for proactive threat detection. Furthermore, the autonomous threat response capabilities and the exceptional proficiency of Darktrace / EMAIL in intercepting and neutralizing sophisticated phishing attacks illustrate a robust defense mechanism that can effectively safeguard users and maintain the integrity of digital ecosystems.
Credit to Patrick Anjos (Senior Cyber Analyst) and Nahisha Nobregas (Senior Cyber Analyst)
Appendices
Darktrace Model Detections
SaaS / Access / M365 High Risk Level Login
SaaS / Access / Unusual External Source for SaaS Credential Use
SaaS / Compromise / Login From Rare Endpoint While User Is Active
SaaS / Compliance / M365 Security Information Modified
SaaS / Compromise / Unusual Login and New Email Rule
The Gartner Magic Quadrant for Email Security is designed to help organizations evaluate which email security solutions might be the best fit for their needs by providing a visual representation of the market vendors and the strengths and cautions of different vendors. We encourage our customers to read the full report to get the complete picture.
Darktrace / EMAIL has a unique AI approach to identifying threats, including NLP and behavioral analysis, instead of traditional security measures like signatures and sandboxing – providing protection against advanced attacks like Business Email Compromise (BEC) and spear phishing. We believe our AI-first approach delivers high-quality solutions that our customers trust, allowing them to stay ahead of sophisticated threats that other tools miss.
We’re proud of Darktrace’s rapid growth, geographic scale, and ability to execute effectively in the email security market, which reflect our commitment to delivering high-quality, reliable solutions that meet the evolving needs of our customers.
What do we believe makes Darktrace the fastest growing email security solution on the market?
An AI-first approach to innovation: Catching the threats others miss
As one of the founders of the ICES category, Darktrace has a long history of innovation, backed by over 200 patents. While other email security solutions are only just starting to apply machine learning (ML) techniques to outdated methods like signature analysis, reputation lists, and sandboxing, Darktrace has redefined the approach to email threat detection with its pioneering AI-driven anomaly detection engine.
Traditional ESPs often miss advanced threats because they rely on rules and signatures that focus on payloads and blindly trust known sources. This approach requires constant updates and frequently fails to detect threats like Business Email Compromise and Spear Phishing. In contrast, Darktrace / EMAIL uses advanced anomaly detection to identify the most sophisticated threats by focusing on unusual patterns and behaviors. This innovative approach has consistently delivered superior detection, stopping on average 58% of the threats that other solutions in the security stack miss.1
But our AI-first approach doesn’t stop at the inbox. At Darktrace, we transcend the limitations of traditional email security by leveraging a platform that unifies insights across multiple domains, providing robust protection against multi-domain threats. Our award-winning solutions defend the most popular attack vectors, including email, messaging, network, and identity protection. By combining signals from all domains, we establish unique behavioral profiles for each device and user, significantly enhancing detection precision.
This pioneering approach has led to introducing industry-first advancements like QR code analysis and automated incident investigations, alongside game-changing functionality including:
Microsoft Teams security with advanced messaging analysis: The ability to identify critical early phishing and insider threats across both email and Microsoft Teams messaging.
AI analyst narratives for improved end user reporting: that reduces phishing investigations by 60% by exposing unique narratives that provide the context of each received email and give feedback to each employee as they interact with their mail.2
Mailbox Security Assistant: to perform advanced behavioral browser analysis and stop malicious links within webpages, detecting and remediating 70% more malicious phishing links than traditional tools.3
AI based, autonomous data loss prevention: to immediately secure your organization from misdirected emails, insider threats, and data loss—both classified and unclassified- without any administrative overhead.
Customer trust that fuels exponential growth
With almost 5,000 customers in under 5 years, we've doubled the growth rate of other vendors in the email security market. Our rapid market penetration, fueled by customer satisfaction and pioneering technology, showcases our revolutionary approach and sets new industry standards.
Darktrace’s exceptional customer retention is fueled by an unparalleled customer experience, extensive regional support, dedicated account teams, and cutting-edge scalable technology. We pride ourselves on having a global network with local expertise, consisting of 110 worldwide offices which provide local language and technical support to offer multilingual, in-house assistance to our customer base.
Check it out – Darktrace / EMAIL has the highest percentage of 5-star ratings with a 4.8 rating on Gartner® Peer Insights™.4
Supporting every stage of your email security journey
Darktrace / EMAIL supports your security maturity journey, from first time security buyers to mature security stacks looking to augment their existing ESPs – by handling advanced threats without extensive tuning. And unlike other solutions that create a siloed and parallel solution, it works harmoniously with native email providers to create a modern email security stack. That’s why Darktrace performs well with first-time email security buyers and has strong renewal rates.
Integrating with Microsoft and Google via API, we replace traditional Secure Email Gateways (SEGs) with a modern, comprehensive email security stack. By combining approaches, our solution merges attack-centric analysis, which learns attack patterns and threat intelligence, with a business-centric approach that understands user behavior and inbox activity to deliver a unified stack that defends the entire threat spectrum – leading Darktrace to be recognized as Microsoft Partner of the year UK 2024.
Our user-friendly, self-learning AI solution requires minimal tuning and deployment, making it perfect for customers looking for a highly usable but lightly configurable solution that will accompany them throughout their lifetime as they mature their email security stack in line with the evolving threat landscape.
Learn more
Get complimentary access to the full Gartner® Magic Quadrant™ for Email Security Platforms here.
1 From September 1 – December 31 2023, 58% of the phishing emails analyzed by Darktrace / EMAIL had already passed through native spam filtering and email security controls. (Darktrace End of Year Threat Report 2023)
2 When customers deployed the Darktrace / EMAIL Outlook Add-in there was a 60% decrease in incorrectly reported phishing emails. Darktrace Internal Research, 2024
3 Once a user reports phishing that contains a link, an automated second level triage engages our link analysis infrastructure expanding the signals analyzed. Darktrace Internal Research, 2024