The State of AI in Cybersecurity: The Impact of AI on Cybersecurity Solutions
13
May 2024
Part 3: This blog discusses the impact of AI on cybersecurity solutions based on data from Darktrace’s State of AI Cybersecurity Report. Get the latest insights into the evolving challenges faced by organizations, the growing demand for skilled professionals, and the need for integrated security solutions by downloading the full report.
About the AI Cybersecurity Report
Darktrace surveyed 1,800 CISOs, security leaders, administrators, and practitioners from industries around the globe. Our research was conducted to understand how the adoption of new AI-powered offensive and defensive cybersecurity technologies are being managed by organizations.
Overwhelming alert volumes, high false positive rates, and endlessly innovative threat actors keep security teams scrambling. Defenders have been forced to take a reactive approach, struggling to keep pace with an ever-evolving threat landscape. It is hard to find time to address long-term objectives or revamp operational processes when you are always engaged in hand-to-hand combat.
The impact of AI on the threat landscape will soon make yesterday’s approaches untenable. Cybersecurity vendors are racing to capitalize on buyer interest in AI by supplying solutions that promise to meet the need. But not all AI is created equal, and not all these solutions live up to the widespread hype.
Do security professionals believe AI will impact their security operations?
Yes! 95% of cybersecurity professionals agree that AI-powered solutions will level up their organization’s defenses.
Not only is there strong agreement about the ability of AI-powered cybersecurity solutions to improve the speed and efficiency of prevention, detection, response, and recovery, but that agreement is nearly universal, with more than 95% alignment.
This AI-powered future is about much more than generative AI. While generative AI can help accelerate the data retrieval process within threat detection, create quick incident summaries, automate low-level tasks in security operations, and simulate phishing emails and other attack tactics, most of these use cases were ranked lower in their impact to security operations by survey participants.
There are many other types of AI, which can be applied to many other use cases:
Supervised machine learning: Applied more often than any other type of AI in cybersecurity. Trained on attack patterns and historical threat intelligence to recognize known attacks.
Natural language processing (NLP): Applies computational techniques to process and understand human language. It can be used in threat intelligence, incident investigation, and summarization.
Large language models (LLMs): Used in generative AI tools, this type of AI applies deep learning models trained on massively large data sets to understand, summarize, and generate new content. The integrity of the output depends upon the quality of the data on which the AI was trained.
Unsupervised machine learning: Continuously learns from raw, unstructured data to identify deviations that represent true anomalies. With the correct models, this AI can use anomaly-based detections to identify all kinds of cyber-attacks, including entirely unknown and novel ones.
What are the areas of cybersecurity AI will impact the most?
Improving threat detection is the #1 area within cybersecurity where AI is expected to have an impact.
The most frequent response to this question, improving threat detection capabilities in general, was top ranked by slightly more than half (57%) of respondents. This suggests security professionals hope that AI will rapidly analyze enormous numbers of validated threats within huge volumes of fast-flowing events and signals. And that it will ultimately prove a boon to front-line security analysts. They are not wrong.
Identifying exploitable vulnerabilities (mentioned by 50% of respondents) is also important. Strengthening vulnerability management by applying AI to continuously monitor the exposed attack surface for risks and high-impact vulnerabilities can give defenders an edge. If it prevents threats from ever reaching the network, AI will have a major downstream impact on incident prevalence and breach risk.
Where will defensive AI have the greatest impact on cybersecurity?
Cloud security (61%), data security (50%), and network security (46%) are the domains where defensive AI is expected to have the greatest impact.
Respondents selected broader domains over specific technologies. In particular, they chose the areas experiencing a renaissance. Cloud is the future for most organizations, and the effects of cloud adoption on data and networks are intertwined. All three domains are increasingly central to business operations, impacting everything everywhere.
Responses were remarkably consistent across demographics, geographies, and organization sizes, suggesting that nearly all survey participants are thinking about this similarly—that AI will likely have far-reaching applications across the broadest fields, as well as fewer, more specific applications within narrower categories.
Going forward, it will be paramount for organizations to augment their cloud and SaaS security with AI-powered anomaly detection, as threat actors sharpen their focus on these targets.
How will security teams stop AI-powered threats?
Most security stakeholders (71%) are confident that AI-powered security solutions are better able to block AI-powered threats than traditional tools.
There is strong agreement that AI-powered solutions will be better at stopping AI-powered threats (71% of respondents are confident in this), and there’s also agreement (66%) that AI-powered solutions will be able to do so automatically. This implies significant faith in the ability of AI to detect threats both precisely and accurately, and also orchestrate the correct response actions.
There is also a high degree of confidence in the ability of security teams to implement and operate AI-powered solutions, with only 30% of respondents expressing doubt. This bodes well for the acceptance of AI-powered solutions, with stakeholders saying they’re prepared for the shift.
On the one hand, it is positive that cybersecurity stakeholders are beginning to understand the terms of this contest—that is, that only AI can be used to fight AI. On the other hand, there are persistent misunderstandings about what AI is, what it can do, and why choosing the right type of AI is so important. Only when those popular misconceptions have become far less widespread can our industry advance its effectiveness.
Oops! Something went wrong while submitting the form.
Newsletter
Stay ahead of threats with the Darktrace blog newsletter
Get the latest insights from the cybersecurity landscape, including threat trends, incident analysis, and the latest Darktrace product developments – delivered directly to your inbox, monthly.
Thanks, your request has been received
A member of our team will be in touch with you shortly.
Oops! Something went wrong while submitting the form.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
A Snake in the Net: Defending Against AiTM Phishing Threats and Mamba 2FA
What are Adversary-in-the-Middle (AiTM) phishing kits?
Phishing-as-a-Service (PhaaS) platforms have significantly lowered the barriers to entry for cybercriminals, enabling a new wave of sophisticated phishing attacks. Among the most concerning developments in this landscape is the emergence of Adversary-in-the-Middle (AiTM) phishing kits, which enhance traditional phishing tactics by allowing attackers to intercept and manipulate communications in real-time. The PhaaS marketplace offers a wide variety of innovative capabilities, with basic services starting around USD 120 and more advanced services costing around USD 250 monthly [1].
These AiTM kits are designed to create convincing decoy pages that mimic legitimate login interfaces, often pre-filling user information to increase credibility. By acting as a man-in-the-middle, attackers can harvest sensitive data such as usernames, passwords, and even multi-factor authentication (MFA) tokens without raising immediate suspicion. This capability not only makes AiTM attacks more effective but also poses a significant challenge for cybersecurity defenses [2].
Mamba 2FA is one such example of a PhaaS strain with AiTM capabilities that has emerged as a significant threat to users of Microsoft 365 and other enterprise systems. Discovered in May 2024, Mamba 2FA employs advanced AiTM tactics to bypass MFA, making it particularly dangerous for organizations relying on these security measures.
What is Mamba 2FA?
Phishing Mechanism
Mamba 2FA employs highly convincing phishing pages that closely mimic legitimate Microsoft services like OneDrive and SharePoint. These phishing URLs are crafted with a specific structure, incorporating Base64-encoded parameters. This technique allows attackers to tailor the phishing experience to the targeted organization, making the deception more effective. If an invalid parameter is detected, users are redirected to a benign error page, which helps evade automated detection systems [5].
Real-Time Communication
A standout feature of Mamba 2FA is its use of the Socket.IO JavaScript library. This library facilitates real-time communication between the phishing page and the attackers' backend servers. As users input sensitive information, such as usernames, passwords, and MFA tokens on the phishing site, this data is immediately relayed to the attackers, enabling swift unauthorized access [5].
Multi-Factor Authentication Bypass
Mamba 2FA specifically targets MFA methods that are not resistant to phishing, such as one-time passwords (OTPs) and push notifications. When a user enters their MFA token, it is captured in real-time by the attackers, who can then use it to access the victim's account immediately. This capability significantly undermines traditional security measures that rely on MFA for account protection.
Infrastructure and Distribution
The platform's infrastructure consists of two main components: link domains and relay servers. Link domains handle initial phishing attempts, while relay servers are responsible for stealing credentials and completing login processes on behalf of the attacker. The relay servers are designed to mask their IP addresses by using proxy services, making it more difficult for security systems to block them [3].
Evasion Techniques
To evade detection by security tools, Mamba 2FA employs several strategies:
Sandbox Detection: The platform can detect if it is being analyzed in a sandbox environment and will redirect users to harmless pages like Google’s 404 error page.
Dynamic URL Generation: The URLs used in phishing attempts are frequently rotated and often short-lived to avoid being blacklisted by security solutions.
HTML Attachments: Phishing emails often include HTML attachments that appear benign but contain hidden JavaScript that redirects users to the phishing page [5].
Darktrace’s Coverage of Mamba 2FA
Starting in July 2024, the Darktrace Threat Research team detected a sudden rise in Microsoft 365 customer accounts logging in from unusual external sources. These accounts were accessed from an anomalous endpoint, 2607:5500:3000:fea[::]2, and exhibited unusual behaviors upon logging into Software-as-a-Service (SaaS) accounts. This activity strongly correlates with a phishing campaign using Mamba 2FA, first documented in late June 2024 and tracked as Mamba 2FA by Sekoia [2][3].
Darktrace / IDENTITY was able to identify the initial stages of the Mamba 2FA campaign by correlating subtle anomalies, such as unusual SaaS login locations. Using AI based on peer group analysis, it detected unusual behavior associated with these attacks. By leveraging Autonomous Response actions, Darktrace was able to neutralize these threats in every instance of the campaign detected.
On July 23, a SaaS user was observed logging in from a rare ASN and IP address, 2607:5500:3000:fea::2, originating from the US and successfully passed through MFA authentication.
Almost an hour later, the SaaS user was observed logging in from another suspicious IP address, 45.133.172[.]86, linked to ASN AS174 COGENT-174. This IP, originating from the UK, successfully passed through MFA validation.
Following this unusual access, the SaaS user was notably observed reading emails and files that could contain sensitive payment and contract information. This behavior suggests that the attacker may have been leveraging contextual information about the target to craft further malicious phishing emails or fraudulent invoices. Subsequently, the user was detected creating a new mailbox rule titled 'fdsdf'. This rule was configured to redirect emails from a specific domain to the 'Deleted Items' folder and automatically mark them as read.
Implications of Unusual Email Rules
Such unusual email rule configurations are a common tactic employed by attackers. They often use these rules to automatically forward emails containing sensitive keywords—such as "invoice”, "payment", or "confidential"—to an external address. Additionally, these rules help conceal malicious activities, keeping them hidden from the target and allowing the attacker to operate undetected.
Blocking the action
A few minutes later, the SaaS user from the unusual IP address 45.133.172[.]86 was observed attempting to send an email with the subject “RE: Payments.” Subsequently, Darktrace detected the user engaging in activities that could potentially establish persistence in the compromised account, such as registering a new authenticator app. Recognizing this sequence of anomalous behaviors, Darktrace implemented an Autonomous Response inhibitor, disabling the SaaS user for two hours. This action effectively contained potential malicious activities, such as the distribution of phishing emails and fraudulent invoices, and gave the customer’s security team the necessary time to conduct a thorough investigation and implement appropriate security measures.
In another example from mid-July, similar activities related to the campaign were observed on another customer network. A SaaS user was initially detected logging in from the unusual external endpoint 2607:5500:3000:fea[::]2.
A few minutes later, in the same manner as demonstrated in the previous case, the actor was observed logging in from another rare endpoint, 102.68.111[.]240. However, this time it was from a source IP located in Lagos, Nigeria, which no other user on the network had been observed connecting from. Once logged in, the SaaS user updated the settings to "User registered Authenticator App with Notification and Code," a possible attempt to maintain persistence in the SaaS account.
Based on unusual patterns of user behavior, a Cyber AI Analyst Incident was also generated, detailing all potential account hijacking activities. Darktrace also applied an Autonomous Response action, disabling the user for over five hours. This swift action was crucial in preventing further unauthorized access, potential data breaches and further implications.
Since the customer had subscribed to Darktrace Security Operations Centre (SOC) services, Darktrace analysts conducted an additional human investigation confirming the account compromise.
How Darktrace Combats Phishing Threats
The initial entry point for Mamba 2FA account compromises primarily involves phishing campaigns using HTML attachments and deceptive links. These phishing attempts are designed to mimic legitimate Microsoft services, such as OneDrive and SharePoint, making them appear authentic to unsuspecting users. Darktrace / EMAIL leverages multiple capabilities to analyze email content for known indicators of phishing. This includes looking for suspicious URLs, unusual attachments (like HTML files with embedded JavaScript), and signs of social engineering tactics commonly used in phishing campaigns like Mamba 2FA. With these capabilities, Darktrace successfully detected Mamba 2FA phishing emails in networks where this tool is integrated into the security layers, consequently preventing further implications and account hijacks of their users.
Mamba 2FA URL Structure and Domain Names
The URL structure used in Mamba 2FA phishing attempts is specifically designed to facilitate the capture of user credentials and MFA tokens while evading detection. These phishing URLs typically follow a pattern that incorporates Base64-encoded parameters, which play a crucial role in the operation of the phishing kit.
The URLs associated with Mamba 2FA phishing pages generally follow this structure [6]:
https://{domain}/{m,n,o}/?{Base64 string}
Below are some potential Mamba 2FA phishing emails, with the Base64 strings already decoded, that were classified as certain threats by Darktrace / EMAIL. This classification was based on identifying multiple suspicious characteristics, such as HTML attachments containing JavaScript code, emails from senders with no previous association with the recipients, analysis of redirect links, among others. These emails were autonomously blocked from being delivered to users' inboxes.
Conclusion
The rise of PhaaS platforms and the advent of AiTM phishing kits represent a concerning evolution in cyber threats, pushing the boundaries of traditional phishing tactics and exposing significant vulnerabilities in current cybersecurity defenses. The ability of these attacks to effortlessly bypass traditional security measures like MFA underscores the need for more sophisticated, adaptive strategies to combat these evolving threats.
By identifying and responding to anomalous activities within Microsoft 365 accounts, Darktrace not only highlights the importance of comprehensive monitoring but also sets a new standard for proactive threat detection. Furthermore, the autonomous threat response capabilities and the exceptional proficiency of Darktrace / EMAIL in intercepting and neutralizing sophisticated phishing attacks illustrate a robust defense mechanism that can effectively safeguard users and maintain the integrity of digital ecosystems.
Credit to Patrick Anjos (Senior Cyber Analyst) and Nahisha Nobregas (Senior Cyber Analyst)
Appendices
Darktrace Model Detections
SaaS / Access / M365 High Risk Level Login
SaaS / Access / Unusual External Source for SaaS Credential Use
SaaS / Compromise / Login From Rare Endpoint While User Is Active
SaaS / Compliance / M365 Security Information Modified
SaaS / Compromise / Unusual Login and New Email Rule
The Gartner Magic Quadrant for Email Security is designed to help organizations evaluate which email security solutions might be the best fit for their needs by providing a visual representation of the market vendors and the strengths and cautions of different vendors. We encourage our customers to read the full report to get the complete picture.
Darktrace / EMAIL has a unique AI approach to identifying threats, including NLP and behavioral analysis, instead of traditional security measures like signatures and sandboxing – providing protection against advanced attacks like Business Email Compromise (BEC) and spear phishing. We believe our AI-first approach delivers high-quality solutions that our customers trust, allowing them to stay ahead of sophisticated threats that other tools miss.
We’re proud of Darktrace’s rapid growth, geographic scale, and ability to execute effectively in the email security market, which reflect our commitment to delivering high-quality, reliable solutions that meet the evolving needs of our customers.
What do we believe makes Darktrace the fastest growing email security solution on the market?
An AI-first approach to innovation: Catching the threats others miss
As one of the founders of the ICES category, Darktrace has a long history of innovation, backed by over 200 patents. While other email security solutions are only just starting to apply machine learning (ML) techniques to outdated methods like signature analysis, reputation lists, and sandboxing, Darktrace has redefined the approach to email threat detection with its pioneering AI-driven anomaly detection engine.
Traditional ESPs often miss advanced threats because they rely on rules and signatures that focus on payloads and blindly trust known sources. This approach requires constant updates and frequently fails to detect threats like Business Email Compromise and Spear Phishing. In contrast, Darktrace / EMAIL uses advanced anomaly detection to identify the most sophisticated threats by focusing on unusual patterns and behaviors. This innovative approach has consistently delivered superior detection, stopping on average 58% of the threats that other solutions in the security stack miss.1
But our AI-first approach doesn’t stop at the inbox. At Darktrace, we transcend the limitations of traditional email security by leveraging a platform that unifies insights across multiple domains, providing robust protection against multi-domain threats. Our award-winning solutions defend the most popular attack vectors, including email, messaging, network, and identity protection. By combining signals from all domains, we establish unique behavioral profiles for each device and user, significantly enhancing detection precision.
This pioneering approach has led to introducing industry-first advancements like QR code analysis and automated incident investigations, alongside game-changing functionality including:
Microsoft Teams security with advanced messaging analysis: The ability to identify critical early phishing and insider threats across both email and Microsoft Teams messaging.
AI analyst narratives for improved end user reporting: that reduces phishing investigations by 60% by exposing unique narratives that provide the context of each received email and give feedback to each employee as they interact with their mail.2
Mailbox Security Assistant: to perform advanced behavioral browser analysis and stop malicious links within webpages, detecting and remediating 70% more malicious phishing links than traditional tools.3
AI based, autonomous data loss prevention: to immediately secure your organization from misdirected emails, insider threats, and data loss—both classified and unclassified- without any administrative overhead.
Customer trust that fuels exponential growth
With almost 5,000 customers in under 5 years, we've doubled the growth rate of other vendors in the email security market. Our rapid market penetration, fueled by customer satisfaction and pioneering technology, showcases our revolutionary approach and sets new industry standards.
Darktrace’s exceptional customer retention is fueled by an unparalleled customer experience, extensive regional support, dedicated account teams, and cutting-edge scalable technology. We pride ourselves on having a global network with local expertise, consisting of 110 worldwide offices which provide local language and technical support to offer multilingual, in-house assistance to our customer base.
Check it out – Darktrace / EMAIL has the highest percentage of 5-star ratings with a 4.8 rating on Gartner® Peer Insights™.4
Supporting every stage of your email security journey
Darktrace / EMAIL supports your security maturity journey, from first time security buyers to mature security stacks looking to augment their existing ESPs – by handling advanced threats without extensive tuning. And unlike other solutions that create a siloed and parallel solution, it works harmoniously with native email providers to create a modern email security stack. That’s why Darktrace performs well with first-time email security buyers and has strong renewal rates.
Integrating with Microsoft and Google via API, we replace traditional Secure Email Gateways (SEGs) with a modern, comprehensive email security stack. By combining approaches, our solution merges attack-centric analysis, which learns attack patterns and threat intelligence, with a business-centric approach that understands user behavior and inbox activity to deliver a unified stack that defends the entire threat spectrum – leading Darktrace to be recognized as Microsoft Partner of the year UK 2024.
Our user-friendly, self-learning AI solution requires minimal tuning and deployment, making it perfect for customers looking for a highly usable but lightly configurable solution that will accompany them throughout their lifetime as they mature their email security stack in line with the evolving threat landscape.
Learn more
Get complimentary access to the full Gartner® Magic Quadrant™ for Email Security Platforms here.
1 From September 1 – December 31 2023, 58% of the phishing emails analyzed by Darktrace / EMAIL had already passed through native spam filtering and email security controls. (Darktrace End of Year Threat Report 2023)
2 When customers deployed the Darktrace / EMAIL Outlook Add-in there was a 60% decrease in incorrectly reported phishing emails. Darktrace Internal Research, 2024
3 Once a user reports phishing that contains a link, an automated second level triage engages our link analysis infrastructure expanding the signals analyzed. Darktrace Internal Research, 2024