Generative AI tools have increased the risk of BEC, and traditional cybersecurity defenses struggle to stay ahead of the growing speed, scale, and sophistication of attacks. Only multilayered, defense-in-depth strategies can counter the AI-powered BEC threat.
As people continue to be the weak link in most organizations’ cybersecurity practices, the growing use of generative AI tools in cyber-attacks makes email, their primary communications channel, a more compelling target than ever. The risk associated with Business Email Compromise (BEC) in particular continues to rise as generative AI tools equip attackers to build and launch social engineering and phishing campaigns with greater speed, scale, and sophistication.
What is BEC?
BEC is defined in different ways, but generally refers to cyber-attacks in which attackers abuse email — and users’ trust — to trick employees into transferring funds or divulging sensitive company data.
Unlike generic phishing emails, most BEC attacks do not rely on “spray and pray” dissemination or on users’ clicking bogus links or downloading malicious attachments. Instead, modern BEC campaigns use a technique called “pretexting.”
What is pretexting?
Pretexting is a more specific form of phishing that describes an urgent but false situation — the pretext — that requires the transfer of funds or revelation of confidential data.
This type of attack, and therefore BEC, is dominating the email threat landscape. As reported in Verizon’s 2024 Data Breach Investigation Report, recently there has been a “clear overtaking of pretexting as a more likely social action than phishing.” The data shows pretexting, “continues to be the leading cause of cybersecurity incidents (accounting for 73% of breaches)” and one of “the most successful ways of monetizing a breach.”
Pretexting and BEC work so well because they exploit humans’ natural inclination to trust the people and companies they know. AI compounds the risk by making it easier for attackers to mimic known entities and harder for security tools and teams – let alone unsuspecting recipients of routine emails – to tell the difference.
BEC attacks now incorporate AI
With the growing use of AI by threat actors, trends point to BEC gaining momentum as a threat vector and becoming harder to detect. By adding ingenuity, machine speed, and scale, generative AI tools like OpenAI’s ChatGPT give threat actors the ability to create more personalized, targeted, and convincing emails at scale.
Large Language Models (LLMs) like ChatGPT can draft believable messages that feel like emails that target recipients expect to receive. For example, generative AI tools can be used to send fake invoices from vendors known to be involved with well-publicized construction projects. These messages also prove harder to detect as AI automatically:
Avoids misspellings and grammatical errors
Creates multiple variations of email text
Translates messages that read well in multiple languages
And accomplishes additional, more targeted tactics
AI creates a force multiplier that allows primitive mass-mail campaigns to evolve into sophisticated automated attacks. Instead of spending weeks studying the target to craft an effective email, cybercriminals might only spend an hour or two and achieve a better result.
Challenges of detecting AI-powered BEC attacks
Rules-based detections miss unknown attacks
One major challenge comes from the fact that rules based on known attacks have no basis to deny new threats. While native email security tools defend against known attacks, many modern BEC attacks use entirely novel language and can omit payloads altogether. Instead, they rely on pure social engineering or bide their time until security tools recognize the new sender as a legitimate contact.
Most defensive AI can’t keep pace with attacker innovation
Security tools might focus on the meaning of an email’s text in trying to recognize a BEC attack, but defenders still end up in a rules and signature rat race. Some newer Integrated Cloud Email Security (ICES) vendors attempt to use AI defensively to improve the flawed approach of only looking for exact matches. Employing data augmentation to identify similar-looking emails helps to a point but not enough to outpace novel attacks built with generative AI.
What tools can stop BEC?
A modern defense-in-depth strategy must use AI to counter the impact of AI in the hands of attackers. As found in our 2024 State of AI Cybersecurity Report, 96% of survey participants believe AI-driven security solutions are a must have for countering AI-powered threats.
However, not all AI tools are the same. Since BEC attacks continue to change, defensive AI-powered tools should focus less on learning what attacks look like, and more on learning normal behavior for the business. By understanding expected behavior on the company’s side, the security solution will be able to recognize anomalous and therefore suspicious activity, regardless of the word choice or payload type.
To combat the speed and scale of new attacks, an AI-led BEC defense should spot novel threats.
Self-Learning AI builds profiles for every email user, including their relationships, tone and sentiment, content, and link sharing patterns. Rich context helps in understanding how people communicate and identifying deviations from the normal routine to determine what does and does not belong in an individual’s inbox and outbox.
Other email security vendors may claim to use behavioral AI and unsupervised machine learning in their products, but their AI are still pre-trained with historical data or signatures to recognize malicious activity, rather than demonstrating a true learning process. Darktrace’s Self Learning-AI truly learns from the organization in which it is installed, allowing it to detect unknown and novel vectors that other security tools are not yet trained on.
Because Darktrace understands the human behind email communications rather than knowledge of past attacks, Darktrace / EMAIL can stop the most sophisticated and evolving email security risks. It enhances your native email security by leveraging business-centric behavioral anomaly detection across inbound, outbound, and lateral messages in both email and Teams.
This unique approach quickly identifies sophisticated threats like BEC, ransomware, phishing, and supply chain attacks without duplicating existing capabilities or relying on traditional rules, signatures, and payload analysis.
The power of Darktrace’s AI can be seen in its speed and adaptability: Darktrace / EMAIL blocks the most novel threats up to 13 days faster than traditional security tools.
Oops! Something went wrong while submitting the form.
Newsletter
Enjoying the blog?
Sign up to receive the latest news and insights from the Darktrace newsletter – delivered directly to your inbox
Thanks for signing up!
Look out for your first newsletter, coming soon.
Oops! Something went wrong while submitting the form.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Author
Carlos Gray
Product Manager
Carlos Gonzalez Gray is a Product Marketing Manager at Darktrace, based in the Madrid Office. As an email security Subject Matter Expert he collaborates with the global product team to align each product with the company’s ethos and ensures Darktrace are continuously pushing the boundaries of innovation. His prior role at Darktrace was in Sales Engineering, leading the Iberian team and specializing in both the email and OT sectors. Additionally, his prior experience as a consultant to IBEX 35 companies in Spain has made him well-versed in compliance, auditing, and data privacy. Carlos holds an Honors BA in Political Science and a Masters in Cybersecurity from IE University.
Bytesize Security: Insider Threats in Google Workspace
What is an insider threat?
An insider threat is a cyber risk originating from within an organization. These threats can involve actions such as an employee inadvertently clicking on a malicious link (e.g., a phishing email) or an employee with malicious intent conducting data exfiltration for corporate sabotage.
Insiders often exploit their knowledge and access to legitimate corporate tools, presenting a continuous risk to organizations. Defenders must protect their digital estate against threats from both within and outside the organization.
For example, in the summer of 2024, Darktrace / IDENTITY successfully detected a user in a customer environment attempting to steal sensitive data from a trusted Google Workspace service. Despite the use of a legitimate and compliant corporate tool, Darktrace identified anomalies in the user’s behavior that indicated malicious intent.
Attack overview: Insider threat
In June 2024, Darktrace detected unusual activity involving the Software-as-a-Service (SaaS) account of a former employee from a customer organization. This individual, who had recently left the company, was observed downloading a significant amount of data in the form of a “.INDD” file (an Adobe InDesign document typically used to create page layouts [1]) from Google Drive.
While the use of Google Drive and other Google Workspace platforms was not unexpected for this employee, Darktrace identified that the user had logged in from an unfamiliar and suspicious IPv6 address before initiating the download. This anomaly triggered a model alert in Darktrace / IDENTITY, flagging the activity as potentially malicious.
Figure 1: A Model Alert in Darktrace / IDENTITY showing the unusual “.INDD” file being downloaded from Google Workspace.
Following this detection, the customer reached out to Darktrace’s Security Operations Center (SOC) team via the Security Operations Support service for assistance in triaging and investigating the incident further. Darktrace’s SOC team conducted an in-depth investigation, enabling the customer to identify the exact moment of the file download, as well as the contents of the stolen documents. The customer later confirmed that the downloaded files contained sensitive corporate data, including customer details and payment information, likely intended for reuse or sharing with a new employer.
In this particular instance, Darktrace’s Autonomous Response capability was not active, allowing the malicious insider to successfully exfiltrate the files. If Autonomous Response had been enabled, Darktrace would have immediately acted upon detecting the login from an unusual (in this case 100% rare) location by logging out and disabling the SaaS user. This would have provided the customer with the necessary time to review the activity and verify whether the user was authorized to access their SaaS environments.
Conclusion
Insider threats pose a significant challenge for traditional security tools as they involve internal users who are expected to access SaaS platforms. These insiders have preexisting knowledge of the environment, sensitive data, and how to make their activities appear normal, as seen in this case with the use of Google Workspace. This familiarity allows them to avoid having to use more easily detectable intrusion methods like phishing campaigns.
Darktrace’s anomaly detection capabilities, which focus on identifying unusual activity rather than relying on specific rules and signatures, enable it to effectively detect deviations from a user’s expected behavior. For instance, an unusual login from a new location, as in this example, can be flagged even if the subsequent malicious activity appears innocuous due to the use of a trusted application like Google Drive.
Credit to Vivek Rajan (Cyber Analyst) and Ryan Traill (Analyst Content Lead)
Appendices
Darktrace Model Detections
SaaS / Resource::Unusual Download Of Externally Shared Google Workspace File
RansomHub Ransomware: investigación de Darktrace sobre la herramienta más nueva en ShadowSyndicate's Arsenal
What is ShadowSyndicate?
ShadowSyndicate, also known as Infra Storm, is a threat actor reportedly active since July 2022, working with various ransomware groups and affiliates of ransomware programs, such as Quantum, Nokoyawa, and ALPHV. This threat actor employs tools like Cobalt Strike, Sliver, IcedID, and Matanbuchus malware in its attacks. ShadowSyndicate utilizes the same SSH fingerprint (1ca4cbac895fc3bd12417b77fc6ed31d) on many of their servers—85 as of September 2023. At least 52 of these servers have been linked to the Cobalt Strike command and control (C2) framework [1].
What is RansomHub?
First observed following the FBI's takedown of ALPHV/BlackCat in December 2023, RansomHub quickly gained notoriety as a Ransomware-as-a-Service (RaaS) operator. RansomHub capitalized on the law enforcement’s disruption of the LockBit group’s operations in February 2024 to market themselves to potential affiliates who had previously relied on LockBit’s encryptors. RansomHub's success can be largely attributed to their aggressive recruitment on underground forums, leading to the absorption of ex-ALPHV and ex-LockBit affiliates. They were one of the most active ransomware operators in 2024, with approximately 500 victims reported since February, according to their Dedicated Leak Site (DLS) [2].
ShadowSyndicate and RansomHub
External researchers have reported that ShadowSyndicate had as many as seven different ransomware families in their arsenal between July 2022, and September 2023. Now, ShadowSyndicate appears to have added RansomHub’s their formidable stockpile, becoming an affiliate of the RaaS provider [1].
Darktrace’s analysis of ShadowSyndicate across its customer base indicates that the group has been leveraging RansomHub ransomware in multiple attacks in September and October 2024. ShadowSyndicate likely shifted to using RansomHub due to the lucrative rates offered by this RaaS provider, with affiliates receiving up to 90% of the ransom—significantly higher than the general market rate of 70-80% [3].
In many instances where encryption was observed, ransom notes with the naming pattern “README_[a-zA-Z0-9]{6}.txt” were written to affected devices. The content of these ransom notes threatened to release stolen confidential data via RansomHub’s DLS unless a ransom was paid. During these attacks, data exfiltration activity to external endpoints using the SSH protocol was observed. The external endpoints to which the data was transferred were found to coincide with servers previously associated with ShadowSyndicate activity.
Darktrace’s coverage of ShadowSyndicate and RansomHub
Darktrace’s Threat Research team identified high-confidence indicators of compromise (IoCs) linked to the ShadowSyndicate group deploying RansomHub. The investigation revealed four separate incidents impacting Darktrace customers across various sectors, including education, manufacturing, and social services. In the investigated cases, multiple stages of the kill chain were observed, starting with initial internal reconnaissance and leading to eventual file encryption and data exfiltration.
Attack Overview
Internal Reconnaissance
The first observed stage of ShadowSyndicate attacks involved devices making multiple internal connection attempts to other internal devices over key ports, suggesting network scanning and enumeration activity. In this initial phase of the attack, the threat actor gathers critical details and information by scanning the network for open ports that might be potentially exploitable. In cases observed by Darktrace affected devices were typically seen attempting to connect to other internal locations over TCP ports including 22, 445 and 3389.
C2 Communication and Data Exfiltration
In most of the RansomHub cases investigated by Darktrace, unusual connections to endpoints associated with Splashtop, a remote desktop access software, were observed briefly before outbound SSH connections were identified.
Following this, Darktrace detected outbound SSH connections to the external IP address 46.161.27[.]151 using WinSCP, an open-source SSH client for Windows used for secure file transfer. The Cybersecurity and Infrastructure Security Agency (CISA) identified this IP address as malicious and associated it with ShadowSyndicate’s C2 infrastructure [4]. During connections to this IP, multiple gigabytes of data were exfiltrated from customer networks via SSH.
Data exfiltration attempts were consistent across investigated cases; however, the method of egress varied from one attack to another, as one would expect with a RaaS strain being employed by different affiliates. In addition to transfers to ShadowSyndicate’s infrastructure, threat actors were also observed transferring data to the cloud storage and file transfer service, MEGA, via HTTP connections using the ‘rclone’ user agent – a command-line program used to manage files on cloud storage. In another case, data exfiltration activity occurred over port 443, utilizing SSL connections.
Lateral Movement
In investigated incidents, lateral movement activity began shortly after C2 communications were established. In one case, Darktrace identified the unusual use of a new administrative credential which was quickly followed up with multiple suspicious executable file writes to other internal devices on the network.
The filenames for this executable followed the regex naming convention “[a-zA-Z]{6}.exe”, with two observed examples being “bWqQUx.exe” and “sdtMfs.exe”.
Figure 1: Cyber AI Analyst Investigation Process for the SMB Writes of Suspicious Files to Multiple Devices' incident.
Additionally, script files such as “Defeat-Defender2.bat”, “Share.bat”, and “def.bat” were also seen written over SMB, suggesting that threat actors were trying to evade network defenses and detection by antivirus software like Microsoft Defender.
File Encryption
Among the three cases where file encryption activity was observed, file names were changed by adding an extension following the regex format “.[a-zA-Z0-9]{6}”. Ransom notes with a similar naming convention, “README_[a-zA-Z0-9]{6}.txt”, were written to each share. While the content of the ransom notes differed slightly in each case, most contained similar text. Clear indicators in the body of the ransom notes pointed to the use of RansomHub ransomware in these attacks. As is increasingly the case, threat actors employed double extortion tactics, threatening to leak confidential data if the ransom was not paid. Like most ransomware, RansomHub included TOR site links for communication between its "customer service team" and the target.
Figure 2: The graph shows the behavior of a device with encryption activity, using the “SMB Sustained Mimetype Conversion” and “Unusual Activity Events” metrics over three weeks.
Since Darktrace’s Autonomous Response capability was not enabled during the compromise, the ransomware attack succeeded in its objective. However, Darktrace’s Cyber AI Analyst provided comprehensive coverage of the kill chain, enabling the customer to quickly identify affected devices and initiate remediation.
Figure 3: Cyber AI Analyst panel showing the critical incidents of the affected device from one of the cases investigated.
In lieu of Autonomous Response being active on the networks, Darktrace was able to suggest a variety of manual response actions intended to contain the compromise and prevent further malicious activity. Had Autonomous Response been enabled at the time of the attack, these actions would have been quickly applied without any human interaction, potentially halting the ransomware attack earlier in the kill chain.
Figure 4: A list of suggested Autonomous Response actions on the affected devices."
Conclusion
The Darktrace Threat Research team has noted a surge in attacks by the ShadowSyndicate group using RansomHub’s RaaS of late. RaaS has become increasingly popular across the threat landscape due to its ease of access to malware and script execution. As more individual threat actors adopt RaaS, security teams are struggling to defend against the increasing number of opportunistic attacks.
For customers subscribed to Darktrace’s Security Operations Center (SOC) services, the Analyst team promptly investigated detections of the aforementioned unusual and anomalous activities in the initial infection phases. Multiple alerts were raised via Darktrace’s Managed Threat Detection to warn customers of active ransomware incidents. By emphasizing anomaly-based detection and response, Darktrace can effectively identify devices affected by ransomware and take action against emerging activity, minimizing disruption and impact on customer networks.
Credit to Kwa Qing Hong (Senior Cyber Analyst and Deputy Analyst Team Lead, Singapore) and Signe Zahark (Principal Cyber Analyst, Japan)
