Generative AI tools have increased the risk of BEC, and traditional cybersecurity defenses struggle to stay ahead of the growing speed, scale, and sophistication of attacks. Only multilayered, defense-in-depth strategies can counter the AI-powered BEC threat.
As people continue to be the weak link in most organizations’ cybersecurity practices, the growing use of generative AI tools in cyber-attacks makes email, their primary communications channel, a more compelling target than ever. The risk associated with Business Email Compromise (BEC) in particular continues to rise as generative AI tools equip attackers to build and launch social engineering and phishing campaigns with greater speed, scale, and sophistication.
What is BEC?
BEC is defined in different ways, but generally refers to cyber-attacks in which attackers abuse email — and users’ trust — to trick employees into transferring funds or divulging sensitive company data.
Unlike generic phishing emails, most BEC attacks do not rely on “spray and pray” dissemination or on users’ clicking bogus links or downloading malicious attachments. Instead, modern BEC campaigns use a technique called “pretexting.”
What is pretexting?
Pretexting is a more specific form of phishing that describes an urgent but false situation — the pretext — that requires the transfer of funds or revelation of confidential data.
This type of attack, and therefore BEC, is dominating the email threat landscape. As reported in Verizon’s 2024 Data Breach Investigation Report, recently there has been a “clear overtaking of pretexting as a more likely social action than phishing.” The data shows pretexting, “continues to be the leading cause of cybersecurity incidents (accounting for 73% of breaches)” and one of “the most successful ways of monetizing a breach.”
Pretexting and BEC work so well because they exploit humans’ natural inclination to trust the people and companies they know. AI compounds the risk by making it easier for attackers to mimic known entities and harder for security tools and teams – let alone unsuspecting recipients of routine emails – to tell the difference.
BEC attacks now incorporate AI
With the growing use of AI by threat actors, trends point to BEC gaining momentum as a threat vector and becoming harder to detect. By adding ingenuity, machine speed, and scale, generative AI tools like OpenAI’s ChatGPT give threat actors the ability to create more personalized, targeted, and convincing emails at scale.
Large Language Models (LLMs) like ChatGPT can draft believable messages that feel like emails that target recipients expect to receive. For example, generative AI tools can be used to send fake invoices from vendors known to be involved with well-publicized construction projects. These messages also prove harder to detect as AI automatically:
Avoids misspellings and grammatical errors
Creates multiple variations of email text
Translates messages that read well in multiple languages
And accomplishes additional, more targeted tactics
AI creates a force multiplier that allows primitive mass-mail campaigns to evolve into sophisticated automated attacks. Instead of spending weeks studying the target to craft an effective email, cybercriminals might only spend an hour or two and achieve a better result.
Challenges of detecting AI-powered BEC attacks
Rules-based detections miss unknown attacks
One major challenge comes from the fact that rules based on known attacks have no basis to deny new threats. While native email security tools defend against known attacks, many modern BEC attacks use entirely novel language and can omit payloads altogether. Instead, they rely on pure social engineering or bide their time until security tools recognize the new sender as a legitimate contact.
Most defensive AI can’t keep pace with attacker innovation
Security tools might focus on the meaning of an email’s text in trying to recognize a BEC attack, but defenders still end up in a rules and signature rat race. Some newer Integrated Cloud Email Security (ICES) vendors attempt to use AI defensively to improve the flawed approach of only looking for exact matches. Employing data augmentation to identify similar-looking emails helps to a point but not enough to outpace novel attacks built with generative AI.
What tools can stop BEC?
A modern defense-in-depth strategy must use AI to counter the impact of AI in the hands of attackers. As found in our 2024 State of AI Cybersecurity Report, 96% of survey participants believe AI-driven security solutions are a must have for countering AI-powered threats.
However, not all AI tools are the same. Since BEC attacks continue to change, defensive AI-powered tools should focus less on learning what attacks look like, and more on learning normal behavior for the business. By understanding expected behavior on the company’s side, the security solution will be able to recognize anomalous and therefore suspicious activity, regardless of the word choice or payload type.
To combat the speed and scale of new attacks, an AI-led BEC defense should spot novel threats.
Self-Learning AI builds profiles for every email user, including their relationships, tone and sentiment, content, and link sharing patterns. Rich context helps in understanding how people communicate and identifying deviations from the normal routine to determine what does and does not belong in an individual’s inbox and outbox.
Other email security vendors may claim to use behavioral AI and unsupervised machine learning in their products, but their AI are still pre-trained with historical data or signatures to recognize malicious activity, rather than demonstrating a true learning process. Darktrace’s Self Learning-AI truly learns from the organization in which it is installed, allowing it to detect unknown and novel vectors that other security tools are not yet trained on.
Because Darktrace understands the human behind email communications rather than knowledge of past attacks, Darktrace / EMAIL can stop the most sophisticated and evolving email security risks. It enhances your native email security by leveraging business-centric behavioral anomaly detection across inbound, outbound, and lateral messages in both email and Teams.
This unique approach quickly identifies sophisticated threats like BEC, ransomware, phishing, and supply chain attacks without duplicating existing capabilities or relying on traditional rules, signatures, and payload analysis.
The power of Darktrace’s AI can be seen in its speed and adaptability: Darktrace / EMAIL blocks the most novel threats up to 13 days faster than traditional security tools.
Oops! Something went wrong while submitting the form.
Newsletter
Stay ahead of threats with the Darktrace blog newsletter
Get the latest insights from the cybersecurity landscape, including threat trends, incident analysis, and the latest Darktrace product developments – delivered directly to your inbox, monthly.
Thanks, your request has been received
A member of our team will be in touch with you shortly.
Oops! Something went wrong while submitting the form.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Author
Carlos Gray
Product Manager
Carlos Gonzalez Gray is a Product Marketing Manager at Darktrace, based in the Madrid Office. As an email security Subject Matter Expert he collaborates with the global product team to align each product with the company’s ethos and ensures Darktrace are continuously pushing the boundaries of innovation. His prior role at Darktrace was in Sales Engineering, leading the Iberian team and specializing in both the email and OT sectors. Additionally, his prior experience as a consultant to IBEX 35 companies in Spain has made him well-versed in compliance, auditing, and data privacy. Carlos holds an Honors BA in Political Science and a Masters in Cybersecurity from IE University.
Navigating buying and adoption journeys for AI cybersecurity tools
Enterprise AI tools go mainstream
In this dawning Age of AI, CISOs are increasingly exploring investments in AI security tools to enhance their organizations’ capabilities. AI can help achieve productivity gains by saving time and resources, mining intelligence and insights from valuable data, and increasing knowledge sharing and collaboration.
While investing in AI can bring immense benefits to your organization, first-time buyers of AI cybersecurity solutions may not know where to start. They will have to determine the type of tool they want, know the options available, and evaluate vendors. Research and understanding are critical to ensure purchases are worth the investment.
Challenges of a muddied marketplace
Key challenges in AI purchasing come from consumer doubt and lack of vendor transparency. The AI software market is buzzing with hype and flashy promises, which are not necessarily going to be realized immediately. This has fostered uncertainty among potential buyers, especially in the AI cybersecurity space.
As Gartner writes, “There is a general lack of transparency and understanding about how AI-enhanced security solutions leverage AI and the effectiveness of those solutions within real-world SecOps. This leads to trust issues among security leaders and practitioners, resulting in slower adoption of AI features” [1].
Given this widespread uncertainty generated through vague hype, buyers must take extra care when considering new AI tools to adopt.
Goals of AI adoption
Buyers should always start their journeys with objectives in mind, and a universal goal is to achieve return on investment. When organizations adopt AI, there are key aspects that will signal strong payoff. These include:
Wide-ranging application across operations and areas of the business
Actual, enthusiastic adoption and application by the human security team
Integration with the rest of the security stack and existing workflows
Business and operational benefits, including but not limited to:
Reduced risk
Reduced time to response
Reduced potential downtime, damage, and disruption
Increased visibility and coverage
Improved SecOps workflows
Decreased burden on teams so they can take on more strategic tasks
Ideally, most or all these measurements will be fulfilled. It is not enough for AI tools to benefit productivity and workflows in theory, but they must be practically implemented to provide return on investment.
Investigation before investment
Before investing in AI tools, buyers should ask questions pertaining to each stage of the adoption journey. The answers to these questions will not only help buyers gauge if a tool could be worth the investment, but also plan how the new tool will practically fit into the organization’s existing technology and workflows.
Figure 1: Initial questions to consider when starting to shop for AI [2].
These questions are good to imagine how a tool will fit into your organization and determine if a vendor is worth further evaluation. Once you decide a tool has potential use and feasibility in your organization, it is time to dive deeper and learn more.
Ask vendors specific questions about their technology. This information will most likely not be on their websites, and since it involves intellectual property, it may require an NDA.
Find a longer list of questions to ask vendors and what to look for in their responses in the white paper “CISO’s Guide to Buying AI.”
Committing to transparency amidst the AI hype
For security teams to make the most out of new AI tools, they must trust the AI. Especially in an AI marketplace full of hype and obfuscation, transparency should be baked into both the descriptions of the AI tool and the tool’s functionality itself. With that in mind, here are some specifics about what techniques make up Darktrace’s AI.
Darktrace as an AI cybersecurity vendor
Darktrace has been using AI technology in cybersecurity for over 10 years. As a pioneer in the space, we have made innovation part of our process.
The Darktrace ActiveAI Security Platform™ uses multi-layered AI that trains on your unique business operations data for tailored security across the enterprise. This approach ensures that the strengths of one AI technique make up for the shortcomings of another, providing well-rounded and reliable coverage. Our models are always on and always learning, allowing your team to stop attacks in real time.
The machine learning techniques used in our solution include:
Unsupervised machine learning
Multiple Clustering Techniques
Multiple anomaly detection models in tandem analyzing data across hundreds of metrics
Bayesian probabilistic methods
Bayesian metaclassifier for autonomous fine-tuning of unsupervised machine learning models
Deep learning engines
Graph theory
Applied supervised machine learning for investigative AI
Neural networks
Reinforcement Learning
Generative and applied AI
Natural Language Processing (NLP) and Large Language Models (LLMs)
Post-processing models
Additionally, since Darktrace focuses on using the customer’s data across its entire digital estate, it brings a range of advantages in data privacy, interpretability, and data transfer costs.
Building trust with Darktrace AI
Darktrace further supports the human security team’s adoption of our technology by building trust. To do that, we designed our platform to give your team visibility and control over the AI.
Instead of functioning as a black box, our products focus on interpretability and sharing confidence levels. This includes specifying the threshold of what triggered a certain alert and the details of the AI Analyst’s investigations to see how it reached its conclusions. The interpretability of our AI uplevels and upskills the human security team with more information to drive investigations and remediation actions.
For complete control, the human security team can modify all the detection and response thresholds for our model alerts to customize them to fit specific business preferences.
Conclusion
CISO’s are increasingly considering investing in AI cybersecurity tools, but in this rapidly growing field, it’s not always clear what to look for.
Buyers should first determine their goals for a new AI tool, then research possible vendors by reviewing validation and asking deeper questions. This will reveal if a tool is a good match for the organization to move forward with investment and adoption.
As leaders in the AI cybersecurity industry, Darktrace is always ready to help you on your AI journey.
Learn more about the most common types of machine learning in cybersecurity in the white paper “CISO’s Guide to Buying AI.”
References
Gartner, April 17, 2024, “Emerging Tech: Navigating the Impact of AI on SecOps Solution Development.”
Triaging Triada: Understanding an Advanced Mobile Trojan and How it Targets Communication and Banking Applications
The rise of android malware
Recently, there has been a significant increase in malware strains targeting mobile devices, with a growing number of Android-based malware families, such as banking trojans, which aim to steal sensitive banking information from organizations and individuals worldwide.
These malware families attempt to access users’ accounts to steal online banking credentials and cookies, bypass multi-factor authentication (MFA), and conduct automatic transactions to steal funds [1]. They often masquerade as legitimate software or communications from social media platforms to compromise devices. Once installed, they use tactics such as keylogging, dumping cached credentials, and searching the file system for stored passwords to steal credentials, take over accounts, and potentially perform identity theft [1].
One recent example is the Antidot Trojan, which infects devices by disguising itself as an update page for Google Play. It establishes a command-and-control (C2) channel with a server, allowing malicious actors to execute commands and collect sensitive data [2].
Despite these malware’s ability to evade detection by standard security software, for example, by changing their code [3], Darktrace recently detected another Android malware family, Triada, communicating with a C2 server and exfiltrating data.
Triada: Background and tactics
First surfacing in 2016, Triada is a modular mobile trojan known to target banking and financial applications, as well as popular communication applications like WhatsApp, Facebook, and Google Mail [4]. It has been deployed as a backdoor on devices such as CTV boxes, smartphones, and tablets during the supply chain process [5]. Triada can also be delivered via drive-by downloads, phishing campaigns, smaller trojans like Leech, Ztorg, and Gopro, or more recently, as a malicious module in applications such as unofficial versions of WhatsApp, YoWhatsApp, and FM WhatsApp [6] [7].
How does Triada work?
Once downloaded onto a user’s device, Triada collects information about the system, such as the device’s model, OS version, SD card space, and list of installed applications, and sends this information to a C2 server. The server then responds with a configuration file containing the device’s personal identification number and settings, including the list of modules to be installed.
After a device has been successfully infected by Triada, malicious actors can monitor and intercept incoming and outgoing texts (including two-factor authentication messages), steal login credentials and credit card information from financial applications, divert in-application purchases to themselves, create fake messaging and email accounts, install additional malicious applications, infect devices with ransomware, and take control of the camera and microphone [4] [7].
For devices infected by unofficial versions of WhatsApp, which are downloaded from third-party app stores [9] and from mobile applications such as Snaptube and Vidmate , Triada collects unique device identifiers, information, and keys required for legitimate WhatsApp to work and sends them to a remote server to register the device [7] [12]. The server then responds by sending a link to the Triada payload, which is downloaded and launched. This payload will also download additional malicious modules, sign into WhatsApp accounts on the target’s phone, and request the same permissions as the legitimate WhatsApp application, such as access to SMS messages. If granted, a malicious actor can sign the user up for paid subscriptions without their knowledge. Triada then collects information about the user’s device and mobile operator and sends it to the C2 server [9] [12].
How does Triada avoid detection?
Triada evades detection by modifying the Zygote process, which serves as a template for every application in the Android OS. This enables the malware to become part of every application launched on a device [3]. It also substitutes system functions and conceals modules from the list of running processes and installed apps, ensuring that the system does not raise the alarm [3]. Additionally, as Triada connects to a C2 server on the first boot, infected devices remain compromised even after a factory reset [4].
Triada attack overview
Across multiple customer deployments, devices were observed making a large number of connections to a range of hostnames, primarily over encrypted SSL and HTTPS protocols. These hostnames had never previously been observed on the customers’ networks and appear to be algorithmically generated. Examples include “68u91.66foh90o[.]com”, “92n7au[.]uhabq9[.]com”, “9yrh7.mea5ms[.]com”, and “is5jg.3zweuj[.]com”.
Figure 1: External Sites Summary Graph showing the rarity of the hostname “92n7au[.]uhabq9[.]com” on a customer network.
Most of the IP addresses associated with these hostnames belong to an ASN associated with the cloud provider Alibaba (i.e., AS45102 Alibaba US Technology Co., Ltd). These connections were made over a range of high number ports over 1000, most commonly over 30000 such as 32091, which Darktrace recognized as extremely unusual for the SSL and HTTPS protocols.
Figure 2: Screenshot of a Model Alert Event log showing a device connecting to the endpoint “is5jg[.]3zweuj[.]com” over port 32091.
On several customer deployments, devices were seen exfiltrating data to hostnames which also appeared to be algorithmically generated. This occurred via HTTP POST requests containing unusual URI strings that were made without a prior GET request, indicating that the infected device was using a hardcoded list of C2 servers.
Figure 3: Screenshot of a Model Alert Event Log showing the device posting the string “i8xps1” to the hostname “72zf6.rxqfd[.]com.
Figure 4: Screenshot of a Model Alert Event Log showing the device posting the string “sqyjyadwwq” to the hostname “9yrh7.mea5ms[.]com”.
These connections correspond with reports that devices affected by Triada communicate with the C2 server to transmit their information and receive instructions for installing the payload.
A number of these endpoints have communicating files associated with the unofficial WhatsApp versions YoWhatsApp and FM WhatsApp [11] [12] [13] . This could indicate that the devices connecting to these endpoints were infected via malicious modules in the unofficial versions of WhatsApp, as reported by open-source intelligence (OSINT) [10] [12]. It could also mean that the infected devices are using these connections to download additional files from the C2 server, which could infect systems with additional malicious modules related to Triada.
Moreover, on certain customer deployments, shortly before or after connecting to algorithmically generated hostnames with communicating files linked to YoWhatsApp and FM WhatsApp, devices were also seen connecting to multiple endpoints associated with WhatsApp and Facebook.
Figure 5: Screenshot from a device’s event log showing connections to endpoints associated with WhatsApp shortly after it connected to “9yrh7.mea5ms[.]com”.
These surrounding connections indicate that Triada is attempting to sign in to the users’ WhatsApp accounts on their mobile devices to request permissions such as access to text messages. Additionally, Triada sends information about users’ devices and mobile operators to the C2 server.
The connections made to the algorithmically generated hostnames over SSL and HTTPS protocols, along with the HTTP POST requests, triggered multiple Darktrace models to alert. These models include those that detect connections to potentially algorithmically generated hostnames, connections over ports that are highly unusual for the protocol used, unusual connectivity over the SSL protocol, and HTTP POSTs to endpoints that Darktrace has determined to be rare for the network.
Conclusion
Recently, the use of Android-based malware families, aimed at stealing banking and login credentials, has become a popular trend among threat actors. They use this information to perform identity theft and steal funds from victims worldwide.
Across affected customers, multiple devices were observed connecting to a range of likely algorithmically generated hostnames over SSL and HTTPS protocols. These devices were also seen sending data out of the network to various hostnames via HTTP POST requests without first making a GET request. The URIs in these requests appeared to be algorithmically generated, suggesting the exfiltration of sensitive network data to multiple Triada C2 servers.
This activity highlights the sophisticated methods used by malware like Triada to evade detection and exfiltrate data. It underscores the importance of advanced security measures and anomaly-based detection systems to identify and mitigate such mobile threats, protecting sensitive information and maintaining network integrity.
Credit to: Justin Torres (Senior Cyber Security Analyst) and Charlotte Thompson (Cyber Security Analyst).
Appendices
Darktrace Model Detections
Model Alert Coverage
Anomalous Connection / Application Protocol on Uncommon Port
Anomalous Connection / Multiple Connections to New External TCP Port
Anomalous Connection / Multiple HTTP POSTS to Rare Hostname
Anomalous Connections / Multiple Failed Connections to Rare Endpoint
Anomalous Connection / Suspicious Expired SSL
Compromise / DGA Beacon
Compromise / Domain Fluxing
Compromise / Fast Beaconing to DGA
Compromise / Sustained SSL or HTTP Increase
Compromise / Unusual Connections to Rare Lets Encrypt
Unusual Activity / Unusual External Activity
AI Analyst Incident Coverage
Unusual Repeated Connections to Multiple Endpoints
![External Sites Summary Graph showing the rarity of the hostname “92n7au[.]uhabq9[.]com” on a customer network.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df74_670d956f4dda5eb6ac9e650f_Screenshot%25202024-10-14%2520at%25203.04.22%25E2%2580%25AFPM.png)
![Screenshot of a Model Alert Event log showing a device connecting to the endpoint “is5jg[.]3zweuj[.]com” over port 32091.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df7d_670d959854eb594420fdc971_Screenshot%25202024-10-14%2520at%25203.05.03%25E2%2580%25AFPM.png)
![Screenshot of a Model Alert Event Log showing the device posting the string “i8xps1” to the hostname “72zf6.rxqfd[.]com.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df80_670d95c562c6e392c7dc3ba8_Screenshot%25202024-10-14%2520at%25203.05.49%25E2%2580%25AFPM.png)
![Screenshot of a Model Alert Event Log showing the device posting the string “sqyjyadwwq” to the hostname “9yrh7.mea5ms[.]com”.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df7a_670d95e4b1dbad771717553c_Screenshot%25202024-10-14%2520at%25203.06.20%25E2%2580%25AFPM.png)
