The State of AI in Cybersecurity: How AI will impact the cyber threat landscape in 2024
22
Apr 2024
Part 2: This blog discusses the impact of AI on the cyber threat landscape based on data from Darktrace’s State of AI Cybersecurity Report. Get the latest insights into the evolving challenges faced by organizations, the growing demand for skilled professionals, and the need for integrated security solutions.
About the AI Cybersecurity Report
We surveyed 1,800 CISOs, security leaders, administrators, and practitioners from industries around the globe. Our research was conducted to understand how the adoption of new AI-powered offensive and defensive cybersecurity technologies are being managed by organizations.
Are organizations feeling the impact of AI-powered cyber threats?
Nearly three-quarters (74%) state AI-powered threats are now a significant issue. Almost nine in ten (89%) agree that AI-powered threats will remain a major challenge into the foreseeable future, not just for the next one to two years.
However, only a slight majority (56%) thought AI-powered threats were a separate issue from traditional/non AI-powered threats. This could be the case because there are few, if any, reliable methods to determine whether an attack is AI-powered.
Identifying exactly when and where AI is being applied may not ever be possible. However, it is possible for AI to affect every stage of the attack lifecycle. As such, defenders will likely need to focus on preparing for a world where threats are unique and are coming faster than ever before.
Are security stakeholders concerned about AI’s impact on cyber threats and risks?
The results from our survey showed that security practitioners are concerned that AI will impact organizations in a variety of ways. There was equal concern associated across the board – from volume and sophistication of malware to internal risks like leakage of proprietary information from employees using generative AI tools.
What this tells us is that defenders need to prepare for a greater volume of sophisticated attacks and balance this with a focus on cyber hygiene to manage internal risks.
One example of a growing internal risks is shadow AI. It takes little effort for employees to adopt publicly-available text-based generative AI systems to increase their productivity. This opens the door to “shadow AI”, which is the use of popular AI tools without organizational approval or oversight. Resulting security risks such as inadvertent exposure of sensitive information or intellectual property are an ever-growing concern.
Are organizations taking strides to reduce risks associated with adoption of AI in their application and computing environment?
71.2% of survey participants say their organization has taken steps specifically to reduce the risk of using AI within its application and computing environment.
16.3% of survey participants claim their organization has not taken these steps.
These findings are good news. Even as enterprises compete to get as much value from AI as they can, as quickly as possible, they’re tempering their eager embrace of new tools with sensible caution.
Still, responses varied across roles. Security analysts, operators, administrators, and incident responders are less likely to have said their organizations had taken AI risk mitigation steps than respondents in other roles. In fact, 79% of executives said steps had been taken, and only 54% of respondents in hands-on roles agreed. It seems that leaders believe their organizations are taking the needed steps, but practitioners are seeing a gap.
Do security professionals feel confident in their preparedness for the next generation of threats?
A majority of respondents (six out of every ten) believe their organizations are inadequately prepared to face the next generation of AI-powered threats.
The survey findings reveal contrasting perceptions of organizational preparedness for cybersecurity threats across different regions and job roles. Security administrators, due to their hands-on experience, express the highest level of skepticism, with 72% feeling their organizations are inadequately prepared. Notably, respondents in mid-sized organizations feel the least prepared, while those in the largest companies feel the most prepared.
Regionally, participants in Asia-Pacific are most likely to believe their organizations are unprepared, while those in Latin America feel the most prepared. This aligns with the observation that Asia-Pacific has been the most impacted region by cybersecurity threats in recent years, according to the IBM X-Force Threat Intelligence Index.
The optimism among Latin American respondents could be attributed to lower threat volumes experienced in the region, but it's cautioned that this could change suddenly (1).
What are biggest barriers to defending against AI-powered threats?
The top-ranked inhibitors center on knowledge and personnel. However, issues are alluded to almost equally across the board including concerns around budget, tool integration, lack of attention to AI-powered threats, and poor cyber hygiene.
The cybersecurity industry is facing a significant shortage of skilled professionals, with a global deficit of approximately 4 million experts (2). As organizations struggle to manage their security tools and alerts, the challenge intensifies with the increasing adoption of AI by attackers. This shift has altered the demands on security teams, requiring practitioners to possess broad and deep knowledge across rapidly evolving solution stacks.
Educating end users about AI-driven defenses becomes paramount as organizations grapple with the shortage of professionals proficient in managing AI-powered security tools. Operationalizing machine learning models for effectiveness and accuracy emerges as a crucial skill set in high demand. However, our survey highlights a concerning lack of understanding among cybersecurity professionals regarding AI-driven threats and the use of AI-driven countermeasures indicating a gap in keeping pace with evolving attacker tactics.
The integration of security solutions remains a notable problem, hindering effective defense strategies. While budget constraints are not a primary inhibitor, organizations must prioritize addressing these challenges to bolster their cybersecurity posture. It's imperative for stakeholders to recognize the importance of investing in skilled professionals and integrated security solutions to mitigate emerging threats effectively.
1. IBM, X-Force Threat Intelligence Index 2024, Available at: https://www.ibm.com/downloads/cas/L0GKXDWJ
2. ISC2, Cybersecurity Workforce Study 2023, Available at: https://media.isc2.org/-/media/Project/ISC2/Main/Media/ documents/research/ISC2_Cybersecurity_Workforce_Study_2023.pdf?rev=28b46de71ce24e6ab7705f6e3da8637e
Like this and want more?
Receive the latest blog in your inbox
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Newsletter
Enjoying the blog?
Sign up to receive the latest news and insights from the Darktrace newsletter – delivered directly to your inbox
Thanks for signing up!
Look out for your first newsletter, coming soon.
Oops! Something went wrong while submitting the form.
Inside the SOC
Darktrace cyber analysts are world-class experts in threat intelligence, threat hunting and incident response, and provide 24/7 SOC support to thousands of Darktrace customers around the globe. Inside the SOC is exclusively authored by these experts, providing analysis of cyber incidents and threat trends, based on real-world experience in the field.
Reimagining Your SOC: How to Achieve Proactive Network Security
Introduction: Challenges and solutions to SOC efficiency
For Security Operation Centers (SOCs), reliance on signature or rule-based tools – solutions that are always chasing the latest update to prevent only what is already known – creates an excess of false positives. SOC analysts are therefore overwhelmed by a high volume of context-lacking alerts, with human analysts able to address only about 10% due to time and resource constraints. This forces many teams to accept the risks of addressing only a fraction of the alerts while novel threats go completely missed.
74% of practitioners are already grappling with the impact of an AI-powered threat landscape, which amplifies challenges like tool sprawl, alert fatigue, and burnout. Thus, achieving a resilient network, where SOC teams can spend most of their time getting proactive and stopping threats before they occur, feels like an unrealistic goal as attacks are growing more frequent.
Despite advancements in security technology (advanced detection systems with AI, XDR tools, SIEM aggregators, etc...), practitioners are still facing the same issues of inefficiency in their SOC, stopping them from becoming proactive. How can they select security solutions that help them achieve a proactive state without dedicating more human hours and resources to managing and triaging alerts, tuning rules, investigating false positives, and creating reports?
To overcome these obstacles, organizations must leverage security technology that is able to augment and support their teams. This can happen in the following ways:
Full visibility across the modern network expanding into hybrid environments
Have tools that identifies and stops novel threats autonomously, without causing downtime
Apply AI-led analysis to reduce time spent on manual triage and investigation
Your current solutions might be holding you back
Traditional cybersecurity point solutions are reliant on using global threat intelligence to pattern match, determine signatures, and consequently are chasing the latest update to prevent only what is known. This means that unknown threats will evade detection until a patient zero is identified. This legacy approach to threat detection means that at least one organization needs to be ‘patient zero’, or the first victim of a novel attack before it is formally identified.
Even the point solutions that claim to use AI to enhance threat detection rely on a combination of supervised machine learning, deep learning, and transformers to
train and inform their systems. This entails shipping your company’s data out to a large data lake housed somewhere in the cloud where it gets blended with attack data from thousands of other organizations. The resulting homogenized dataset gets used to train AI systems — yours and everyone else’s — to recognize patterns of attack based on previously encountered threats.
While using AI in this way reduces the workload of security teams who would traditionally input this data by hand, it emanates the same risk – namely, that AI systems trained on known threats cannot deal with the threats of tomorrow. Ultimately, it is the unknown threats that bring down an organization.
The promise and pitfalls of XDR in today's threat landscape
Enter Extended Detection and Response (XDR): a platform approach aimed at unifying threat detection across the digital environment. XDR was developed to address the limitations of traditional, fragmented tools by stitching together data across domains, providing SOC teams with a more cohesive, enterprise-wide view of threats. This unified approach allows for improved detection of suspicious activities that might otherwise be missed in siloed systems.
However, XDR solutions still face key challenges: they often depend heavily on human validation, which can aggravate the already alarmingly high alert fatigue security analysts experience, and they remain largely reactive, focusing on detecting and responding to threats rather than helping prevent them. Additionally, XDR frequently lacks full domain coverage, relying on EDR as a foundation and are insufficient in providing native NDR capabilities and visibility, leaving critical gaps that attackers can exploit. This is reflected in the current security market, with 57% of organizations reporting that they plan to integrate network security products into their current XDR toolset[1].
Why settling is risky and how to unlock SOC efficiency
The result of these shortcomings within the security solutions market is an acceptance of inevitable risk. From false positives driving the barrage of alerts, to the siloed tooling that requires manual integration, and the lack of multi-domain visibility requiring human intervention for business context, security teams have accepted that not all alerts can be triaged or investigated.
While prioritization and processes have improved, the SOC is operating under a model that is overrun with alerts that lack context, meaning that not all of them can be investigated because there is simply too much for humans to parse through. Thus, teams accept the risk of leaving many alerts uninvestigated, rather than finding a solution to eliminate that risk altogether.
Darktrace / NETWORK is designed for your Security Operations Center to eliminate alert triage with AI-led investigations , and rapidly detect and respond to known and unknown threats. This includes the ability to scale into other environments in your infrastructure including cloud, OT, and more.
Beyond global threat intelligence: Self-Learning AI enables novel threat detection & response
Darktrace does not rely on known malware signatures, external threat intelligence, historical attack data, nor does it rely on threat trained machine learning to identify threats.
Darktrace’s unique Self-learning AI deeply understands your business environment by analyzing trillions of real-time events that understands your normal ‘pattern of life’, unique to your business. By connecting isolated incidents across your business, including third party alerts and telemetry, Darktrace / NETWORK uses anomaly chains to identify deviations from normal activity.
The benefit to this is that when we are not predefining what we are looking for, we can spot new threats, allowing end users to identify both known threats and subtle, never-before-seen indicators of malicious activity that traditional solutions may miss if they are only looking at historical attack data.
AI-led investigations empower your SOC to prioritize what matters
Anomaly detection is often criticized for yielding high false positives, as it flags deviations from expected patterns that may not necessarily indicate a real threat or issues. However, Darktrace applies an investigation engine to automate alert triage and address alert fatigue.
Darktrace’s Cyber AI Analyst revolutionizes security operations by conducting continuous, full investigations across Darktrace and third-party alerts, transforming the alert triage process. Instead of addressing only a fraction of the thousands of daily alerts, Cyber AI Analyst automatically investigates every relevant alert, freeing up your team to focus on high-priority incidents and close security gaps.
Powered by advanced machine-learning techniques, including unsupervised learning, models trained by expert analysts, and tailored security language models, Cyber AI Analyst emulates human investigation skills, testing hypotheses, analyzing data, and drawing conclusions. According to Darktrace Internal Research, Cyber AI Analyst typically provides a SOC with up to 50,000 additional hours of Level 2 analysis and written reporting annually, enriching security operations by producing high level incident alerts with full details so that human analysts can focus on Level 3 tasks.
Containing threats with Autonomous Response
Simply quarantining a device is rarely the best course of action - organizations need to be able to maintain normal operations in the face of threats and choose the right course of action. Different organizations also require tailored response functions because they have different standards and protocols across a variety of unique devices. Ultimately, a ‘one size fits all’ approach to automated response actions puts organizations at risk of disrupting business operations.
Darktrace’s Autonomous Response tailors its actions to contain abnormal behavior across users and digital assets by understanding what is normal and stopping only what is not. Unlike blanket quarantines, it delivers a bespoke approach, blocking malicious activities that deviate from regular patterns while ensuring legitimate business operations remain uninterrupted.
Darktrace offers fully customizable response actions, seamlessly integrating with your workflows through hundreds of native integrations and an open API. It eliminates the need for costly development, natively disarming threats in seconds while extending capabilities with third-party tools like firewalls, EDR, SOAR, and ITSM solutions.
Unlocking a proactive state of security
Securing the network isn’t just about responding to incidents — it’s about being proactive, adaptive, and prepared for the unexpected. The NIST Cybersecurity Framework (CSF 2.0) emphasizes this by highlighting the need for focused risk management, continuous incident response (IR) refinement, and seamless integration of these processes with your detection and response capabilities.
Despite advancements in security technology, achieving a proactive posture is still a challenge to overcome because SOC teams face inefficiencies from reliance on pattern-matching tools, which generate excessive false positives and leave many alerts unaddressed, while novel threats go undetected. If SOC teams are spending all their time investigating alerts then there is no time spent getting ahead of attacks.
Achieving proactive network resilience — a state where organizations can confidently address challenges at every stage of their security posture — requires strategically aligned solutions that work seamlessly together across the attack lifecycle.
References
1. Market Guide for Extended Detection and Response, Gartner, 17thAugust 2023 - ID G00761828
RansomHub Ransomware: Darktrace’s Investigation of the Newest Tool in ShadowSyndicate's Arsenal
What is ShadowSyndicate?
ShadowSyndicate, also known as Infra Storm, is a threat actor reportedly active since July 2022, working with various ransomware groups and affiliates of ransomware programs, such as Quantum, Nokoyawa, and ALPHV. This threat actor employs tools like Cobalt Strike, Sliver, IcedID, and Matanbuchus malware in its attacks. ShadowSyndicate utilizes the same SSH fingerprint (1ca4cbac895fc3bd12417b77fc6ed31d) on many of their servers—85 as of September 2023. At least 52 of these servers have been linked to the Cobalt Strike command and control (C2) framework [1].
What is RansomHub?
First observed following the FBI's takedown of ALPHV/BlackCat in December 2023, RansomHub quickly gained notoriety as a Ransomware-as-a-Service (RaaS) operator. RansomHub capitalized on the law enforcement’s disruption of the LockBit group’s operations in February 2024 to market themselves to potential affiliates who had previously relied on LockBit’s encryptors. RansomHub's success can be largely attributed to their aggressive recruitment on underground forums, leading to the absorption of ex-ALPHV and ex-LockBit affiliates. They were one of the most active ransomware operators in 2024, with approximately 500 victims reported since February, according to their Dedicated Leak Site (DLS) [2].
ShadowSyndicate and RansomHub
External researchers have reported that ShadowSyndicate had as many as seven different ransomware families in their arsenal between July 2022, and September 2023. Now, ShadowSyndicate appears to have added RansomHub’s their formidable stockpile, becoming an affiliate of the RaaS provider [1].
Darktrace’s analysis of ShadowSyndicate across its customer base indicates that the group has been leveraging RansomHub ransomware in multiple attacks in September and October 2024. ShadowSyndicate likely shifted to using RansomHub due to the lucrative rates offered by this RaaS provider, with affiliates receiving up to 90% of the ransom—significantly higher than the general market rate of 70-80% [3].
In many instances where encryption was observed, ransom notes with the naming pattern “README_[a-zA-Z0-9]{6}.txt” were written to affected devices. The content of these ransom notes threatened to release stolen confidential data via RansomHub’s DLS unless a ransom was paid. During these attacks, data exfiltration activity to external endpoints using the SSH protocol was observed. The external endpoints to which the data was transferred were found to coincide with servers previously associated with ShadowSyndicate activity.
Darktrace’s coverage of ShadowSyndicate and RansomHub
Darktrace’s Threat Research team identified high-confidence indicators of compromise (IoCs) linked to the ShadowSyndicate group deploying RansomHub. The investigation revealed four separate incidents impacting Darktrace customers across various sectors, including education, manufacturing, and social services. In the investigated cases, multiple stages of the kill chain were observed, starting with initial internal reconnaissance and leading to eventual file encryption and data exfiltration.
Attack Overview
Internal Reconnaissance
The first observed stage of ShadowSyndicate attacks involved devices making multiple internal connection attempts to other internal devices over key ports, suggesting network scanning and enumeration activity. In this initial phase of the attack, the threat actor gathers critical details and information by scanning the network for open ports that might be potentially exploitable. In cases observed by Darktrace affected devices were typically seen attempting to connect to other internal locations over TCP ports including 22, 445 and 3389.
C2 Communication and Data Exfiltration
In most of the RansomHub cases investigated by Darktrace, unusual connections to endpoints associated with Splashtop, a remote desktop access software, were observed briefly before outbound SSH connections were identified.
Following this, Darktrace detected outbound SSH connections to the external IP address 46.161.27[.]151 using WinSCP, an open-source SSH client for Windows used for secure file transfer. The Cybersecurity and Infrastructure Security Agency (CISA) identified this IP address as malicious and associated it with ShadowSyndicate’s C2 infrastructure [4]. During connections to this IP, multiple gigabytes of data were exfiltrated from customer networks via SSH.
Data exfiltration attempts were consistent across investigated cases; however, the method of egress varied from one attack to another, as one would expect with a RaaS strain being employed by different affiliates. In addition to transfers to ShadowSyndicate’s infrastructure, threat actors were also observed transferring data to the cloud storage and file transfer service, MEGA, via HTTP connections using the ‘rclone’ user agent – a command-line program used to manage files on cloud storage. In another case, data exfiltration activity occurred over port 443, utilizing SSL connections.
Lateral Movement
In investigated incidents, lateral movement activity began shortly after C2 communications were established. In one case, Darktrace identified the unusual use of a new administrative credential which was quickly followed up with multiple suspicious executable file writes to other internal devices on the network.
The filenames for this executable followed the regex naming convention “[a-zA-Z]{6}.exe”, with two observed examples being “bWqQUx.exe” and “sdtMfs.exe”.
Additionally, script files such as “Defeat-Defender2.bat”, “Share.bat”, and “def.bat” were also seen written over SMB, suggesting that threat actors were trying to evade network defenses and detection by antivirus software like Microsoft Defender.
File Encryption
Among the three cases where file encryption activity was observed, file names were changed by adding an extension following the regex format “.[a-zA-Z0-9]{6}”. Ransom notes with a similar naming convention, “README_[a-zA-Z0-9]{6}.txt”, were written to each share. While the content of the ransom notes differed slightly in each case, most contained similar text. Clear indicators in the body of the ransom notes pointed to the use of RansomHub ransomware in these attacks. As is increasingly the case, threat actors employed double extortion tactics, threatening to leak confidential data if the ransom was not paid. Like most ransomware, RansomHub included TOR site links for communication between its "customer service team" and the target.
Since Darktrace’s Autonomous Response capability was not enabled during the compromise, the ransomware attack succeeded in its objective. However, Darktrace’s Cyber AI Analyst provided comprehensive coverage of the kill chain, enabling the customer to quickly identify affected devices and initiate remediation.
In lieu of Autonomous Response being active on the networks, Darktrace was able to suggest a variety of manual response actions intended to contain the compromise and prevent further malicious activity. Had Autonomous Response been enabled at the time of the attack, these actions would have been quickly applied without any human interaction, potentially halting the ransomware attack earlier in the kill chain.
Conclusion
The Darktrace Threat Research team has noted a surge in attacks by the ShadowSyndicate group using RansomHub’s RaaS of late. RaaS has become increasingly popular across the threat landscape due to its ease of access to malware and script execution. As more individual threat actors adopt RaaS, security teams are struggling to defend against the increasing number of opportunistic attacks.
For customers subscribed to Darktrace’s Security Operations Center (SOC) services, the Analyst team promptly investigated detections of the aforementioned unusual and anomalous activities in the initial infection phases. Multiple alerts were raised via Darktrace’s Managed Threat Detection to warn customers of active ransomware incidents. By emphasizing anomaly-based detection and response, Darktrace can effectively identify devices affected by ransomware and take action against emerging activity, minimizing disruption and impact on customer networks.
Credit to Kwa Qing Hong (Senior Cyber Analyst and Deputy Analyst Team Lead, Singapore) and Signe Zahark (Principal Cyber Analyst, Japan)