Cyber attacks on internet-facing systems
In the first half of 2024, the Darktrace Threat Research team observed multiple campaigns of threat actors targeting vulnerabilities in internet-facing systems, including Ivanti CS/PS appliances, Palo Alto firewall devices, and TeamCity on-premises.
These systems, which are exposed to the internet, are often targeted by threat actors to gain initial access to a network. They are constantly being scanned for vulnerabilities, known or unknown, by opportunistic actors hoping to exploit gaps in security. Unfortunately, this exposure remains a significant blind spot for many security teams, as monitoring edge infrastructure can be particularly challenging due to its distributed nature and the sheer volume of external traffic it processes.
In this blog, we discuss a vulnerability that was exploited in Fortinet’s FortiClient Endpoint Management Server (EMS) and the post-exploitation activity that Darktrace observed across multiple customer environments.
What is FortiClient EMS?
FortiClient is typically used for endpoint security, providing features such as virtual private networks (VPN), malware protection, and web filtering. The FortiClient EMS is a centralized platform used by administrators to enforce security policies and manage endpoint compliance. As endpoints are remote and distributed across various locations, the EMS needs to be accessible over the internet.
However, being exposed to the internet presents significant security risks, and exploiting vulnerabilities in the system may give an attacker unauthorized access. From there, they could conduct further malicious activities such as reconnaissance, establishing command-and-control (C2), moving laterally across the network, and accessing sensitive data.
CVE-2023-48788
CVE-2023-48788 is a critical SQL injection vulnerability in FortiClient EMS that can allow an attacker to gain unauthorized access to the system. It stems from improper neutralization of special elements used in SQL commands, which allows attackers to exploit the system through specially crafted requests, potentially leading to Remote Code Execution (RCE) [1]. This critical vulnerability was given a CVSS score of 9.8 and can be exploited without authentication.
The affected versions of FortiClient EMS include:
- FortiClient EMS 7.2.0 to 7.2.2 (fixed in 7.2.3)
- FortiClient EMS 7.0.1 to 7.0.10 (fixed in 7.0.11)
The vulnerability was publicly disclosed on March 12, 2024, and an exploit proof of concept was released by Horizon3.ai on March 21 [2]. Starting from March 24, almost two weeks after the initial disclosure, Darktrace began to observe at least six instances where the FortiClient EMS vulnerability had likely been exploited on customer networks. Seemingly exploited devices in multiple customer environments were observed performing anomalous activities, including the installation of Remote Monitoring and Management (RMM) tools, which was also reported by other security vendors around the same time [3].
Darktrace’s Coverage
Initial Access
To understand how the vulnerability can be exploited to gain initial access, we first need to explain some components of the FortiClient EMS:
- The service FmcDaemon.exe is used for communication between the EMS and enrolled endpoint clients. It listens on port 8013 for incoming client connections.
- Incoming requests are then sent to FCTDas.exe, which translates requests from other server components into SQL requests. This service interacts with the Microsoft SQL database.
- Endpoint clients communicate with the FmcDaemon on the server on port 8013 by default.
Therefore, an SQL injection attack can be performed by crafting a malicious payload and sending it over port 8013 to the server. To carry out RCE, an attacker may send further SQL statements to enable and use the xp_cmdshell functionality of the Microsoft SQL server [2].
Shortly before post-exploitation activity began, Darktrace had observed incoming connections to some of the FortiClient EMS devices over port 8013 from the external IPs 77.246.103[.]110, 88.130.150[.]101, and 45.155.141[.]219. This likely represented the threat actors sending an SQL injection payload over port 8013 to the EMS device to validate the exploit.
Establish C2
After exploiting the vulnerability and gaining access to an EMS device on one customer network, two additional devices were seen with HTTP POST requests to 77.246.103[.]110 and 212.113.106[.]100 with a new PowerShell user agent.
Interestingly, the IP 212.113.106[.]100 has been observed in various other campaigns where threat actors have also targeted internet-facing systems and exploited other vulnerabilities. Open-source intelligence (OSINT) suggests that this indicator of compromise (IoC) is related to the Sliver C2 framework and has been used by threat actors such as APT28 (Fancy Bear) and APT29 (Cozy Bear) [4].
Unusual file downloads were also observed on four devices, including:
- “SETUP.MSI” from 212.32.243[.]25 and 89.149.200[.]91 with a cURL user agent
- “setup.msi” from 212.113.106[.]100 with a Windows Installer user agent
- “run.zip” from 95.181.173[.]172 with a PowerShell user agent
The .msi files would typically contain the RMM tools Atera or ScreenConnect [5]. By installing RMM tools for C2, attackers can leverage their wide range of functionalities to carry out various tasks, such as file transfers, without the need to install additional tools. As RMM tools are designed to maintain a stable connection to remote systems, they may also allow the attackers to ensure persistent access to the compromised systems.
A scan of the endpoint 95.181.173[.]172 shows various other files such as “RunSchedulerTask.ps1” and “anydesk.exe” being hosted.
![Screenshot of the endpoint 95.181.173[.]172 hosting various files [6].](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/66eb0afcc6a12ac4c8d91f16_66eb0a19eef352705e3e3cf1_Screenshot%25202024-09-18%2520at%252010.12.48%25E2%2580%25AFAM.png)
Shortly after these unusual file downloads, many of the devices were also seen with usage of RMM tools such as Splashtop, Atera, and AnyDesk. The devices were seen connecting to the following endpoints:
- *[.]relay.splashtop[.]com
- agent-api[.]atera[.]com
- api[.]playanext[.]com with user agent AnyDesk/8.0.9
RMM tools have a wide range of legitimate capabilities that allow IT administrators to remotely manage endpoints. However, they can also be repurposed for malicious activities, allowing threat actors to maintain persistent access to systems, execute commands remotely, and even exfiltrate data. As the use of RMM tools can be legitimate, they offer threat actors a way to perform malicious activities while blending into normal business operations, which could evade detection by human analysts or traditional security tools.
One device was also seen making repeated SSL connections to a self-signed endpoint “azure-documents[.]com” (104.168.140[.]84) and further HTTP POSTs to “serv1[.]api[.]9hits[.]com/we/session” (128.199.207[.]131). Although the contents of these connections were encrypted, they were likely additional infrastructure used for C2 in addition to the RMM tools that were used. Self-signed certificates may also be used by an attacker to encrypt C2 communications.
Internal Reconnaissance
Following the exploit, two of the compromised devices then started to conduct internal reconnaissance activity. The following figure shows a spike in the number of internal connections made by one of the compromised devices on the customer’s environment, which typically indicates a network scan.
Reconnaissance tools such as Advanced Port Scanner (“www[.]advanced-port-scanner[.]com”) and Nmap were also seen being used by one of the devices to conduct scanning activities. Nmap is a network scanning tool commonly used by security teams for legitimate purposes like network diagnostics and vulnerability scanning. However, it can also be abused by threat actors to perform network reconnaissance, a technique known as Living off the Land (LotL). This not only reduces the need for custom or external tools but also reduces the risk of exposure, as the use of a legitimate tool in the network is unlikely to raise suspicion.
Privilege Escalation
In another affected customer network, the threat actor’s attempt to escalate their privileges was also observed, as a FortiClient EMS device was seen with an unusually large number of SMB/NTLM login failures, indicative of brute force activity. This attempt was successful, and the device was later seen authenticating with the credential “administrator”.
Lateral Movement
After escalating privileges, attempts to move laterally throughout the same network were seen. One device was seen transferring the file “PSEXESVC.exe” to another device over SMB. This file is associated with PsExec, a command-line tool that allows for remote execution on other systems.
The threat actor was also observed leveraging the DCE-RPC protocol to move laterally within the network. Devices were seen with activity such as an increase in new RPC services, unusual requests to the SVCCTL endpoint, and the execution of WMI commands. The DCE-RPC protocol is typically used to facilitate communication between services on different systems and can allow one system to request services or execute commands on another.
These are further examples of LotL techniques used by threat actors exploiting CVE-2023-48788, as PsExec and the DCE-RPC protocol are often also used for legitimate administrative operations.
Accomplish Mission
In most cases, the threat actor’s end goal was not clearly observed. However, Darktrace did detect one instance where an unusually large volume of data had been uploaded to “put[.]io”, a cloud storage service, indicating that the end goal of the threat actor had been to steal potentially sensitive data.
In a recent investigation of a Medusa ransomware incident that took place in July 2024, Darktrace’s Threat Research team found that initial access to the environment had likely been gained through a FortiClient EMS device. An incoming connection from 209.15.71[.]121 over port 8013 was seen, suggesting that CVE-2023-48788 had been exploited. The device had been compromised almost three weeks before the ransomware was actually deployed, eventually resulting in the encryption of files.
Conclusion
Threat actors have continued to exploit unpatched vulnerabilities in internet-facing systems to gain initial access to a network. This highlights the importance of addressing and patching vulnerabilities as soon as they are disclosed and a fix is released. However, due to the rapid nature of exploitation, this may not always be enough. Furthermore, threat actors may even be exploiting vulnerabilities that are not yet publicly known.
As the end goals for a threat actor can differ – from data exfiltration to deploying ransomware – the post-exploitation behavior can also vary from actor to actor. However, AI security tools such as Darktrace / NETWORK can help identify and alert for post-exploitation behavior based on abnormal activity seen in the network environment.
Despite CVE-2023-48788 having been publicly disclosed and fixed in March, it appears that multiple threat actors, such as the Medusa ransomware group, have continued to exploit the vulnerability on unpatched systems. With new vulnerabilities being disclosed almost every other day, security teams may find it challenging continuously patch their systems.
As such, Darktrace / Proactive Exposure Management could also alleviate the workload of security teams by helping them identify and prioritize the most critical vulnerabilities in their network.
Credit to Emily Megan Lim (Cyber Security Analyst) and Ryan Traill (Threat Content Lead)
Appendices
References
[1] https://nvd.nist.gov/vuln/detail/CVE-2023-48788
[2] https://www.horizon3.ai/attack-research/attack-blogs/cve-2023-48788-fortinet-forticlientems-sql-injection-deep-dive/
[3] https://redcanary.com/blog/threat-intelligence/cve-2023-48788/
[4] https://www.fortinet.com/blog/threat-research/teamcity-intrusion-saga-apt29-suspected-exploiting-cve-2023-42793
[5] https://redcanary.com/blog/threat-intelligence/cve-2023-48788/
[6] https://urlscan.io/result/3678b9e2-ad61-4719-bcef-b19cadcdd929/
List of IoCs
IoC - Type - Description + Confidence
- 212.32.243[.]25/SETUP.MSI - URL - Payload
- 89.149.200[.]9/SETUP.MSI - URL - Payload
- 212.113.106[.]100/setup.msi - URL - Payload
- 95.181.173[.]172/run.zip - URL - Payload
- serv1[.]api[.]9hits[.]com - Domain - Likely C2 endpoint
- 128.199.207[.]131 - IP - Likely C2 endpoint
- azure-documents[.]com - Domain - C2 endpoint
- 104.168.140[.]84 - IP - C2 endpoint
- 77.246.103[.]110 - IP - Likely C2 endpoint
- 212.113.106[.]100 - IP - C2 endpoint
Darktrace Model Detections
Anomalous Connection / Callback on Web Facing Device
Anomalous Connection / Multiple HTTP POSTs to Rare Hostname
Anomalous Connection / New User Agent to IP Without Hostname
Anomalous Connection / Posting HTTP to IP Without Hostname
Anomalous Connection / Powershell to Rare External
Anomalous Connection / Rare External SSL Self-Signed
Anomalous Connection / Suspicious Self-Signed SSL
Anomalous Server Activity / Rare External from Server
Anomalous Server Activity / New User Agent from Internet Facing System
Anomalous Server Activity / Server Activity on New Non-Standard Port - External
Compliance / Remote Management Tool On Server
Device / New User Agent
Device / New PowerShell User Agent
Device / Attack and Recon Tools
Device / ICMP Address Scan
Device / Network Range Scan
Device / Network Scan
Device / RDP Scan
Device / Suspicious SMB Scanning Activity
Anomalous Connection / Multiple SMB Admin Session
Anomalous Connection / New or Uncommon Service Control
Anomalous Connection / Unusual Admin SMB Session
Device / Increase in New RPC Services
Device / Multiple Lateral Movement Breaches
Device / New or Uncommon WMI Activity
Device / New or Unusual Remote Command Execution
Device / SMB Lateral Movement
Device / Possible SMB/NTLM Brute Force
Unusual Activity / Successful Admin Brute-Force Activity
User / New Admin Credentials on Server
Unusual Activity / Enhanced Unusual External Data Transfer
Unusual Activity / Unusual External Data Transfer
Unusual Activity / Unusual External Data to New Endpoint
Device / Large Number of Model Breaches
Device / Large Number of Model Breaches from Critical Network Device
MITRE ATT&CK Mapping
Tactic – ID: Technique
Initial Access – T1190: Exploit Public-Facing Application
Resource Development – T1587.003: Develop Capabilities: Digital Certificates
Resource Development – T1608.003: Stage Capabilities: Install Digital Certificate
Command and Control – T1071.001: Application Layer Protocol: Web Protocols
Command and Control – T1219: Remote Access Software
Execution – T1059.001: Command and Scripting Interpreter: PowerShell
Reconnaissance – T1595: Active Scanning
Reconnaissance – T1590.005: Gather Victim Network Information: IP Addresses
Discovery – T1046: Network Service Discovery
Credential Access – T1110: Brute Force
Defense Evasion,Initial Access,Persistence,Privilege Escalation – T1078: Valid Accounts
Lateral Movement – T1021.002: Remote Services: SMB/Windows Admin Shares
Lateral Movement – T1021.003: Remote Services: Distributed Component Object Model
Execution – T1569.002: System Services: Service Execution
Execution – T1047: Windows Management Instrumentation
Exfiltration – T1041: Exfiltration Over C2 Channel
Exfiltration – T1567.002: Exfiltration Over Web Service: Exfiltration to Cloud Storage
![External Sites Summary Graph showing the rarity of the hostname “92n7au[.]uhabq9[.]com” on a customer network.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df74_670d956f4dda5eb6ac9e650f_Screenshot%25202024-10-14%2520at%25203.04.22%25E2%2580%25AFPM.png)
![Screenshot of a Model Alert Event log showing a device connecting to the endpoint “is5jg[.]3zweuj[.]com” over port 32091.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df7d_670d959854eb594420fdc971_Screenshot%25202024-10-14%2520at%25203.05.03%25E2%2580%25AFPM.png)
![Screenshot of a Model Alert Event Log showing the device posting the string “i8xps1” to the hostname “72zf6.rxqfd[.]com.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df80_670d95c562c6e392c7dc3ba8_Screenshot%25202024-10-14%2520at%25203.05.49%25E2%2580%25AFPM.png)
![Screenshot of a Model Alert Event Log showing the device posting the string “sqyjyadwwq” to the hostname “9yrh7.mea5ms[.]com”.](https://cdn.prod.website-files.com/626ff4d25aca2edf4325ff97/670d96b065612663d098df7a_670d95e4b1dbad771717553c_Screenshot%25202024-10-14%2520at%25203.06.20%25E2%2580%25AFPM.png)
