Wyatt Roersma Blog

Update:This will not help you with versions of CryptoWall 2.0 and up they have since patched this issue. 

A couple of weeks ago I got a call from a client that one of their employees had clicked on an attachment named “electronic_fund_transfer.zip” in a spam email. Naturally, the employee opened the PDF from within the zip file and then clicked “Run” to launch the executable inside.

In a typical organization, the main concern in such a situation would be what data was exfiltrated from the environment, not the data that was lost due to not having proper backups. You could just wipe the system and restore any lost data from backup, and spend your time figuring out what the malware accomplished while on the system. Well, this client didn’t have working backups in place and the user had also mounted file server shares to his laptop. So, not only did his data get encrypted, some of the data on the file server did as well, with no backups for months of either system.

When the client first contacted me they called it “CryptoDefense” which we can indeed decrypt without issue because there is plenty of data out there on how to do that. Naturally, once we arrived on site, we quickly found out that this was the much more advanced CryptoWall malware that doesn’t store the private key needed to decrypt the files on the local system.

At that point I was between a rock and a hard place because we had initially told them it was possible to recover the encrypted data. However, the majority of resources on the Internet indicate that it’s not possible to recover data at all when CryptoWall is installed as opposed to CryptoDefense. Most file recovery methods suggest using VSS copies to recover the data or backups, otherwise you’re simply out of luck. In these cases, I do not consider paying the data terrorists as an option.

So, as part of our basic Triage process, we obtained memory and disk images. These helped a lot in understanding how this malware works and achieving the ultimate goal of recovering the data.

To work with the malware for this blog post I created a virtual machine, fresh to launch this malware on, to run a few scans and tests.

My first goal was to determine how this malware was encrypting the data and in which method it was deleting the original files. I searched the strings file created from the volatility strings plugin. I used IDA pro on the “vofse.exe” file that does the encryption part (it’s the second file that is downloaded after sicac.exe). The ransomware was simply using the DeleteFile Function to remove the files after making a copying from the original file.

This screenshot is where the ransomware finds the file and creates a copy of it.

Here is a screenshot with the DeleteFile function highlighted.

So, I made a disk image before installing the malware and another after letting the malware run to “encrypt” the files.

First, to get a list of the files it “encrypted” I printed out the list it makes in the registry. To find the registry key that the malware created, look in “HKCU\Software\<unique ID>\ CRYPLIST\” as it contains a list of the encrypted files.

root@BT:~/volatility-master# python vol.py -f win7.vmem –profile=Win7SP1x64 printkey -K “Software\1C1AA48085BD197637A78463CBBE8BC2\CRYPTLIST”

Legend: (S) = Stable   (V) = Volatile

—————————-

Registry: \??\C:\Users\malware\ntuser.dat

Key name: CRYPTLIST (S)

Last updated: 2014-06-09 12:45:57 UTC+0000

Subkeys:

Snip…

Values:

REG_DWORD     C:\Users\malware\Documents\00698_snowmountains_1920x1200.jpg : (S) 2054688515

2054688515

Snip…

REG_DWORD     C:\Users\malware\Documents\400 – Linux Software RAID Rebuild.doc : (S) 2054688515

REG_DWORD     C:\Users\malware\Documents\Cisco_2014_ASR.pdf : (S) 2054688515

REG_DWORD     C:\Users\malware\Documents\GrrCON-Challenge.docx : (S) 2054688515

REG_DWORD     C:\Users\malware\Documents\Microsoft SQL Server AlwaysOn Solutions Guide for High Availability and Disaster Recovery.docx : (S) 2054688515

REG_DWORD     C:\Users\malware\Documents\Sophos_ZeroAccess_Botnet.pdf : (S) 2054688515

REG_DWORD     C:\Users\malware\Documents\YARA User’s Manual 1.6.pdf : (S) 2054688515

REG_DWORD     C:\Users\malware\Downloads\00698_snowmountains_1920x1200.jpg : (S) 2054688515

Snip…

 

I then opened “GrrCON-Challenge.docx” (one of the encrypted documents) in a hex editor, copied the raw hex values, and searched through both of my disk images, before the malware, and after the malware. While the file itself was removed according to the malware it was still in the same spot on disk at offset 004c000 in both disk images.

 

Here is the file in the “malwarevm-before.001”:

Here is the “GrrCON-Challenge.docx” document at the same offset 004c000 on the “aftermalwarevm-cryptowall.001”

I also opened the encrypted version of “GrrCON-Challenge.docx” and then searched the “aftermalwarevm-cryptowall.001” to find where it was located on disk to confirm they create a new file compared to the old disk image. I will note one very interesting fact: while I have two versions of the “GrrCON-Challenge.docx” file, the malware only created one encrypted version.

Now that I have confirmed that the malware isn’t fully deleting the files, I want to test it out on some more files to see how much data I can recover.

I ran one of my favorite file recovery tools called R-Studio (Yes, I like it because it’s fast and saves me some time from the more manual solutions available). R-Studio is a very simple program. Basically, all you have to do is mount a drive and scan it, and then choose which file types you want to look for. I just used the default configuration on the “aftermalwarevm-cryptowall.001”. I selected the Microsoft Word 2007 XML Document (.docx) and noticed I had two sets of files. The ones with file names are the encrypted version by CryptoWall and the ones without names are the files that were deleted recoverable from slack space.

Based on the file size and hex values I know “70.docx” is my “GrrCON-Challenge.docx” file, so I clicked on the view process before recovering the file to ensure it’s the proper file from the hex view and not the encrypted file. I was indeed correct because the picture above lists the file header as “PK” which is the proper file header in the docx file.

Listed below is a screenshot of the encrypted file and its header so we know if we are looking at an encrypted file or not.

In my test example all encrypted files start with the header “CE FE” in hex, which will be different on each system that is hit will CryptoWall.

So I recovered all the files and ran a sha256 hash verification to ensure all my recover files matched the original files.

Recovered File Hash Check

Original File Hash Check

I only tested with a few files in the following formats, docx, doc, pdf, and jpg. As a disclaimer with all file recovery some files may not be recoverable if the slack space was overwritten. In each separate case, file recovery will depend on how many files were encrypted, and how much free space the drive had before it had to start overwriting old slack space for new files.

If anyone wants to look at the attachment that launched the attack, here it is, along with the malwr.com automated analysis, which is somewhat helpful. *Note: you have to sign up to download the file, and use at your own risk. I am not responsible for any damages you cause to yourself.

https://malwr.com/analysis/ZmU1MzE0MmU5MmExNDc5YjkxMDQ2ZmI1ZWEyYjM4Nzk/

In my client’s case I was able to recover approximately %95 of the files, however based on their actual need we only searched for the critical documents and nothing that was lost from the user’s personal files. Just because ransomware encrypts the data doesn’t mean it’s lost forever because to actual erase the files would be much more time consuming and resource intensive. It also would be impossible to actually zero out the files on a file server because the user doesn’t have the proper permissions to access the raw drive in that method (*If permissions are properly setup that is, and User quotas are in place). If anyone would like the files used in this blog post, I would be happy to share the full images. I would also be happy to accept comments and questions just send me an email to wyattroersma (at) gmail.

If anyone would like to know more about memory forensics I’d recommend pre-ordering this The Art of Memory Forensics http://www.amazon.com/The-Art-Memory-Forensics-Detecting/dp/1118825098. Also if anyone needs assistance with Cryptowall or any other piece of malware please just email me.

I’d like to Thank Andrew Case and Michael Hale Ligh for their advise and review of this blog post write  up. I’d also like to thank DoctorW0rm a reddit user who found some technical flaws which I have corrected from “CryptoLocker to CryptoDefense”

Edit:This will not help you with versions of CryptoWall 2.0 and up they have since patched this issue. 

Another long gap between since my last post, what else is new.  I have no idea where the time has gone between working on development projects and testing too many things to list. So to kick off this post here is some interesting personal perspective on security in virtualization.

Now I am more than aware that 99% of the virtualization technology in production today is VMware and that Hyper-V is not at the top of most organizations’ consideration. I hope to change that perspective through this small review of some approaches that NVINT is taking as a small company. I am not here to try and sell Microsoft products (I am more of a Google Fan than anything) but when something is done right for more than one reason I always will like that product.

This will be a multi-part blog post that covers a few topics across Server 2012’s Hyper-V 3.0 to the 2012 System Center product family as they relate to the DFIR world.

Hyper-V

I will start with Hyper-V 3.0 inside server 2012 because there are a lot of great new features that really put it above other hypervisors. The first thing that really stands out as a major feature in 3.0 is that you can fully encrypt your storage clusters and allow access to that Shared SMB cluster over the network. This allows you to separate the Hyper-V boxes hosting the Virtual Machines from the machines hosting the SMB Storage cluster. This allows you to lock down access from only the Hyper-V servers to the storage cluster and monitor what accesses that data directly.

At NVINT I currently work with a Hyper-V 2008 R2 cluster that hosts over one-hundred virtual machines across six HP blade servers that make up the Hyper-V host machines. This Hyper-V cluster hosts the production virtual servers for our clients. I have been testing Hyper-V 3.0 for the past ten months, and I still feel like I have only opened one of the presents Santa left me under the tree. In my experience the stability and redundancy is really now on par with other hypervisors as where our current production 2008 R2 Hyper-V cluster lacks.

Windows Server 2012 features a lot of new capabilities, such as the new cloud networking features that scale a higher level network across WAN connections. It also contains the same powerful security features from the past like ASLR and DEP that are critical for security. I am not going to go into the infrastructure aspects because that has been done by the Microsoft engineers and I have provided the links below.

Check out these links if you want to know more about Server 2012 Hyper-V features.

Windows Server 2012 Hyper-V & Network Card (NIC) Teaming – http://www.aidanfinn.com/?p=12924

Hyper-V Network Virtualization Gateway Architectural Guide – http://technet.microsoft.com/en-us/library/jj618319.aspx

Hyper-V Network Virtualization Overview – http://technet.microsoft.com/library/jj134230.aspx

Network Virtualization technical details – http://technet.microsoft.com/en-US/library/jj134174.aspx

Windows Server 2012 Hyper-V Replica http://www.aidanfinn.com/?p=12147

The major upside to Hyper-V for NVINT is the fact that the Datacenter edition of windows allows you to host unlimited Standard editions of Windows in virtual machines without any extra license fees. (Note: this has limitations but that’s outside the scope of this post)

There is one really big feature of Hyper-V that I want to point out, as I feel there is a huge benefit from an Incident Response/Forensics point of view in terms of what you can do with the system that other hypervisors lack. For starters there are a large number of tools that you can use to analyze the Windows OS in many different ways. One of the best features is the ability to take a memory dump of the host os itself.

I have taken memory captures up to 80GB in the system and used Volatility to analyze those dumps successfully. This allows me to peer into the virtualization platform with more in depth detail that can be critical to understanding what is going on in the environment. Most people use VMware which makes it the most targeted Platform. Advanced techniques can be used to detect if the system is a virtual machine and even exploit out of the system to the host causing problems they were intended to prevent.  This may change in the future, but currently Hyper-V is not the largest target on the market in its virtual machines. I am well aware the Windows OS is targeted by everything because it’s used by almost any originations today. I also feel that VMware currently has risks with USB support, currently however that isn’t a drastic problem yet. Someday however I would guess that it will become a more targeted exploit.

Alongside Server 2012 and the features it brings to the table the system center products allow a great deal of management and monitoring across all typed of devices outside the Microsoft Family.

DPM 2012

I currently work with Data Protection Manager almost every single day, and I have come to enjoy this product for a few good reasons. A major benefit for DPM is that it allows you to actively back up the memory as a scheduled process that Microsoft calls a “System State” backup. It also has standard features with Full-Disk and Incremental backups based on a scheduled format. It does anything from attached SAN storage to Tape backups which allows you to scale it out fairly easily.

At NVINT we maintain 15 days of full disk backups and the “System State” included. This allows me to restore a Virtual Machine to a saved state inside Hyper-V or to a network location much like the features inside VMware consolidated backup. The huge benefit from an analysis standpoint is that Hyper-V stores the system RAM for a virtual machine in two separate files the .bin and the .vsv which can be used with vm2dpm  http://archive.msdn.microsoft.com/vm2dmp to convert them into a crash dump format that is compatible with Volatility or Microsoft Debugger for analysis. In an incident I can go into my Hyper-V server and use FTK Imager to create a copy of the live system’s disc and memory without installing anything in the Virtual Machine.

A key benefit of this process with FTK is to gather evidence without direct system interaction thus reducing the footprint on the compromised system. As a hosting provider this is critical because we can’t actively monitor with tools inside the OS itself. The tool Vm2dmp can also convert the DPM backup files because the .bin and .vsv file format is used to store the “System State” for protected systems. I feel that if something happens, that this type of insight can be critical with huge amounts of detail in what happened in the past for a better investigation. These backup files could reduce the data lost by an attacker using anti-forensics techniques. In most cases today the bad guys are getting really good at this process of covering their tracks so the DFIR community needs to get ahead of the problem before it occurs. Outside of virtual machines DPM can still perform “System State” backups of physical machines from Windows 7, Windows 8, Server 2008 R2 and Server 2012.

Here is some quick information on how to use vm2dmp

Create a dump file using virtual machine state files:

vm2dmp.exe -bin C:\VM\example.bin -vsv C:\VM\example.vsv -dmp C:\VM\memory.dmp

Create a dump file from virtual machine and snapshot name:

vm2dmp.exe –vm SERVER2008R2SP1-ENT-64 -dmp C:\VM\memory.dmp

vm2dmp.exe –vm vm SERVER2008R2SP1-ENT-64 –snap “vm SERVER2008R2-ENT-64 -snapshot-SP1” -dmp C:\VM\memory.dmp

Create a dump file using local symbols:

vm2dmp.exe -bin C:\VM\example.bin -vsv C:\VM\example.vsv -dmp C:\VM\memory.dmp –sym C:\symbols

For more information download the vm2dmp guide here: http://archive.msdn.microsoft.com/vm2dmp/Release/ProjectReleases.aspx?ReleaseId=3866

SCOM 2012

System Center Operations manager allows you to manage your network and give some detailed information into your environment in a central location. It supports Windows/Linux and SNMP devices alike. It allows options from an installed agent or agentless monitoring for information.

It has plenty of powerful rules that integrate into active directory and auto discovery rules to find new devices in the environment. This management server allows a centralized event log collection instead of taking up critical space on production systems. It can alert you via text or email if an event occurs based on custom rules. It even provides infrastructure information for System Administrators on descriptions, possible resolutions, and common causes.

I feel like it’s a more advanced Splunk that allows you target information with custom dashboards. I will not go into all the details because it can almost do anything and I provided some links to check out if you are interested.

For more information on SCOM check out these websites

http://blogs.technet.com/b/kevinholman/archive/2011/07/26/deploying-opsmgr-2012-a-quick-start-guide.aspx

http://www.scom2k7.com/installing-the-scom-2012-web-console-prerequisites-the-easy-way/

http://blogs.technet.com/b/kevinholman/archive/2011/07/26/deploying-opsmgr-2012-a-quick-start-guide.aspx

http://www.scom2k7.com/installing-the-scom-2012-web-console-prerequisites-the-easy-way/

http://technet.microsoft.com/library/hh278852.aspx

http://blogs.technet.com/b/momteam/

http://www.windowsnetworking.com/articles_tutorials/Introduction-System-Center-Operations-Manager-2012-Part1.html

http://technet.microsoft.com/library/hh278852.aspx

http://blogs.technet.com/b/momteam/

http://www.windowsnetworking.com/articles_tutorials/Introduction-System-Center-Operations-Manager-2012-Part1.html

 

VMM 2012

Virtual Machine Manger simply allows you to deploy fully updated virtual machines to your Infrastructure that follow a “Best Practice” standard with repeatable processes to avoid human mistakes. Even the simplest detail can be catastrophic for security protections in a complex environment. This also allows you to save time for the infrastructure guys allowing them to focus on much more important tasks.

I would love to go into more detail but I feel this post has covered enough for the day. Please feel free to leave any questions below or email me and I will respond as quickly as possible. Constructive criticism is always desired so please don’t hesitate.

I would to thank Andrew Case and Michael Hale Ligh for some proof reading and great suggestions. They did more than I expected as my first version was a little rough.