Windows: 2012 Server: Restoration of server from ransomware brute force attack – Real time experience

First of all, it’s a lengthy article, but interesting one.

Its all about restoring your windows server from ransomware attacks, we worked for straight 14 hours to restore the server from ransomware attack.

How the server is configured?

  • Installed with windows server 2012

  • Hosted with AD, DNS and File Server

  • ESET Remote Administrator and Antivirus was installed

  • Remote desktop to server was enabled through Natting using public IP

  • hMailServer as mail server was installed

  • Shadow copies were enabled on important data drives

  • Automated full backups were scheduled to run every six hours (as we have less data and high amount of space)

  • Some third party ERP solutions was also installed

What happened to the server?

  • Server was infected by ransomware named “!!! READ THIS - IMPORTANT !!!.hta”, which ever folder you open you will see a copy of this file.

  • It has encrypted all data on the server including our full back up files

  • ESET was uninstalled when we noticed the server was infected

  • All the files were renamed with some strange format like filename_decrypt_something_something_something

  • You cannot open any of your data because it was encrypted by this ransomware

Now we have lost our backup’s and data, how do we restore the server back. Its almost 10 years critical and important data that we need to restore now. Luckily we were able to login to the OS without any problem but we can’t launch any app like server manager, cmd or any other except powershell.

How the server and its data was restored?

  • First we booted the server using ESET bootable cd and removed the auto executables

  • Then we tried to find any unknown apps loading during startup, tried to find any unknown executables from task manager but none has identified.

  • So before proceeding with restoration we searched for ransomware traces and deleted all of them from all folders (we searched with the name and found 1000’s of them). Now the server speed was increased and auto executables were removed.

  • At this point in time our life saver was shadow copies configured on data drives, so we verified which drives have these shadow copies configured and tried to open those using explorer.

  • We were sure that the shadow copies were configured on both data and backup drives, but only data drive showing shadow copies and backup drive was not.

  • Now we need AD also to be kept safe along with data, so I built a brand new windows 2012 server and made it as secondary domain controller so entire active directory got replicated to the new one, I’m safe on user accounts and other details, also I removed it from network to keep it safe once its promoted as DC.

  • Then I used ntdsutil to see if there are any snapshots of ad taken before.

  • I entered the snapshot prompt where we can list out existing snapshots, they are nothing but shadow copies. The beauty of this command is that we can mount them to a folder, with this I got to know a surprising fact that there are shadow copies also created on backup drive but somehow they are invisible.

  • But I haven’t used this option to restore data, I opened an existing shadow copy from data drive which has absolute fine copy from earlier day and started copying data back to external drive. But we faced couple of other challenges like slowness, file path too long to copy etc.

  • So I found another alternative to fix the challenge, that we can mount a specific shadow copy to a drive letter instead of opening it from explorer directly which avoids file path length issues. Besides I used robocopy to copy file from the shadow copy.

  • To do all this I used diskshadow command, where we can list all the existing shadow copies and also we can mount them to a drive letter.

  • To restore all the data, it took about 6 hours for me, now the next challenge is OS restoration. So I mounted the shadow copy on backup drive again using same diskshadow command and copied my last full back up to external drive which was a vhdx file.

  • I opened this vhdx file using diskmgmt.msc from file menu attach vhd option. It opened on to a drive letter where I can see my OS and applications perfectly fine. Ofcourse I was able to extract one of my full backups but the backup utility in windows was unable to identify this back up as its catalog was corrupted by ransomware. Now next challenge

  • I created a wim file from this vhdx using new-windowimage command powershell command.

  • And finally after all my data is restored, ad was safe on other dc and now I booted the server from 2012 bootable cd and entered repair mode.

  • I have applied this newly created wim file using dism command from command prompt to c drive.

  • It restored my OS and AD to its previous state, now the server is completely restored.

After my restoration was successful, I updated my antivirus and scanned for viruses. So the data was safe and I left it alone.

Later that day, I got a question in my mind what if the hacker has created any accounts in AD before he inserted the ransomware?

Immediately I logged into the server and my doubt was right he created two accounts and made them members of administrator group, so once I found them I have deleted them without second thought. He created these accounts again after the restoration, that’s the surprising thing, so he is willing to repeat the same feat again. Anyhow I have deleted those accounts.

Now one more interesting thing is he injected an exe called NLBrute and launched it to system tray with the name as ESET monitor. When I opened it I found few more interesting things, this tool giving him RDP access to the server along with credentials and port change details. It has generated a couple of log files in which I found, my admin account name, password and my natted public ip. This is how he tried to enter into my server.

So I removed RDP access and disabled internet access on the server, this is how I was finally able take over the control of my server. Since then no attack was identified.

Guys be careful it might happen with your server as well, hope this article helps you solving issues like these.

If you need any help in solving issues like these please get in touch with me.


Popular posts from this blog

VMWare Interview Questions & Tips

Windows and VMware : System Admin Responsibilities