Author Topic: October 22 events  (Read 644 times)

Slash

  • Priest of Roguetemple
  • Administrator
  • Protector of the @
  • *****
  • Posts: 1126
    • View Profile
    • Slashie.net
    • Email
October 22 events
« on: October 22, 2016, 01:01:13 AM »
The forums had been unavailable for about 2 days, likely due to the common issues we had had previously with the server (disk full because of logs most of time)

However, on October 22 the server was "suspended" by VPS.net, they suspected it was compromised by the Linux.Isiport trojan (https://www.symantec.com/security_response/writeup.jsp?docid=2016-030913-4222-99&tabid=2 ) which would allow the server to be used for DDoS attacks (likely because of the massive attacks that happened that same day) so they took it down. After some talking with their support team I was able to get the node restarted in recovery mode so I could go in and try to fix and recover stuff.

Luckily I was able to recover the most precious data which was the forums database. I almost screw up since in order to dump the data I had to reinstall mysql server and the data files for the pre-existing databases were not overwritten (they were safe on a mounted volume). That was a very close call, for some minutes I thought we had lost the entire forums database.

Then I was forced to do something I had pending doing for a long time: Moving again the forums to a new private server.

We are now running in our own node at digital ocean, this node is EXCLUSIVE for roguetemple, thanks to your contributions. This will help isolate any errors and prevent other apps from taking the site down. I'm also going to set up automated backups so our data is safe. As always please contribute if you can to make our infrastructure ever stronger! http://roguetemple.com/support/
« Last Edit: October 22, 2016, 04:53:32 PM by Slash »

Aleksanderus

  • Bishop
  • ***
  • Posts: 180
  • Champion of the unknown
    • View Profile
Re: October 22 events
« Reply #1 on: October 22, 2016, 02:30:39 PM »
Holy shit, dude! That was dangerous, but it's good that you were able to recover data!

jere

  • Bishop
  • ***
  • Posts: 222
    • View Profile
    • Email
Re: October 22 events
« Reply #2 on: October 26, 2016, 12:54:06 AM »
Great job Slash. That sounds absolutely terrifying though.
Golden Krone Hotel -- available on Steam Early Access now