Page 1 of 1

Server Downtime - March 16th, 2010 - Disk Migration

PostPosted: Wed Mar 16, 2011 12:03 am
by EagleRock
Hey everyone,

I am going to be performing a data migration on Delorean tonight at 1AM EST (0500GMT). The forums will be down for a short period of time (maximum 30 minutes). The server itself will be up, so you can check http://www.eagleworld.net to see if the migration is complete (I'll make a new blog post when I'm done). I'll explain this in three technical levels:

Technical Level (Byte)

The minecraft server is bottlenecked by disk i/o, as keeping track of the game world is disk-intensive. It does not hold data via database, but rather by flatfiles. As such, many reads/writes cause the game to lag. Also, I have a grief tracking database that spikes the mysql database I hold for it. So, /var and /srv (which are on the same disk) cause large amounts of iowait and lag the game server, as well as other system services.

To alleviate this, I am migrating both the mysql databases and the minecraft server directory to a new SSD I purchased. The SSD can perform sustained writes of over 250MB/sec and is about 5 times as fast as the disk I am using now. Also, this will take load off the system disk and provide much more i/o (50K iops, specifically) available to these two services.

Since phpBB runs off a mysql database, it will take down the forum during the disk migration. It will not take long, though, as I am simply aliasing /var/lib/mysql to the new mount point and editing a single mysql config. The mysql server will be up before the minecraft server, so phpBB will be back relatively quickly.

The disk I'm using is a 60GB OCZ Vertex 2 2.5" SSD, which despite being MLC, has a 3 year warranty. Stuff will be well-backed up, so I frankly don't care if I kill the drive, as I don't plan on using it for more than 3 years anyway. The disk itself now has its own volume group, which I've added two 20GB partitions on, one for minecraft, one for mysql. I'm using ext4 for the system partitions, and can easily extend them once I can predict partition growth needs.

Layman Explanation

I run a Minecraft server, which is utilizing the server disks very heavily. I have purchased a solid-state-drive, which is much faster than regular hard disks, which will house that server. I am also moving my databases (one which is used for this forum) to this disk, so I will need to have the site down during the move.

While the most benefit will be felt by the Minecraft players, the EGB forums will run much faster as well, thanks to the new disk.

"English Please" Explanation (Cobra)

I need to move stuff around. The forums will be down. Gimme 30 minutes, and it'll be quicker.

:-)

Re: Server Downtime - March 16th, 2010 - Disk Migration

PostPosted: Wed Mar 16, 2011 1:08 am
by EagleRock
Okay, I'm done. It took a LOT less time than I expected. I was able to do the majority of the migration beforehand, and moving the databases took about 60 seconds. The website itself was down for only about a minute or so...lol.

Minecraft is up, too. That was only down for about 5 minutes. All systems go!

Re: Server Downtime - March 16th, 2010 - Disk Migration

PostPosted: Wed Mar 16, 2011 12:18 pm
by SlntCobra1
FYI, I got the Layman's Terms explanation just fine. I may not know all the technical jargon in the IT field, but as far as basics, I'm just fine.

Re: Server Downtime - March 16th, 2010 - Disk Migration

PostPosted: Tue Apr 12, 2011 6:03 am
by EagleRock
SlntCobra1 wrote:FYI, I got the Layman's Terms explanation just fine. I may not know all the technical jargon in the IT field, but as far as basics, I'm just fine.


Sheesh, someone doesn't know how to take a joke. :-/

Re: Server Downtime - March 16th, 2010 - Disk Migration

PostPosted: Tue Apr 12, 2011 2:41 pm
by SlntCobra1
:p I do to, but seriously, sometimes you give me too little credit.

Re: Server Downtime - March 16th, 2010 - Disk Migration

PostPosted: Tue Apr 12, 2011 10:40 pm
by ByteSlinger
60GB SSD? Nice. Someone decided to spend a few shekels on that little upgrade. 100's of times faster than a mechanical drive, with no moving parts. Just make sure you keep that toy cool - if she overheats, it's going to be a bad time for all of us.

Out of curiosity, where did you pick that up, and how much did it set you back?

Very cool. I know that I've seen a 1TB SSD for about $3K - definitely pricey today - but in 5 years, the SATA drive will be obsolete. Mark my words!

Re: Server Downtime - March 16th, 2010 - Disk Migration

PostPosted: Tue Apr 12, 2011 11:14 pm
by EagleRock
ByteSlinger wrote:60GB SSD? Nice. Someone decided to spend a few shekels on that little upgrade. 100's of times faster than a mechanical drive, with no moving parts. Just make sure you keep that toy cool - if she overheats, it's going to be a bad time for all of us.

Out of curiosity, where did you pick that up, and how much did it set you back?

Very cool. I know that I've seen a 1TB SSD for about $3K - definitely pricey today - but in 5 years, the SATA drive will be obsolete. Mark my words!


Yeah, she's not bad. She has added some considerable beef to the Minecraft and MySQL servers on here. The SSD only set me back about $110 after taxes and rebate. Works swimmingly.

Of course, the ubergeek I am, I didn't actually mount the SSD inside the tower yet. Instead, I plopped it inside the RAID enclosure until I have to shut down the machine for something else. I've got about 146 days uptime, and I'm not stopping there! :D