FAQ  •  Register  •  Login

Server Downtime - July 31 - August 12, 2012 - Power Outage

<<

EagleRock

User avatar

Chief Barracks Officer
Chief Barracks Officer

Posts: 487

Joined: Fri Jun 04, 2010 1:21 am

Location: 127.0.0.1

Post Mon Aug 13, 2012 9:15 am

Server Downtime - July 31 - August 12, 2012 - Power Outage

Hi, everyone!

Sorry for the extended downtime...the EGB forums were unavailable for a while thanks to a power outage that occurred on July 31st, 2012.

While I was on vacation in South Carolina, a heavy storm hit Northern NJ, which caused many people to lose power completely for 4-6 hours. While I thankfully live very close to my power station (and therefore almost never lose power), I was getting brownouts and quick blips of outages for a while. As such, my UPS kept switching back and forth between running on the mains and running on battery. Eventually, the battery hit a sufficiently low level of power and initiated a shutdown of Delorean.

Thankfully, Delorean shut down successfully and gracefully, but the server never came back up. While I have my system configured to power on whenever it senses the power come on, the power never was fully lost. The UPS did hold the whole time...it just hit the 10% battery reserve threshold. Normally this wouldn't be an issue, but since I was away between July 26th and August 12th, nobody was there to hit the power button.

Anyway, the server is back up and running, and appears to be glad for the break. The system heatsink was very caked with dust, and I was able to disassemble the hardware a good amount and clear out a good bit of dust. Also, I was able to get the solid-state drive I installed a while ago for the Minecraft server fully installed inside the chassis (not just inside the external disk array).

OS-wise, the system is fine (not that I was really worried that Debian would fail, but after almost 700 days of updates and no reboots, sometimes you worry if you somehow uninstalled the kernel or something stupid...). The system is now running on Debian stable's latest kernel and everything is running just fine. The only issue at boot time was that the partitions all needed filesystem checks, which took about an hour to complete.

The only issue left at this point is that the backup disk array came up degraded, so I had to manually reintroduce the second RAID disk and initiate a resync. That will finish in about 6 hours or so, and I don't really want to bring up most of the servers until that's done. However, the EGB Forums will not be affected, as the web server and mysql array it uses do not cause much of a load on the system.

Sorry for the huge downtime, folks!
Image
<<

ByteSlinger

User avatar

Master of the Portal
Master of the Portal

Posts: 687

Joined: Thu Jun 17, 2010 9:40 am

Location: East Coast of the USA

Post Thu Aug 16, 2012 4:26 pm

Re: Server Downtime - July 31 - August 12, 2012 - Power Outa

Glad to hear that all is well with you and your hardware. Saw on FB that you were having a grand old time on your vacation with a very fetching young woman. Good to see you smile and be happy again, my friend!
:D
Image

Return to Server Info/FAQ's

Who is online

Users browsing this forum: No registered users and 24 guests

cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by Vjacheslav Trushkin for Free Forums/DivisionCore.
Americanized by Maël Soucaze.