Conquer Club

Prevent Server Outages

Suggestions that have been archived.

Moderator: Community Team

Prevent Server Outages

Postby bedub1 on Mon Oct 13, 2008 2:30 pm

Twill wrote:Update #3:

We're going to probably have to go back through some backups, dig out just off-topics and the lost usergroups and restore them individually.

This probably wont happen until Tuesday (it's thanksgiving weekend here in Canada and so our families seem to want to spend time with us for some odd and unknown reason), so hang tight until then, off-topics and your clan forums will be coming back as soon as we can get them there.

We've heard conflicting things from the data center - we've been variously told it's the memory controller, then the hard drive something or other and then finally "sorry if the chiller failure caused you any problems"...which is what took out the database 3 months ago.

I'll post more when I know more.

Twill

Ensure there is a battery backup for the RAID controller write caching. Raid controller cards typically have 128-512mb of ram onboard, and require a battery to keep the data in the ram in case of problems. If the database is in the process of a write, and it fails, with the battery the data will get written to the drives when the system comes back up. If the battery is dead or not there, then when the system comes back online, the data will be lost, and this can cause serious and random database corruption.

EDIT: The battery is a little battery inside the server screwed to the raid controller card...it's not a UPS for power for the entire server etc...
Colonel bedub1
 
Posts: 1005
Joined: Sun Dec 31, 2006 4:41 am

Re: Prevent Server Outages

Postby blakebowling on Mon Oct 13, 2008 8:40 pm

You do realize that the RAID failure was just an excuse, their cooling systems keep going offline (Rackspace's, not CC's)
Private blakebowling
 
Posts: 5093
Joined: Wed Jan 23, 2008 12:09 pm
Location: 127.0.0.1

Re: Prevent Server Outages

Postby hecter on Mon Oct 13, 2008 10:23 pm

Perhaps they should upgrade to some sort of fancy liquid cooling system? Ya... And put Kool Aid in it! That'll keep the temperatures down, fosho!
In heaven... Everything is fine, in heaven... Everything is fine, in heaven... Everything is fine... You got your things, and I've got mine.
Image
User avatar
Private 1st Class hecter
 
Posts: 14632
Joined: Tue Jan 09, 2007 6:27 pm
Location: Tying somebody up on the third floor

Re: Prevent Server Outages

Postby hwhrhett on Mon Oct 13, 2008 10:25 pm

hecter wrote:Perhaps they should upgrade to some sort of fancy liquid cooling system? Ya... And put Kool Aid in it! That'll keep the temperatures down, fosho!



or some nice hawaiian punch, thatll make it faster right?
Image
User avatar
Cook hwhrhett
 
Posts: 3120
Joined: Fri Jun 02, 2006 8:55 pm
Location: TEXAS --- The Imperial Dragoons

Re: Prevent Server Outages

Postby bedub1 on Mon Oct 13, 2008 11:36 pm

blakebowling wrote:You do realize that the RAID failure was just an excuse, their cooling systems keep going offline (Rackspace's, not CC's)

yes, and cooling causes cpu's to lock up mid-stream...and thus the entire system, and then the shit in raid ram gets dumped instead of being written to the drives.
Colonel bedub1
 
Posts: 1005
Joined: Sun Dec 31, 2006 4:41 am

Re: Prevent Server Outages

Postby blakebowling on Tue Oct 14, 2008 12:34 am

bedub1 wrote:
blakebowling wrote:You do realize that the RAID failure was just an excuse, their cooling systems keep going offline (Rackspace's, not CC's)

yes, and cooling causes cpu's to lock up mid-stream...and thus the entire system, and then the shit in raid ram gets dumped instead of being written to the drives.

My vote is to leave Rackspace altogether and move the servers elsewhere
Private blakebowling
 
Posts: 5093
Joined: Wed Jan 23, 2008 12:09 pm
Location: 127.0.0.1


Return to Archived Suggestions

Who is online

Users browsing this forum: No registered users