[Discuss] Server Room Power

[gaf at blu.org (Jerry Feldman)]

On 10/13/2011 01:49 PM, Matt Shields wrote:
> NEMA 5-20 is both the connector type, but also tells you it's a 20amp 
> circuit.  To be safe and up to fire code you CANNOT use more than 80% 
> of your available power.  So for a 20amp circuit you can use up to 
> 16amps.  Keep in mind if you have two different 20amp circuits to 
> distribute the load of a rack of servers and you're hovering at 16amps 
> (actually anything more than 8-9), when you lose power on circuit 1, 
> you will trip circuit 2 because it cannot handle all the load. 
>  Remember that dual power supply servers draw half the load from both 
> power supplies, so when one fails it draws full load from the circuit 
> that's still up.
>
> Also, calculating server amperage is not an exact science.  Even 
> though Watts = Amps x Volts, it's possible that your server with two 
> 1000 watt power supplies is only drawing 250 watts (around 2 amps) 
> total right now, but when it first boots up it will spike up while all 
> the fans and drives rev at full RPM and can draw around 5-6 amps. 
>  Usually after a few minutes the amperage goes back down but it can 
> spike back up if there is heavy usage.
>
> One common mistake I've seen, is when they are installing their 
> servers (usually one at a time), they power on servers and continue to 
> watch the amps until they reach their 16amps.  So for the time being 
> their servers are online.  The problem they have is after a power 
> failure they turn every server on one immediately after another which 
> causes a huge spike and trips the circuit.
>
> A long time ago I got fed up with trying to calculate amperage, so I 
> invested in a clamp on ammeter.  Then I test my servers when I get 
> them and record the high and average usage.  I also use APC rack 
> rebooters which have a display of how many amps per circuit (not per 
> server) I'm using and have SNMP capabilities so I can run Nagios for 
> alerting and Cacti for trending.
>
Yes. I'm well aware of that NOW :-). When I set this up, I was not an IT 
guy, period. I knew how to set up a rack, but I knew little about power. 
Fortunately, our systems are not critical. Most of the people who use 
our systems are in the office or working from home or hiding in Toronto 
:-). As we got more people and got busier not only with more software, 
we also became involved with the design of a new software system being 
developed for us in Europe. Back when I set the system up, our two dual 
power supply units always ran on a single power supply. If we end up 
moving at the end of our lease or if IBM moves us I'll have to do a bit 
of planning. I don't have a clamp on ammeter, but our CDU does display 
amps, so I can play with that. At the end of our lease we could stay 
here, but we may need more space and I doubt Regus could accommodate us, 
so we could move into our own space where I would have to plan the power 
and AC in a new computer room. The AC is probably the most critical 
since I know exactly what I need for the 240V circuit. Our current 
computer room is in a Regus office and is shared by all the companies in 
the office. We have an ancient Liebart AC that does provide enough AC. 
Depending on location and facilities available, not only AC, but I would 
have to look at Internet. Currently Regus officers its clients a 2Mb 
symetric feed with horrible latency. We opted for RCN business where we 
get 60/10 cable modem that is sufficient for our needs. Moving could 
give us some choices (such as Comcast or Verizon FIOS), but I might have 
to factor in VOIP. If the computer room is inconvenient, I might have to 
plan in things like rack rebooters. We have one for the BLU. So, there 
is much more planning in our future, and this talk  has been quite useful.

-- 
Jerry Feldman<gaf at blu.org>
Boston Linux and Unix
PGP key id:3BC1EB90
PGP Key fingerprint: 49E2 C52A FC5A A31F 8D66  C0AF 7CEA 30FC 3BC1 EB90