Click here for M.B. Foster sponsor message
                
                Staying Online Through Days of Ice
                
                Worst ice storm in 50 years cant keep 3000
supplier M.B. Foster
                offline
                
What started as a flicker late on the night of January 7
escalated
                into a complete outage covering thousands of Canadian customers
                in the eastern part of Ontario where the company is
headquartered.
                Birket Foster said his company had to scramble when they
learned
                after 48 hours of no power that the utility might not be able
                to restore electric service for up to three weeks.
                
Residential and business losses in Canada were in the $1
billion
                range from the storm, a multi-day affair that delivered six
inches
                of ice to knock out 600,000 customers power,
including much of
                the Ottawa River valley in eastern Ontario and western Quebec.
                M.B. Foster is headquartered in Chesterville, the second
largest
                
employer in a village of 1,600 about 30 minutes from Ottawa,
deep
                in Canadian farmland.
                
While technical support came to a halt at places like
Air Canada
                in Montreal as a result of the storm, M.B. Foster used employee
                skills that dont get exercised everyday to power its
systems,
                phones and offices through the blackout. 
                
Thank the lord for the staff we have, Foster
said. One is a
                certified electrician and Ontario Hydro linesman [Trevor
Treurniet],
                one is a mechanic, and the final touch is an R&D guy
who was a
                radar technician. There was a guardian angel looking after M.B.
                Foster Associates. 
                
The R&D staffer, John Middelvein, knew a ton
about both oscilliscopes
                and old generators. He was able to synch the generator up at 60
                hz so we could run the computers and phones.
                
Once it became apparent that electric service to the
companys
                campus wouldnt be restored, a marine generator became
the primary
                responsibility of Rob Foster. The 5KV generator wasnt
delivering
                power at the correct frequency, however, keeping it from
serving
                the three HP 3000s that run support, development, e-mail
and customer
                tracking systems for the company. Middlevein, an NT support
technician
                who had joined the company only in November, had military
experience
                with the particular model of oscilloscope M.B. Foster had
borrowed
                to condition the generators power.
                
While many businesses endured one- to two-week shutdowns as a result of the storm, the teamwork at M.B. Foster had the company online after a long weekend with little sleep. Foster said he turned some of his staff back to the community to do door-to-door checks on residents, ensuring residents had food and fuel for heat or helping them to a local shelter.
The problems of finding adequate fuel for generators
(diesel comes
                in two grades, and only one is useful in below-freezing
conditions),
                getting the generator warm enough to start (a local dairy
farmer
                across the road warmed it up in a barn) and adjusting power to
                control harmonics for computers got solved over the
weekend. More
                than a week after the blackout the local utility restored
power,
                but delivered it with a reversed phase  which made
the dairy
                farmers milking machines run backwards, and would
have fried
                out M.B. Fosters computers. A warning from across the
road saved
                the equipment, just one of the many good things that
happened
                as a result of the storm.
                
We have the makings of a disaster recovery plan now, Foster said, using the 5KV generator that now kicks in automatically on a blackout. I wish it would have been planned first and disastered later, but the plan has the benefit of being built under sustained disaster conditions. We all got a PhD in power, he said.