After Katrina, Haas gave Verizon close scrutiny because he wanted to feel confidant that its network was prepared to handle a similar disaster. He says he reviewed Verizon's own disaster-recovery plan before signing it up to provide the court MPLS network. "If a key piece of the puzzle doesn't fit, we're screwed," he says.
There were a few minor items the disaster plan overlooked, he says. For example, the plan called for extended-life batteries for everyone's laptops but not for their mobile phones.
And the backup generator was attached to the backup air-conditioning unit for the data center, but not hooked up to the primary unit. While the backup was being switched on, the data center temperatures got a little high, making it necessary to shut down non-essential machines in the data center. "There were little things," Haas says. "Not show-stoppers."
In addition having the backup data center ready to switch on, Haas also made provisions to rebuild key servers should both the primary and backups be lost. Each of the court's eight IT staffers carried 8GB flash drives that contained network diagrams, device configurations and vendor contact information so the network could be rebuilt from scratch if necessary. Even the clerk of courts who rode out the storm in Florida had one of the flash drives just in case something happened to the IT staff, Haas says.
Practice drills have become part of the routine at the courts since Katrina, and an emergency drill was carried out two weeks before Gustav hit to test out recovery preparedness.