The NSW Fire Brigades' has adopted virtualisation for its new disaster recovery strategy and in the process moved its core services from Windows to SUSE Linux.
NSW Fire Brigades’ IT infrastructure operations manager, Matthew Robey, said with 60 physical servers in a single production data centre there were increasing server costs, no more “real estate” to accommodate servers and a limited DR capacity.
“We consolidated servers using Oracle RAC and VMware, and moved from Windows to SLES 10,” Robey said.
Servicing more than 140,000 incidents per year, NSW Fire Brigades is a $500 million a year organisation with 7000 employees, 340 fire stations and some 900 fire trucks. To improve its level of DR, Fire Brigades built a new production data centre and used its existing Sydney city data centre as the DR site.
Speaking at this year’s VMware conference in Sydney, Robey said the Fire Brigades is now using EMC mirrorview for DR with 2.5GB, dual path fibre links between the sites.
There are five Dell R900 servers in a VMware ESX cluster at each site and three Dell R900 servers for Oracle RAC at each site.
After a three-month project to go live with the new DR infrastructure, NSW Fire Brigades is now using VMware Site Recovery Manager (SRM) for centralised recovery.
Robey said this works well as DR tests can be run without actually performing any DR and virtual machines can be prioritised to start up depending on resources.
Not all point-and-click
While the new DR process is a big improvement at NSW Fire Brigades, Robey said there are some outstanding issues with the software.
“Failover is automated, but not automatic so someone still has to make a decision to do DR,” he said. “SRM runs on Windows and is not set to automatically restart and Linux machines with raw devices doesn't work, but maybe in the next release it will.”
“Fallback also requires manual intervention, so it's not a five-minute job.”
The “old school” backup procedure where every vm is a physical server and backed up over the network is still being used, but Robey is keen to do SAN, rather than network, backups and use VMware’s continuous availability options.
Shane Moore from EMC said one of the biggest motivators for adopting virtualisation software is to do better DR and business continuity.