How to prevent IT department overload
- 20 May, 2013 10:14
Not long ago, IT consultant Mark A. Gilmore was called in to help an IT department that was struggling with project overload. "They'd gotten this kind of attitude -- the executive vice president calls it 'Burger King Syndrome,'" he recalls. "Their approach was, 'You can have it your way.'"
The business executives believed IT could supply whatever they wanted, whenever they wanted it. Salespeople had gotten into the habit of asking the development team to create applications within a week to fulfill promises they'd made to customers. As a result, IT employees were spending about 80% of their time reacting to crises or struggling to meet impossible deadlines rather than calmly planning their workloads, says Gilmore, president of Wired Integrations in San Jose.
In the meantime, basic technology improvements weren't getting done. For example, Gilmore was surprised to discover that, though the company had a large data center with several hundred servers, there was almost no virtualization.
"You can't operate that way because it creates chaos," he says. "The quality of the work gets degraded. People's happiness level gets degraded, and it becomes a miserable environment."
Unfortunately, this very situation has become the norm in many IT departments. "It turns out to be a chronic problem," says Gartner analyst Robert Handler, who notes that his firm's research suggests that at least one-third of funded technology projects are currently in a backlog, waiting for IT to start on them. That's not a good sign, he says -- especially since there's strong evidence that overloaded IT professionals are measurably less productive than ones with reasonable workloads.
An improving economy is probably to blame for the added strain. In Computerworld's Forecast 2013 survey, 43% of respondents said they expected their IT budgets to rise this year, up from 36% last year. Sixty-four percent anticipated making a major IT investment. At the same time, 59% reported that containing costs was a priority. In the real world, that translates into a growing number of projects flowing through IT departments whose staffing levels have remained flat.
"Over the years, there had been pretty steady improvement, with backlogs going down and developer productivity going up," Handler says. "The most plausible explanation is that the credit collapse of 2008 led to companies stopping everything they possibly could." In 2010, he notes, IT productivity again began to slip, leading him to suspect techies were once again getting overloaded. Sure enough: "We started looking at other data sources and saw backlogs building up," Handler says. Piling more and more work onto IT is like pouring too much water into a funnel, he says: It works for a while, but then "all of a sudden there's too much and it makes a big mess."
Should You Reserve Some IT Capacity?
It's always a bad idea to plan more projects than your IT department has the capacity to carry out. But should you plan substantially fewer, keeping some IT work hours in reserve for contingencies?
That's what Gartner analyst Robert Handler advises. "In theory, if everyone came to the table during budget time with information on all the systems they need, it might be possible" to plan to work at full capacity, he says. "But a week or two after budgets are done, there are already a lot of requests for new stuff. We're in a complex world and there are changes constantly coming from markets and legislature. They destroy the predictability of projects."
Few IT operations are effective at dealing with the unpredictable nature of their work, says Handler. So he looked at other fields for inspiration. He found it in new product development. "Their response is to maintain reserve capacity for uncertainty," he says. He believes IT departments should do the same.
"Some business leader will say, 'We need this project to do business,' and if the CIO says, 'No, we can't, we're at capacity,' the answer will be, 'Then we'll get it elsewhere because we need it!' I suggest you reserve capacity for that situation." IT's goal should be to run at 80% of capacity, reserving the extra 20% for "things that come out of nowhere," he says.
Todd S. Coombes, executive vice president and CIO at ITT Educational Services, disagrees. "I've worked in environments where you set up contingencies, and I prefer to work based on historic data," he says. "If my data from past projects tells me I need to reserve a certain amount of time for unplanned activities, we'll work toward that, rather than assume we need to build in an extra 20% thinking things might go wrong." When people know they have that leeway, they tend to use it, he explains, adding, "I like to have things a little tighter."
Coombes says he uses detailed planning of every IT employee's time, and then has them track their activities as projects progress. "We can see historically what they actually spent their time on," he says. "It's kind of a feedback loop of planning and setting our capacity target, collecting actual information and then studying the data." That process allows for increasingly accurate planning.
This way, Coombes can plan for the unexpected on those projects that warrant it. "I don't know exactly what unexpected thing will happen, but historically I know it's going to be something," he says. "So we will build that into our capacity model. But it's based on what we know to expect."
-- Minda Zetlin
A High-Level View
How do you stop the madness? It begins with a long-term, high-level approach that takes IT's most important goals into account. Unfortunately, many IT shops aren't taking such an approach. "When I stepped into this role a couple of years ago, we probably had more than 200 projects going at any given time, but we were responding to a lot of quick-reaction type things. There wasn't much of a coherent strategy that linked all those things together," says Joe Mahaffee, executive vice president and chief information security officer at Booz Allen Hamilton, a management and IT consultancy in McLean, Va., that had revenue of $5.86 billion in 2012.
So Mahaffee and his team worked with corporate leaders to identify seven strategic initiatives they believed would be important and then plan what needed to be done to complete those projects within a couple of years. For instance, a decision to move to unified communications allowed the firm to stop spending money on extensive PBX systems. "Now if we're modernizing an office, we invest in voice-over-IP technology instead," he says.
A strategic approach won't work without the support and participation of upper management. That's why many IT departments find that establishing a governing group of some sort -- one made up of IT leaders and their upper-level business counterparts -- is the first step to taming a chaotic IT workload.
"About a year ago, we changed the model of how we govern all IT projects," Mahaffee says. "There were four governance models that had some sort of contact with IT, and we centralized all that. Now we have one governing body providing direction and helping us define priorities." That group includes Mahaffee, Booz Allen CIO Kevin Winter and leaders from each of the company's marketing teams and major departments. All in all, the group is about 15 people who meet fairly frequently. "It helps me keep alignment with the business," says Winter. "Requests get funneled to this body so decisions aren't made in a vacuum. Everybody around the table gets a say in what gets funded."
Knowing When to Say No
More important, there is top-level backing for decisions about what doesn't get funded. Experts agree: The only way to put an end to IT overload is with the support of upper-level management. One of Gilmore's first acts at the company with the overloaded IT department was to decree that IT would not take on new projects for a time. And he did that with the complete support of the company's top executive, who had heard about enough problems with technology projects to know something had to change.
"If you try to start doing this without top-level support, business group leaders will go back to the top executives and say, 'IT isn't giving me what I need and therefore I'm not meeting the goals you set for me,'" he says.
Contractors Take Up the Slack When IT Departments Are Overloaded
When work simply has to get done and IT employees are overloaded, one solution is to outsource some of the work for a new project. Contractors have their limitations -- it may not be appropriate to outsource project management, and they won't have a detailed knowledge of how a particular company functions or what its priorities are. But working with contractors does give many strapped IT departments a flexible workforce when projects pile up. "I've worked with a lot of companies who use the rule that one-third of IT project work is done in-house, and two-thirds is outsourced," reports Bruce Myers, managing director at AlixPartners.
For Mazda's North American operations, relying on IT contractors is a way of life, according to CIO Jim DiMarzio. This is partly because the auto industry in general strives to keep full-time head counts low, but using contractors also gives the IT department, which has 42 full-time employees, the ability to shrink and expand at will, says DiMarzio, noting that while his Hiroshima-based Mazda Motor Corp. is a $21 billion global business, the automaker's U.S. operation is relatively small.
"Because we knew we were head-count-constrained, we put together a strategy where most of our full-time employees are analysts and project managers," he says. "We want our staff to be the people who could run this place. We can always go find programmers when we need them."
On most projects, Mazda IT employees serve as lead analysts and subject-matter experts, while contractors do the actual coding. "While they're off doing the coding, our staff will be working on other projects. We try to prioritize so that there's a focus on a primary project and there's always a secondary project they can work on at the same time."
But when crunch times really hit, such as during model year changes or the beginning of the fiscal year, Mazda can increase contractor participation. "If we find we are out of good systems analysts, we'll take one of the smaller projects, package it up and have one of the vendors do it from soup to nuts," DiMarzio says.
When that happens, "we insist that there be fixed-price bids on those projects," he adds. "That helps make sure they stay within their time frames and pay attention to the projects. It's not a never-ending supply of money coming their way."
Mazda also gets the most benefit from its contractors by having one or two representatives of each service provider on-site, so they can get to know the company. That's important, because Mazda has its own methodology for tech projects and insists that contractors follow it.
Some contractors have become virtual employees, working on-site on an ongoing basis. "There are enough projects that we always keep them fully occupied," DiMarzio says. "We want to keep them on our account rather than someone else's account." And when IT is ready to hire someone full time, they're ready and usually willing, says DiMarzio, adding "I've converted some contractors to employees."
-- Minda Zetlin
"I typically am involved in conversations about projects that have to be delayed -- but our business leadership is also involved," says Todd S. Coombes, executive vice president and CIO at ITT Educational Services, a postsecondary education company based in Carmel, Ind., with 140 campuses around the country. "Our group of high-level executives works together well, and we're all in the discussion before a decision is made. Typically, I don't have to deliver the message at that level. I may have to at a lower level, and I don't mind because I have the backing of my boss."
When you do have to say no to a project, your goal should be for the person who hears that no to feel good about the rejection. This is especially true if you're seeking to reduce or eliminate shadow IT operations, which are typically set up by business executives who decide to take matters into their own hands when they can't get IT to provide a desired technology quickly enough. "If they hear no without having bought into how that no was arrived at, they'll get it from someone else," Handler warns.
The key is transparency. "If you have a CIO deciding what gets done and what doesn't, the people who get their projects done will be happy," he continues. "The people who don't get their jobs done, if they think the CIO was fair and really thought it through, and if they understand the reasons for the decision, they'll still be happy 80% of the time."
You might be able to enhance that dynamic by getting your company's executives to literally sign on to the process, he adds. "If you get people to agree to both the objectives of the process and the process itself, they will tend to accept it because of a phenomenon called 'commitment consistency,'" he says. That effect doubles when people agree to something without feeling forced and have done something active to signal that agreement.
Measuring IT Capacity
How do you know how much you can take on in the first place? How many projects are too many?
The only way to find out for sure is to track IT capacity -- the number of working hours available in your department. "You have a mixture of both projects that create some kind of improvement and 'keep the lights on' activities," Coombes says. "That's the demand. We have to make sure we have enough capacity to handle those lights-on jobs, and then figure out how to provide capacity for the new projects."
Coombes uses what he calls the "capacity model" to plan IT employees' workloads. "We actually plan for the period before a release what we expect for individual people working on a project, based on their availability," he says. "We plan for a full eight-hour day, but we're not going to book eight hours of development time for a developer. We may need to set aside two hours for administrative tasks and answering questions that come up. So there might be six hours available for software development."
In that case, he says, the developer may be booked to work two hours on one project, two hours on a second and two hours on a third. And that's it. His capacity for the day is used up. "That's the only way to do it," Coombes says. "Otherwise, we tend to overbook people."
"There's often this perception that people who are working eight hours a day have another eight hours available," Handler notes dryly.
The Challenge of Key Employees
Measuring capacity alone isn't good enough, since not all IT employee hours are created equal. "You need to think about it in granular terms," Coombes says. "Not only hours of work but development hours, testing, architecture, project management."
Indeed, the need to find people with both the right skills and enough free time often stops projects in their tracks. And since technical work can often be outsourced, the missing resource is usually project management and/or business expertise.
"Are you comfortable outsourcing the project management function?" asks Bruce Myers, managing director at consulting firm AlixPartners. "Some companies are fine doing that, but others aren't. That is most often the limiting factor."
"You really are constrained not only by hours and functions, but also by the expertise you have in the business context," Coombes says. "That really is the key to understanding what a subject-matter expert is. You may have a developer who's good at development work and an architect who really understands how a system is put together. But to meet the demands of the business, you have to have people who really understand the needs of the business. Those are the people who are hard to find and to hang on to."
That's why Coombes and his team sometimes review the time commitments of specific individuals when planning projects. "We ask who do we need on this project to guarantee it will be successful? A certain key individual might be needed on two or three different projects at the same time, and that creates a constraint that's difficult to deal with."
At the company with the overworked IT department, Gilmore says management had been addressing that issue with a bit of magical thinking: There were only two managers in application development so their names appeared on every project. "Any time anything new came in, one of them got put on it," Gilmore says. "They were listed to all these action items, and one of them alone took six months!"
When IT shops face such situations, there's a danger that people may wind up in roles they can't handle. "Your bottleneck might be the business analyst," says Handler. "Offshore you can get a double Ph.D. for next to nothing to do the technical work, so a lot of companies send that work overseas and keep their business analysts as busy as possible. Then when they get overloaded, they say, 'Let's get Bob to do it. He's in IT finance -- that's like a business analyst.' And then Bob makes a big mess."
The only solution, Handler says, is to know what your department's limitations are and respect them. "Most of the time, the constraining resource is humans, and a good portion of the time it's humans with technical skills. Sometimes it's cash. On rare occasions, I've seen it be conference rooms. But whatever it is, you've got to identify the constraining resource and stop approving things when it looks like you're out of that resource."
At Gilmore's client, the move to setting realistic limits seems to be working. "So far, so good," he says. "Projects are on track, resources are allocated, and people are happy."
The company's IT strategy is set for the rest of 2013, and it's planning for 2014, identifying which projects will need new hires or outside contractors. Meanwhile, business executives are learning to trust IT. "We're being honest with them and saying, 'Based on our workload, we can't get to you till nine months from today.'" Gilmore says. "But then after that period has passed, we're coming back and saying, 'Now we can start on this.' So they see it's working."
Zetlin is a technology writer and co-author of The Geek Gap: Why Business and Technology Professionals Don't Understand Each Other and Why They Need Each Other to Survive. Contact her at email@example.com.
Read more about management in Computerworld's Management Topic Center.