Microsoft has told business customers that when they run into problems with Windows 7's new patch maintenance regime they should file a ticket with the company's support desk.
But a pair of patch experts doubt that that -- or Microsoft's other remediation suggestions -- will prevent the new cumulative updates from disrupting business.
"Microsoft's response is to open a support case to alert [the company] to the issue," said Susan Bradley in an email reply to questions. "I challenge Microsoft to open a support case and pretend to not be an enterprise customer with Premier Support, and call [the toll-free number] and open a case. It takes the patience of Job to work with the outsourced vendors that Microsoft has hired."
Bradley, an IT professional, is best known for her column on patching in the Windows Secrets newsletter. She is also the moderator of the PatchMangement.org mailing list, where business IT administrators discuss update tradecraft.
Microsoft's advice was included in a post to a company blog last week, when it elaborated on the massive change to the servicing model for Windows 7 and Windows 8.1. The new plan was announced in mid-August.
Rather than offer individual updates -- which have patched one or more discrete vulnerabilities -- starting today Microsoft will issue cumulative updates for Windows 7 and 8.1. The cumulative model, which debuted last year with Windows 10, provides the contents of all previous releases along with the new fixes. More importantly to enterprises, however, is that cumulative also means that the updates cannot be broken into their parts.
For decades, Windows users have been able to apply an individual patch and reject others, or accept most fixes but block one or more that had proved flawed or even dangerous. That practice has been invalidated by the conjoined nature of the new updates.
The loss has been among the most debated -- and criticized -- aspects of Windows 10, and IT professionals have voiced the same unease about today's change to Windows 7 and 8.1.
Microsoft responded to the concerns with some advice last week. "If any issues are encountered, we recommend stopping or pausing deployment of the update and contacting Microsoft Support as soon as possible," wrote Michael Niehaus, a director of product marketing, in a blog post. Further steps, Niehaus continued, include, "rolling back the update on affected machines while the issue is being investigated" and "working with the publisher (ISV) for an affected application."
Neither of those steps is revolutionary: The former has been the go-to response to a buggy update since personal computers were invented.
But with cumulative updates, either solution will be dicey said Chris Goettl, program product manager for patch management vendor Shavlik. Rolling back a Windows 7 update will pose a devil's dilemma: Apply the update and break something, perhaps a business-critical application, or roll it back, leaving who-knows-how-many-other vulnerabilities unpatched.
Nor will it always be possible to "work with the publisher" of an affected application. "Look back at January, when a Windows 10 cumulative update broke the Citrix [WorkstationOS Virtual Delivery Agent]," Goettl said. "Citrix was big enough and was able to react fast enough" to the incompatibility between the Windows 10 update and its software to generate a software update. "But that's not going to be the case for everyone."
Goettl ticked off the kind of ISVs that don't have the resources to jump on a problem caused by an OS update, including small publishers, niche publishers such as those that write software for medical devices, and finally, those long out of business. Alternately, a publisher may have already dealt with the underlying problem that led to update-application incompatibility, but packaged it in a newer version that comes with a price tag.
The new patching model, Goettl predicted, "Will have a lot more impact on [software] vendors supplying companies. There are a lot who cannot react very fast."
Nor was Bradley counting on Microsoft itself to always rapidly react to quality-control problems with the new updates.
"The investigation process [with Microsoft support] is not fun, not efficient and often takes several days for the support team to understand the issue and repro[duce] the problem," she said. "Then it will take several more days for the known issue to be documented in the KB [knowledge database] and often longer still for a note to be posted to the KB."
And for many customers, Microsoft's advice was expensive, Bradley noted. It costs $499 to open a case if a customer is not on a support plan. Although that fee should be automatically refunded if the problem is in a security update, Bradley said that in several instances she has had to formally request the refund before receiving it.
"Not all issues are found by enterprises with TAMs [technical account managers employed by Microsoft] and support reps on speed dial that have key contacts to make the support process a breeze," Bradley said. "Some of the issues are found in the community where people do not relish paying (even temporarily) Microsoft a fee of $499 to tell them they have a bug in their code."