As my colleague Martin Heller recently observed, smart coders always optimize the slowest thing. Trying to optimize every trivial performance issue in your code is just chasing your own tail. You should find the one problem that's causing the biggest performance hit and fix that first.
But what if you go over and over your code and you can't find what's wrong? In fact, what if it turns out that there isn't anything wrong with your code? What if "the slowest thing" is actually the code supplied by your vendor?
That's exactly the situation that Vipul Ved Prakash discovered when he started tinkering with one of his company's Linux boxes. For some reason, Perl code on the server seemed to be running at least 100 times slower than expected. To his surprise, Prakash found that the parts of his application that were eating up the most CPU involved "bless" and "overload" -- core Perl language functions that are used in the process of instantiating new objects.
Hold your flames, please. This isn't a story about how object-oriented Perl is slow and inefficient, because it isn't -- at least, not normally. Prakash had never encountered a problem like this before. Identical code ran on his MacBook without a hitch.
So what was the problem? Simply put, the low-performing instance of the code was running on a CentOS Linux server, using Perl packages built from code maintained by Red Hat.
This Perl is a lemon
Prakash has posted a detailed description of the problem on his blog. To make a long story short, he got rid of the Perl executable that came with his CentOS installation, compiled a new one from stock source code, and the bug disappeared. Clearly, the Perl hackers are blameless in this case. The fault lies squarely with Red Hat for distributing a buggy version of the interpreter.
What's more disturbing, however, is that it turns out that this Red Hat Perl performance issue is a known bug. It was documented and verified long before Prakash ever raised a stink about it. How long? Try 2006, according to Red Hat's own Bugzilla database.
The current bug has been open since late last year. That's bad enough, but similar slowdowns were reported in 2007 and 2006, in versions of Red Hat's code base dating back to Fedora Core 7. Nonetheless, despite being characterized as "medium severity," the priority of fixing the current bug is listed as "low."
That's unfortunate, because the impact of this bug could be far-reaching. Red Hat Enterprise Linux customers are undoubtedly susceptible, as are users of Fedora (Red Hat's community-maintained development branch) and CentOS (a free Linux distribution built from Red Hat's source code).