The Sherlock Holmes Guide to Programming, Debugging, and Performance Tuning
Years ago, I wrote about my Three Laws of Programming but nobody cares about my “laws.” Maybe they would take more notice if I attributed them to…
…Sherlock Holmes. Is this a type of reverse plagiarism?
I will state as a matter of belief that Holmes would have made an excellent programmer if he had lived one hundred years later. And, yes, I do realize that he never actually lived at all. The various pronouncements that he makes to Dr. Watson, his amanuensis, are evidence that he would have approached programming in the same systematic and thorough manner as he did detection.
There are perhaps three related disciplines when it comes to developing software, starting with “programming” or, as it is fashionable to call it these days, “coding.” The first is design — where a programmer connects abstract data types together in order to satisfy a particular use case. Ideally, this design phase might consist of three phases: diagramming (for example, UML, E-R, sequence diagrams, etc.); writing the unit tests; and then coding (translating the designed logic into a programming language). I would guess from my own experience that most people just skip the diagramming phase. And many people wait until later to write unit tests — a capital mistake, in my opinion.
The tests are run, and any non-conformity (a “bug”) is noted. These bugs are the crimes that have been committed, and which await detection of the perpetrators, or causes, of the bugs. This detection process is of course more commonly known as debugging, our second discipline.
The third discipline is performance tuning. Of course, if the design process was done well and the code reflects that design correctly, then there should be no performance tuning to be carried out. But, it is when the unit tests are all working, and the integration tests are run, that sometimes a performance requirement that was not originally noted may arise.
Sherlock Holmes’ guidelines will of course mostly apply to the detection (debugging) phase.
From The Sign of Four, one of the early novellas, he writes (Holmes’ first law):
“Eliminate all other factors, and the one which remains must be the truth.”
Debugging typically consists of “ruling out” possible causes — just as a medical clinician might do — until only one possible cause remains. By Holmes’ first law, this must therefore be the cause.
Holmes makes a very similar observation in The Adventure of the Beryl Coronet:
“It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth.”
My own original version of this “law” was expressed slightly differently: If you can’t find the bug where you’re looking, then you’re looking in the wrong place. I have spent many hours looking at code where I thought the problem “must” be only to realize later that some other part of the code was misbehaving. The trick is to eliminate each candidate problem as quickly as possible.
It is all too easy to assume that some part of the code (which was “working before” or which has been tested by someone else, etc.) is perfect. Sherlock Holmes puts it well in The Adventure of the Reigate Squires:
“Now, I make a point of never having any prejudices, and of following docilely wherever fact may lead me, …”
This observation applies manifestly to performance tuning also. It is so easy to make assumptions, such as “if I cache this, then the performance must improve.” Never do any such thing without testing for its consequences.
When faced with a plethora of possibly conflicting results, it is important to know which you should trust the most. For example, you cannot trust the order in which buffered I/O occurs. If you need to be sure of the order, then you should use logs or unbuffered I/O.
Sherlock Holmes summed it up thus, again from the same story:
“It is of the highest importance in the art of detection to be able to recognize, out of a number of facts, which are incidental and which vital. Otherwise, your energy and attention must be dissipated instead of being concentrated.”
So, to employ our example above, the order of buffered output is incidental whereas the order in the logs is (usually) vital. By the way, this is a good advertisement to use functional programming with effects because you can rely on the order of such effects.
A second “law” has to do with the situation you sometimes find yourself in where there are two seemingly independent problems with your code. Let’s say you are concentrating on problem A, which is proving challenging but so far intractable, while you are aware of an apparently minor problem B for which you think you have a simple solution. It’s tempting to concentrate your efforts on the more interesting problem (A). But you would be well advised to take a slight detour and fix problem B. You never know: that fix might also be the solution to problem A (it’s happened to me many times).
Holmes understood this of course (his Second Law), as evidenced by this comment from The Adventure of the Musgrave Ritual:
“‘At least,’ said [Holmes], ‘it gives us another mystery, and one which is even more interesting than the first. It may be that the solution of the one may prove to be the solution of the other.”
The Third Law relates to the practice of peer programming. I can’t count the number of times I’ve asked someone for help and then, midway through explaining the background of the problem, I’ve realized my own error. Holmes was aware of this phenomenon too, for he states in The Adventure of the Blue Carbuncle:
“Not at all. I am glad to have a friend with whom I can discuss my results.”
And, even more explicitly, he discusses it in The Adventure of Silver Blaze:
“At least I have got a grip of the essential facts of the case. I shall enumerate them to you, for nothing clears up a case so much as stating it to another person, and I can hardly expect your co-operation if I do not show you the position from which we start.”
A certain amount of imagination is also extremely helpful when trying to solve a problem. If you imagine a particular scenario, it may follow that the currently mystifying behavior of your code comes to be a natural outcome of your imagined situation. Again from Silver Blaze (incidentally, one of the very best stories, in my opinion):
”See the value of imagination,” said Holmes. “It is the one quality which Gregory lacks. We imagined what might have happened, acted upon the supposition, and find ourselves justified. Let us proceed.”
Sometimes a clue comes to you not from observed behavior but from expected behavior that you do not observe. Many’s the time I have instrumented some method with a log message or unbuffered print statement only to find that I get no output whatsoever. This usually is enough to tell me that, despite my expectations, the method was never actually called. One of the most famous exchanges of Sherlock Holmes covers this point (again from Silver Blaze):
[Inspector Gregory] “Is there any other point to which you would wish to draw my attention?”
“To the curious incident of the dog in the night time.”
“The dog did nothing in the night-time.”
“That was the curious incident,” remarked Sherlock Holmes.
Let us now return to the third passage quoted above, having to do with casting any prior prejudices aside. I would venture to suggest that this is perhaps the most important guideline that Holmes gives us: to carefully gather as much of our evidence as possible before forming a theory. He sums this attitude up in the very first of the Sherlock Holmes stories published in the Strand Magazine — A Scandal in Bohemia:
“This is indeed a mystery,” I [Watson] remarked [to Holmes]. “What do you imagine that it means?”
“I have no data yet. It is a capital mistake to theorise before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”
His basic theme is similar in the following statement from The Sign of Four:
“No, no: I never guess. It is a shocking habit, — destructive to the logical faculty.”
I hope that these utterances of Sherlock Holmes will help you take the proper course of action when presented with a problem in programming, debugging, or performance tuning.
