Saturday, November 9, 2013

When all you have is a hammer, everything looks like a nail.

I frequently run across questions like this one on Stackoverflow.  The person is asking how they might avoid hand-coding complex INSERT statements, using TADOQuery.   My comment in this question reads:

You might find it easier to use a dataset (TADODataset or TADOTable) to do a dataset-like-job, and use a TADOQuery to do a query-like-job. Oh, and there is TADOCommand to run a Command. So if you're writing SQL "insert" strings, you might want to look at running those with TADOCommand. And you might want to avoid writing them at all, and just set field values and insert into a TADODataset.
 This is a common coding anti-pattern:

The "All I Have Is A Hammer, So That Must Be a Nail" Anti-Pattern:

  1. Try something, it works on Tuesday July 1st, 2001 to solve the problem you had on Tuesday July 1st, 2001. 
  2. This is now the Standard Way You Do Everything Until the Day You Retire Forever from Coding.
  3. Do not reflect on whether you are forcing round pegs into square holes, but simply continue to develop a large body of "cargo cult" programming practices that were functional once, and are optimal almost never.

Why do we developers do this?  Because we know how to build one solution to a problem, we often stop looking for more ways to do this.    Learning about using the IDE (especially the debugger), and learning about using (and writing) components, understanding the entire Pascal language, these are all dimensions of learning to be a Delphi developer.   As a single developer learning, the pitfall is to stop too soon.

As a team, this pitfall morphs into a collective trap.  Developers working on a team must communicate "rules" and "best practices" to each other, and must agree and work together.  Working together is a whole other topic that I wish to brush past, but let's just do that, and ask, "how do teams create and modify the set of established rules and conventions they use, and do they ever fall into trap B while they try to avoid falling into trap A?".   Do rules that teams make, ever become "dogma"?  Are they written without qualifiers, or exceptions?  If so, then your team practices can force developers into the "Everything is a Nail", so "Hit it with the Hammer" trap.
 
I can list about 20 common "Hammer/Nail traps" that I have seen developers fall into, and they usually involve the word "always" or "never", and seldom involve the words "think about it", or "use when appropriate":

  1. Never use EXIT
  2. Always use EXIT
  3. Never use GOTO
  4. Always use GOTO
  5. Always put begin and end and never use single statements.
  6. Never put begin and end around single statements.
  7. Always use Data-Aware-Controls.
  8. Never use Data-Aware-Controls. (See Footnote)
  9. Always use Test Driven Development.
  10. Never use Test Driven Development.
  11. Always use Objects, Generics, and Gang-of-Four Patterns, and write your Pascal code like it was Java.
  12. Never use Objects, Generics, or Patterns, and write your Pascal code like it was Turbo Pascal and it's 1989.
Why do developers get stuck in the "always do X" and "never do X" patterns? Why do we argue and fuss about these rules?  I would like to suggest a constructive idea about why:

Solving problems is hard.  When the solution space (the number of things you have to try) gets large, your brain shuts down.  By closing down the "Go Left, Go Right, and Go South" rules in your brain, and leaving only the "Go North" rule, as the One Rule you always follow, until you hit a wall, at which point, your "Rotate Counterclockwise 90 degrees" rule kicks in, developers find it easier to navigate the complex maze of decisions that we make every day when we seek for solutions to our coding problems.   When a solution has worked many times in the past, we sometimes promote our solutions into dogma, like this:

1.  Once, I found it confusing that someone put an exit statement in the code, and I didn't notice it when reading the code, and so the flow of the program was confusing to me as a human being even though it was not confusing to the compiler.
2.  Due to my inability to read and see Exit statements, they are similar to Goto statements, and since they are similar to Goto Statements, and Goto Statements are "Considered Harmful", as everybody knows, they go on the no fly list.

Now let's look at code written without exit statements:

procedure  TMyForm.MyButtonClick(Sender:TObject);
begin
  if ValidationFunction1(param1,param2,param3) then
  begin
     if ValidationFunction2(param4,param5,param6) then
    begin
        if ValidationFunction3(param1,param2,param3) then
        begin
        if ConfigurationStateDetection(param1,param2,param3) then
        begin
           DoSomething;      
        end
        else
        begin
           DoSomethingElse;
        end;
    end;
  end;
end;

This could code be written better as:

procedure  TMyForm.MyButtonClick(Sender:TObject);
begin
  if not ValidationFunction1(param1,param2,param3) then
    exit;
 
  if not ValidationFunction2(param4,param5,param6) then
    exit;

  if not ValidationFunction1(param1,param2,param3) then
    exit;
  
  if ConfigurationStateDetection(param1,param2,param3) then
  begin
      DoSomething;      
  end
  else
  begin
      DoSomethingElse;
  end;
end;

The above is intended to be a sample that is an order of magnitude less complex than the worst "nesting of begins and ends" that I have seen in the field, at places where "exit" is discouraged or banned.   Practice 1 (which has some merits, I understand about exit being confusing to developers), causes Problem 2.

If you're going to make the rule "no exits", then you should have a better solution for a flow chart like this, that is better than nesting 30 blocks deep with begin and end:



There are exit-free solutions but most of them are far more baroque than just using exit, and if you are comitted to your rule "no exits" you should choose one, and make sure people know how to construct something that is less of a mess than a block of 30-nested begins and ends.  But then, when you're done, ask whether that finite state machine engine you invented isn't just your brain hiding an Exit-like and Goto-like language statement underneath some new layer of bafflement.

Let me suggest a better set of rules, either with the always/never removed, or at least, weakened with an appeal to developer rationality.



  1. Never use EXIT, when something better exists.
  2. Always use EXIT, when no better solution exists.
  3. Never use GOTO, when something better exists.
  4. Always use GOTO, when no better solution exists.
...  I think I can leave the rest of the list as an exercise for the reader.

So here is my Rule about Coding Rules:

A.  When you think of a coding rule or best-practice, keep your mind engaged while using that practice, and look for places where that practice creates as much or more trouble than it solves.
B.   When you share your ideas about coding rules within your team,  add conditions like the ones I added above in blue, that make it clear to team members that these are not Cargo Cult practices, but active thought processes that the team uses to develop software.

As a single developer in a team, it's your job not to create endless waves of complaints, but when you see something that is not working, or which is causing problems, it's important to find ways to discuss these issues.  At all times, you should avoid insults or personal comments.  If the team's best practice is to indent or format or organize code in some unusual way you've never seen before, and which you find yourself unable to understand and deal with (something I experienced personally at one place I worked),  you may find yourself tested to the very limits of your ability to cope with the zeitgeist at that team. I know I did.  If you're like me, and you like there to be a "why" or a reason for things being the way they are it may frustrate you to no end that people are in fact, not automatons whose behaviour can be predicted or explained by mere rational analysis, and often do the same things over and over for reasons that may be inscrutable to you, not knowing or caring why they do them this way. I'd like to reiterate why teams are like this:

People do things the same way over and over because that's how the human brain works.

Becoming aware of this, and trying to point out that people are hammering screws into plywood with a hammer, because all they've ever used before is a nail, is a bit of an extreme metaphor, and seems insulting, really, at some level.  But we have to remember that "nails" may have been 99.9% effective in all the places where developers have tried to use them.  Your team might think "TADOQuery is good, and TADOCommand and TADODataset and TADOTable are bad", for instance, because that got you out of some bad situation once, and now that is your rule.  But ask yourself, are you being introspective and is your team able to discuss this, or do you just "put your head down" and not ask questions, not think outside the box.

Going back to the StackOverflow question that inspired this post, I would like to point out that I am not trying to pick on a new Delphi user who is just learning.   But I do find that I myself sometimes fail to learn all that could be learned about a tool or an environment, and I tend to repeat using the same pattern or solution myself, and that I notice this problem in myself, and that I humbly offer this reflection to you, if it is of value:

Ask yourself, every day: "Are there ways to do this that I have not thought of?", and "If I don't know a better way but I sense something is off here, can I ask a colleague for suggestions, and could we find some more efficient or more optimal way to do this?".    So, in the end, the StackOverflow person who asked this question is more right, and more of a good example, because he or she asked a question, and got a lot of feedback from the big crowd of Delphi Geeks on the Internet.

That might be the smartest development best-practice of all.



11 comments:

  1. Few of us have a deep understanding of the entire Delphi language and the idiosyncrasies of its many libraries, frameworks, and components. It is ordinary human behavior to settle into patterns, and our patterns are influenced by the particular environments in which we labor. For example, if I spend years working with vendor A's components, and then move to a job where vendor B's (mostly equivalent) components are used, I will need to reconsider many of my patterns, as not all comparable components implement the same behaviors. This is disorienting, at the outset, but is probably beneficial in the long run.

    I pointed out to a colleague the other day that one of the benefits of Delphi was the absence of multiple inheritance. He questioned that, but I pointed out that one of the uglier aspects of the Delphi with clause is when someone writes with a, b do... Setting aside the debugging issues, and ignoring that the code is difficult to read because we must mentally supply the missing qualifiers in order to comprehend it, what if there is a member name collision in the a and b objects? That he was able to understand immediately, and apply to the question of multiple inheritance.

    All the nevers and don'ts will not prevent bad coding; they merely limit its range.

    In recent history we have read many comments form Nick Hodges, as he was refactoring some remarkably bad code. None of us, I suspect, started out writing code well. We all had to learn and grow, as in any other aspect off life. I suspect, as well, that we have all had the experience of revisiting code we had written some years earlier, and finding it odd to read, and possibly even embarrassing. That's another measure of growth, and couples neatly to the question of what we do after learning to apply new methods. Do we revisit hoary old code which could benefit form a rewrite? Or do we close our eyes and leave that to others?

    Code needs maintenance; entropy increases. These are hard realities. But a real test of our skills is how we respond to requirements, not only on new designs, but perhaps more importantly, in maintenance.

    ReplyDelete
  2. Wow very intelligent piece, well written. The fact that human brain just tend to go into this trap (if it is a fact but evidence shows that it is) makes this post a worthy reminder :)

    ReplyDelete
  3. Thank you for the good read, we all fall into these traps.

    ReplyDelete
  4. How about this one. Never use a dark blue font on a black background. (Yeah, I'm referring to part of this post).

    ReplyDelete
    Replies
    1. I wrote a small piece of script for this page. It replaces the background to white and the white text to black. Then this page really looks good and everything is readable way way better.

      Before the page was like an acid attack on my retina...
      But I would like to read the contents as it is much better than the way it is displayed.

      And you are right, the blue text is nearly unreadable on the black standard background unless one marks it...

      Delete
  5. Define "dark blue"? I see at least a 5:1 BRIGHTNESS difference between the blue I used and the black.

    ReplyDelete
    Replies
    1. Reminds me of "but it works on my computer".
      If you write for your readers, make it easier for them to read, not harder.
      I too had to select the paragraph to be able to read it.
      UI 101: Do not rely on color differentiation. Some people are color blind.

      Delete
  6. I won't define dark blue, but I will say that the blue you used is more effective on some monitors than others. However, it is well to keep in mind that the human eye does not resolve all colors to the same degree of resolution. As it happens, blue appears less sharp than the other primaries. Also, in television, blue is only 11% of the signal content, and has the lowest bandwidth because giving it more is wasted.

    In general, it is best if the text presents at least a 50% differential to the background in terms of its monochrome value. You can derive that as M = 0.3R + 0.59G + 0.11B.

    ReplyDelete
    Replies
    1. Try this inside the console of your browser (F12 normally):
      document.body.style.backgroundColor = "#FFFFFF";
      document.body.style.color = "#000000";
      document.getElementsByClassName("date-outer")[0].style.backgroundColor = "#FFFFFF";

      Not perfect, but much better in my opinion.

      Delete
  7. I'll try to avoid any less than 90%-brightness colors, so that the contrast is maintained for most readers.

    If people generally all hate the black theme, please tell me and I'll change it.

    ReplyDelete
  8. I do not hate the black theme. It does require some care in selecting text colors. The blue used in message headings is very readable; the blue under discussion or complaint is not.

    ReplyDelete