When All You Have Is a Hammer (Part Deux) - A Cry For Professionalism

A couple years ago I wrote a blog post called "When All You Have Is A Hammer, Everything Looks like a Nail".  I was bemoaning the way that software developers (including me) will sometimes use the one thing they know and avoid learning new things.  I have further come to believe that even though we start out passionate, and with a desire to learn more, and to find ways to keep getting better, many of us still periodically have days or weeks where our output decreases in quality, sometimes we even make a complete hash of things, we who perhaps might be expected to know better.  Most often it happens when developers "phone it in" for a day, a week, or a month. Maybe they've been demoralized by micro-management, or continual hectoring.  Let's be kind to people when they are in ruts, because we never know when it will be us.  Let's ask ourselves why we build bad things, and not hector others.   Somehow, our process and its outputs are no good.  Later on this will be discovered, and someone has to clean it up.  Sometimes I am the person who has to clean up my own mess, several weeks or months later, when it becomes clear my changes were a bad idea. Sometimes I am cleaning up someone else's mess.  I ought to do this without grumbling or attacking others. That's part of being a professional. But, since I have observed this category of problems over and over again, and like any hard problem in computing (naming things, being smart and good at your job without becoming an arrogant jerk), it admits of no final solutions.   But there are things we can do, and if the software you make is making the world a better place, then you should care enough to try to do your job better, every week, every month, every year.

First, begin your coding day by asking yourself if the work you're doing today matters. For some of you, that's easy.  Even if curing cancer is out of your reach, then at least increasing the effectiveness of breast cancer screening protocols counts, if you work at a place that does that.  But let's say you spend five minutes considering the business you work in and you still haven't thought of the reason why what you will do today (fix a bug, work on a feature) matters.   Here's my last line, if all else fails;   If what you do today keep your businesses (and your customer's businesses) running, then you're not only earning the dollars that feed you, and your children today, you're also helping other businesses that employ people who are hopefully going to be able to feed and clothe their children, and give those kids somewhere to sleep.  And if you like me are a regular contributor to the charities that matter, then every organization that employs the men and women who give to others, is not only doing its own work, and helping its own employees and customers, it's also, hopefully making the world a better place.  So, you're rooted now? You're ready to begin your day? You care. You are checked in fully.  This is step one to avoiding doing bad engineering work. You have to care about quality.  If you're anything like me, you need to connect all the wires up, from the battery, to the lightbulb, and do that almost daily to keep caring.

Second, all day long, when you find something stupid, that makes no sense, avoid the temptation to stop caring, and push on through, so you can close your bug-tracker ticket, and go home.  Ask the questions. Find the subject-matter experts. Engage in constructive, rational critique of the system. Don't jump into ripping it apart and rewriting it, and don't jump into "must not touch legacy code" fear mode, either. In between, find a way to make the customer's life better. First, the person who is forced by the conditions of their employment to use your software is your customer. Second, the person who signed the cheque to buy your software from your employer is your customer. That business has to succeed, to keep everybody working. Nobody succeeds until everybody succeeds.   This is the second step in avoiding making these mistakes.


Third, keep learning.  If you are stuck on Delphi 5, it's time to move up and onwards. You won't even know what all the reasons why are, until you do. That's just the nature of the game. If you are stuck in a rut, defending the rut won't get you unstuck.  Fine, your code is on a modern delphi version? Next, if all you know is Delphi, then go learn Python. If you do not know any version control tools at all other than zip, then you my friend are not a professional developer.  Go learn one now, I'll wait here.  If all you know is Subversion, go learn Mercurial. If you don't know how to automate your build, at least build a batch file and invoke msbuild and have that build your Delphi, C#, and C++ targets for you.  If you're still using the lame old solution of batch files, then level up, my friend. It's time to learn how to use FinalBuilder, or WANT. If you have a build tool but no CI server software experience, it's time to learn Jenkins (or Hudson).      If you have never done SmokeTesting, you better learn how.  A smoketest job runs on Jenkins (or Hudson) and makes sure everything that is checked into source control still builds, and runs after every single checkin.   Now you have arrived at this point, it might be a good time to notice if the unit tests you write are actually finding any bugs. Does the smoketest job go red not only for compilation failures, but also for unit test failures?   If you have zero unit test failures over 60 days, chances are that you actually have zero meaningful unit tests.  Go analyze why changes that resulted in regressions that your QA person found, were not found in the unit tests.

The more I think about the problems of "hammer-thinking", the more I think there is no real solution other than the basic patterns I see in all good engineers.  Ask why.  Grow and improve. Learn.  Add new tools to your toolbox.  Build new tools when none exist.  Never stop growing.

If you don't want to do this, you should go find a new line of work.  I'm not kidding.



Postscript:

Agile is Dead. Agile values have been totally lost behind the implementation.
Dave Thomas (of The Pragmatic Programmer)

How To Find and Fix Problems In Your Component Packages

A very useful technique that all Delphi developers should know, is how to debug a your own DLLs (IDE experts) or  BPLs (binary component packages) when they exhibit problems when they run inside Delphi itself.  You could think if it as "debugging delphi inside of delphi", except you're not really doing that.   For one thing, the Delphi main IDE executable (BDS.exe) is not your application and it doesn't have full debug symbols, so you can't see all the names of procedures and objects inside of it, but what you can do, and what matters for the purposes of this quick tutorial, is that you can set a breakpoint in a unit that is installed as a designtime Component, or as an IDE Expert, that you have built with full Debug information, and that you have the source code for.

If you have some bug in a commercial library (such as Developer Express, or Teechart), and you have the source code for it (your purchase included more than just BPL and DCP, and DCU files, and also included the .PAS files and the .DPK/.DPROJ project files), then you can find and fix problems with them, as well.

When Not To Use This Technique

• You could just use logging instead.  Inside your design-time component, you could just add some CodeSite logging, to help you figure out if the math or logic in your component is working properly at designtime.
• You could simulate this within your own code, at runtime.
• You could just install MadExcept into the Delphi IDE, and catch your exception and get a stack trace, and figure out from the stack trace, what calling sequence is causing the exception, and fix it without actually needing to debug.  If you don't have MadExcept yet, go buy it. I'll wait. You're back? Okay good.   (There's a free non-commercial version, but you are not the kind of limp-brain who refuses to support a pillar of the community like Matthias, are you? You bought it anyways right? Good for you.)

What is the difference between Runtime and Designtime, for your code?

• Every class that inherits from TComponent has a property called ComponentState.  It is a set of bit-flags, and one of those flags is called csDesigning. When csDesigning bit is on, we are in designtime mode. A common beginner mistake is to forget to protect code that should not run at designtime with guard blocks like this:

procedure TMyComponent.SomePropertySetter(Value:Integer);
begin
  FValue := Value; // designtime+runtime
  if not (csDesigning in MyComponent.ComponentState) then
    DoRuntimeOnlyThing();
end;

• Your objects are instantiated in designtime mode when they are being loaded by Delphi into a Form, when you are in the Design tab of the IDE.  This means your code can do some pretty amazing things for you.  You could add menus that appear when you right-click on your component, to automatically do some task for you that would be helpful when you're building applications using your component. These designtime actions are known as Verbs. The default one is known as Edit. You could have a dialog box open to help you configure your component.  TeeChart makes excellent use of these designtime actions. If you want to see how to add one to your component, there are many excellent tutorials around on the web, check out Zarko Gajic's excellent series on delphi.about.com.

Examine And Understand Your Current IDE and Project Settings, and Make Sure You Know How to Compile and Link A Package So that the BPL that Is Loaded Into The IDE Contains Full Debug Information

• Before you start changing anything, be sure you understand your working environment. Spend a minute to review the the following aspects of your IDE's configuration, and your project configuration:
• Look at your IDE Library  Path. What folders will be searched when you look for DCU and DCP files? 
• Look at your IDE Default DCP and BPL paths. What paths are used? Is it the default or a custom setting?
• Look at your project search path. What folders will be searched when building this project in debug mode, and then again, in release mode? Look at both.
• Look at your project BPL and DCP output path. Are you using the default (that field is blank in your project), or are you overriding it? Where do you expect the BPL to be output to when you build your package?  Is that the same place, when you are in debug mode, or in a different place?
• Look in the IDE Options at the Environment variables, and examine your path. What folders currently contain your BPL files? Do you understand what packages you want to load from what locations?
• You will want your package to be compiled in Debug mode, that is, compiler and linker settings set to include debug information.
• Somehow we need to get the IDE to load our debug package, and every package it depends on, from the set of folders in my PATH.  This might be as simple as hitting compile once on your package, while the Debug settings or the debug configuration are active. In ancient Delphi versions which lack a separate Debug/Release mode profile setting, this is, paradoxically, more straightforward, just check the Compiler debug info and Linker debug info checkboxes are on.  On modern versions it gets more tricky.  For example, if you are a tidy sort of person, and you have set up your packages so that your debug and release mode packages output their BPL files into different folders for your Release and Debug configurations, you may run into some headaches at this point, getting everything to load. In the end you have a set of DLLs that have the file-extension BPL instead. Now you want to load a mix of debug and release packages and say you have both the debug and release BPL output folders in your path. How do you suppose that's going to work out for you? Suppose your BPL depends on another set of BPLs.  The longer I consider the mess that I have sometimes made by having multiple BPL output folders, the more I don't like doing that. I would have only ONE package BPL output folder, but have separate DCP and DCU output folders.  Changing from Debug to Release mode when building a Package, I would want all my outputs to go to the same binary output BPL folder, so I only have ONE folder to manage in my PATH.   Switching executable/dll search PATH values in and out is a source of headaches and pain for me, and I suggest you avoid that pain.
• At this point, before you continue, you should be able to build your package, know that it is built with debug DCUs, and that it will load into your IDE, without errors. To verify that you have accomplished at least this much, you should click Component menu, then click Install Packages, and find your package in your installed packages list.  If the checkbox is unchecked, or if you saw some errors when starting your IDE, or loading your project which uses your component, you have not yet succeeded in compiling your package with debug information turned on, and still retaining a loadable state.  Make sure that BPL that is in the IDE has the same FULLY QUALIFIED FILENAME INCLUDING ITS ENTIRE PATH and not just a package with the same name, in a different directory. If your Debug version of your BPL is output to some different path than your release BPL, then you shall have pain here.  I said that already above, and I'll say it again, and still you'll plough on past me, and then wonder why you can't set breakpoints inside your BPL, and generally debug things.

Click Run → Parameters and Set the Host Application

For most modern delphi versions, type $(BDS)\bin\bds into the Host application edit box.
If you're a sad-sack using some ancient Delphi 5/6/7 version, put the path to your main Delphi32.exe into the Host Application box. Now click Ok.  Now click the debugger Run icon, and hold onto your hat, because Kansas is going Bye-Bye.  Things are about to get weird if this is the first time you've ever done this.

   

If you did this right, you'll start out seeing the Delphi splash screen, then probably your whole IDE (if it's maximized on a one monitor system) will be replaced by another IDE window.  I recommend you un-maximize, and then move your second Delphi instance over to a different monitor at this point.  Which one is which? Look carefully.  Which one has the Debugger desktop-profile selected, and has [Running] in the Caption of the window.  I always keep that one to the left of my workstation (either on a left monitor or to the left side of my main monitor) and I keep the application under test to the right of my monitor, or on a secondary monitor.

Now I can work with the Delphi form editor, perhaps create a new project, drop the component I'm debugging onto the form, and start setting properties via the property inspector. I could open up the Component Editor (which is a designtime-only bit of code that may contain a wizard to help me configure my component's properties), and debug that.  If I'm debugging an IDE Expert, I could click Tools → My Wizard → Do Something, and watch my Wizard Do Something, or have the IDE stop my wizard for me, on first chance exception, because my wizard is accessing an object which is NIL, so of course, I'll get an access violation. Now I can see where that code is, and examine it.

If you find you can not debug your package in this situation, then you messed up in the areas I talk about in the first half of this blog post. If you thought I was belaboring my points above, please rest assured, I was only trying to be helpful to you. Are you really aware of every detail of your environment? What your windows environment variable named PATH is? What every project has as its output folders, not only in Debug mode, but also in Release mode?   What all your compiler and linking settings are, and how they affect your environment? Sure, that's a lot of information. But what are you? Are you a developer, or a whiner?  Woman Up, or Man Up, as the case may be. Dig in. Solve your own problem. Nobody else has the tools in front of them to solve this. You do.  Go to it.

Thoughts on Delphi 10 Seattle, and Idera

Most people who follow what's going on in the Delphi world are aware that Embarcadero was sold by Thoma Cressey Bravo to Idera.  I had not heard of Idera and when the acquisition happened, like a lot of people, I googled the name, found the company website, and did a little reading.   I could see the synergy between their MS SQL Monitoring flagship product line and a lot of Embarcadero's database tooling but the Development Tools market appears to be kind of new to them.     I'm a member of the "MVP" program and I've sent my thoughts and ideas along to some of the folks in the new leadership team, and I'm looking forward to seeing some fresh ideas in growing Delphi's market share, and getting more people using Delphi.

In my opinion, Delphi "10 Seattle" (once I have gotten used to the change away from the XE naming convention, which I quite liked) is a good release, and worth upgrading to.  The extra memory available inside the IDE, due to the large address aware IDE core, is a welcome improvement. The IDE is also faster, and more stable than ever, partly due to the increased memory available, and also because it seems a lot of work has been done to make startup, shutdown, building, running, debugging, and loading and unloading packages work better, and faster.  

That being said, I have to admit, I still get Delphi crashes.  One of the difficult parts of being a Delphi developer is my inability to tell the difference between a Delphi crash caused by a bug in Embarcadero's code, and a crash brought on by the components and experts I've loaded into the IDE. Since I can't work without my components, and only really can choose to turn Experts on or off, it's really a difficult problem.  One hopes, that if stability is still an issue for people, that there might be some ways to tell the difference between a designtime package gone amok, and a real Delphi bug.  Maybe the designtime packages should be loaded into a different process, one that only contains the form designer, and not the rest of the IDE's code-space.   Then the form designer could say "oh snap! something's gone wrong!" without the rest of the IDE going snap too. Some day, right?

The best technique currently available is to run the IDE inside the IDE.  Delphi can debug itself.  Which is cool. Unfortunately, unless you already know a likely way to cause a crash, this is not a great help, as the inner IDE instance will run quite slowly.  I have taken to instrumenting my packages, adding OutputDebugString messages to my packages, in key module initialization and finalization sections, so I can see the packages loading and unloading.  That gets a bit difficult after a while, and in my case, with over 247 designtime and runtime packages (the designtime packages depend on runtime packages, so in total, I have 247 bpl files loading into my IDE) it's too much for me to figure out.  The codebase I work on right now is the largest I've ever worked on. It's impressive, and in Seattle it's working fine, but I have some doubts about the stability of the latest Developer Express packages inside Delphi 10 Seattle. There's something weird going on, and I'm not sure what's up. Once I isolate it, I'll report it both to Developer Express and to Embarcadero, but due to the complexity, it's not easily done.

I have reported some issues with High DPI systems, and I'm pleased to see some progress on this with the recent "Update 1".  It seems there is more still to be done in supporting the latest hardware, including 4K and 5K monitors, with their lovely crisp 150 dpi to 200 dpi pixel densities.   I'm using Windows 10, and Windows 10 is really quite slick on systems with multiple monitors and each monitor can have a different DPI scaling factor.  Delphi has some initial support for this, but where I work we already have our own classes to handle this, so we have the interesting problem of trying to decide how to merge the new VCL DPI awareness functionality with our own.

I'd like to hear your comments, if you're reading this, and you've tried Delphi 10. How are you finding it so far?