Merry Christmas to all.

Merry Christmas everyone. Here are some fun things for you to play with over the Christmas break.  Remember to put your laptop down and just hang out with those family people.

Some Toys to play with:
First, a list of interesting free stuff, a present from Scott Hanselman.

Some Delphi Specific Fun Things:

Apologies if you've seen these before, but hopefully these will be new to some of you.

• The Itinerant Developer blogged about a Firemonkey Container for VCL forms in July, I missed it then, if you did too, check it out. Pretty cool. 
• Nick Hodge's awesome new book mentions Delphi-Mocks by Vincent Parrett.    I'm playing with this right now, to increase my unit-testing powers. If you never read Vincent's intro post about it, this is a good read, even though it's been a year. I get the idea that lots of people still haven't seen the need for this yet, which means they aren't testing enough.
• If you still haven't downloaded it, you owe it to yourself to go get Andreas Hausladen's latest IDE Fix Pack for your Delphi/RAD Studio.  Thanks Andreas for building such a great add-on. I use it and swear by it, for Delphi XE2, XE4, and XE5 which are the versions I have active coding duties in right now.

Update:  Wow, great link at The Old New Thing to James Micken's blog.  Worth reading everything that Raymond Chen has linked to there.  Every serious Delphi (or Win32 C/C++) developer should read Raymond Chen's blog, The Old New Thing.  But you all are already reading it, right? I especially love the first link, about systems programming, called "The Night Watch".

What do you lose by moving to distributed version control from Subversion? When is using Subversion the right choice?

This is a follow-up post with a few counterpoints to the last post about distributed version control.  There are a few things that I can see that Subversion does better than either GIT or Mercurial. In the interest of fairness, I will point them out here:

1.  It appears that Subversion, given adequate server hardware, and adequate local area network bandwidth, may be a better solution for teams who need to do checkouts more than 2 gigabytes in size, especially if you simply must check in binaries and version control them. Your mileage may vary, in your situation, you may find that the point of inflection is 750 megabytes.    A common example of this is video-game development, where assets easily hit 10-20 gigabytes, and are commonly versioned with Subversion.

2. Subversion has support for partial and  sparse checkouts, something that you don't get with distributed version controls systems,  and all the attempts to add sparse checkouts to DVCS have been so flawed that I would not use them.  The nearest relevant and useful DVCS equivalent  is submodules.  Most users who need to do partial checkouts in subversion will find that they want to investigate using submodules in a DVCS.  If submodules do not meet your needs, then maybe CVCS is best for your team. If you need different users to have different subsets of the repo, using scatter/gather workflows, or otherwise do sparse checkouts (svn co http://server/trunk/path/to/sub/module3 rather than being forced in Git or mercurial to do a clone which is equivalent roughly to svn co http://server/trunk/ ) you may find Subversion meets your needs better.  It is a common rookie mistake  to conflate DVCS repo scope with CVCS repo scope.   DVCS repos are typically simpler and smaller intentionally, rather than the subversion "this server is your whole code-universe" monster-mega-repo strategy that Subversion limits you to. 

3.  Subversion has global version commit numbering, that is your ONE and only Subversion server has a commit counter, and since this global asset is shared among everybody, you can never have a "commit #3" on one computer be anything other than "commit #3" on anyone else's computers. On Git and Mercurial the system generates globally unique hash tags instead to identify commits, and the commit numbers, if available, should generally just be ignored as they are different for each user.  For some workflows you might find this global commit numbering system suits you better than referring to the hex strings of 8 to 24 characters that identify your commits, which have no ordering.

If I've missed anything, I'll add it in an update. Anyways, those are the three golden reasons that I know of that some teams will want to evaluate DVCS, and then stick right where they are with Subversion, which by the way, does seem to me to be the best open-source CVCS out there right now.  I only wish there was a better GUI than Tortoise, and that they'd do a little work to make command line merging less stupid.

Update: Someone was confused about why you would want users to "generate" hash keys.  This means I didn't explain it properly. The version control system generates hash keys, and "hashing" means feeding the input of your changeset through a cryptographic hashing function. The possibility of a hash collision is so low, that you will never encounter one.  Git and Mercurial both use them, and I have never heard of a hash collision, ever.  My only reason for mentioning it is that in a distributed system there is no single incrementing counter available to give you unique incrementing numbers. Not a big deal, but something to know before you leap.  More info here. 

Update 2:  Today I spent some time fighting Subversion working copy folder corruption. Issues like this one that were a big problem in 2008 and 2010 are still a big problem in 2014.  That's bad news for Subversion users.

Update 3: The big thing you'll lose when you leave subversion behind is all the bugs and the missing features. Subversion, and TortoiseSVN are giant piles of bad design, festering technical debt, and the parts that work bug free still have glaring functional deficiencies.  I don't think I'd miss Subversion one bit if I could move the projects that use it to something else, I would.

Update 4 (2016): Subversion is junk. Full of client and server side bugs.  I take back most of my compliments above, I'm sick of Subversion and want to kill it with fire.  Git can do sparse checkouts and with Git and GitLab you can even Lock files (a bad idea, but technically possible now). So there are zero technical reasons to keep using Subversion.

What am I missing out on if I'm not using Distributed Version Control?

As requested, I'm posting my thoughts about Centralized version control versus Distributed version control.  Git and Mercurial are the two tools I'm grouping together in this post as Distributed.

 Centralized version control, like Subversion, Perforce, and Team Foundation Server were, for their day, wonderful things.   They are dated technology now, and for many teams, the benefits of moving to a distributed version control system could be huge.

But I Want One Repo To Rule Them All!

Fine. You can still have that.  You can even prevent changes you don't want to get into that One Repo from getting in there, with greater efficiency.  Distributed Version control systems actually provide greater centralized control  than central version control systems.    That should sound to you like heresy at first, if you don't know DVCS, and should be obvious to you, once you've truly grasped DVCS fundamentals.

Even if having a real dedicated host server is technically optional, you can still build your workflow around centralized practices, when they suit you.

Disclaimer: There Are No Silver Bullets.  DVCS may not be right for you.

Note that I am not claiming a Silver Bullet role for this type of tool.  Also, for some specific teams, or situations, Centralized version control may still be the best thing.  There is no one-size-fits-all solution out there, and neither Mercurial nor Git are a panacea.  They certainly won't fix your broken processes for you.   Expectations often swirl around version control systems as if they could fix a weak development methodology.  They can't.  Nor can any tool.

However, having a more powerful tool or set of tools, expands your capabilities. Here are some details on how you can expand your capabilities using a better, and more powerful set of tools.

1.  Centralized version control makes the default behavior of your system that it inflicts your changes on others as soon as you commit them yourself.   This is the single greatest flaw in central version control. This in turn can lead to people committing less often, because they either have to (a) not commit, or (b) decided to just go ahead and commit and inflict their work on others, or (c) create a branch (extra work) and then commit, and then merge it later (yet more extra work).   Fear of branching and merging is still widespread, even today, even with Subversion and Perforce, especially when changes get too large and too numerous to ever fall back on manual merging.  I recently had a horrible experience with Subversion breaking on a merge of over 1800 modified files, and I still have no idea why it broke.  I suspect something about '--ignore-ancestry" and the fact that Subversion servers permit multiple client versions to commit inconsistent metadata into your repository, because Subversion servers are not smart middle tier server, they're basically just dumb HTTP-DAV stores. I fell back to manually redoing my work, moving changes using KDiff3, by hand.    With Distributed Version control, you can take control,  direct when changes land in trunk, without blocking people's work, or preventing them from committing, or forcing them to create a branch on the server and switch their working copy onto that branch, which in some tools, like Subversion, is a painful, slow, and needlessly stupid process.

2.  Centralized version control limits the number of potential workflows you could use, in a way that may prevent you from using version control to give your customers, and the business, the best that your team can give it. Distributed Version Control Systems encourage lightweight working copy practices.  It is easy to make a new local clone, and very very fast, and requires no network traffic.  Every single new working copy I have to create from Subversion uses compute and network resources that are shared with the whole team.  This creates a single point of failure, and a productivity bottleneck.


3.  Centralized version control systems generally lack the kind of merging and branching capabilities that distributed version control systems provide.  For example, Mercurial has both "clones as branches", and "branching inside a single repo".    I tend to use clones and merge among them, because that way I have a live working copy for each and don't need to use any version control GUI or shell or command line commands to switch which branch I'm on.     These terms won't make sense to you until you try them, but you'll find that having more options opens up creative use of the tools.  Once you get it, you'll have a hard time going back to the bad old days.

4.  For geographically distributed teams, Distributed Version control systems have even greater advantages.  A centralized version control system is a real pain over a VPN.  Every commit is a network operation, like it or not.  DVCS users can sync using a public or private BitBucket repository at very low cost, and don't even have to host their own central servers.


5. For open source projects, Distributed Version control permits the "pull request" working model.  This model can be a beneficial model for commercial closed source projects too.   Instead of a code review before each commit, or a code review before merging from a heavyweight branch, you could make ten commits, and then decide to sync to your own central repository.   Once the new code is sitting there, it's still not in the trunk until it is reviewed and accepted by the Code-Czar.

6.  For working on your own local machine, if you like to develop in virtual machines, having the ability to "clone" a working copy quickly from your main physical machine down into a VM, or from VM to VM, using a simple HTTP-based clone operation, can really accelerate your use of VMs.  For example, my main home Delphi PC is a Dell Workstation that has Windows 8.1 on the Real Machine, and runs Hyper-V with a whole bunch of VMs inside it. I have most of the versions of Windows that I need in there. If I need to reproduce a TWAIN DLL bug that only occurs in Terminal Server equipped Windows Server 2008 R2 boxes, I can do it. I can have my repos cloned and moved over in a minute or two.  And I'm off.

7.  Rebasing is a miracle.  I'll leave this for later. But imagine this:  I want to commit every five minutes, and leave a really detailed history while I work on some low level work.  I want to be able to see the blow-by-blow history, and commit things in gory detail.  When I commit these changes to the master repo, I want to aggregate these changes before they leave my computer, and give them their final form.  Instead of having 80 commits to merge, I can turn it into one commit before I send that up to the server.

8.  Maintaining an ability to sync fixes between multiple branches that have substantial churn in code over time, is possible, although really difficult.  By churn, I mean that there are both "changes that need merging and changes that don't".  This is perhaps the biggest source of pain for me with version control, with or without distributed version control systems..  Imagine I'm developing Version 7 and Version 8 of AmazingBigDelphiApp.   Version 7 is running in Delphi 2007, and Version 8 is running in Delphi XE5, let's say.  Even with Distributed Version Control (git or mercurial), this is still really really hard.   So hard that many people find it isn't worth doing.  Sure, it's easy to write tiny bug fixes in version 7 and merge them up to version 8, unless the code for version 8 has changed too radically. But what happens when both version 7 and version 8 have heavy code churn? No version control system on earth does this well. But I will claim that Mercurial (and Git) will do it better than anybody else.   I have done fearsome merge ups and merge downs from wildly disparate systems, and I will take Mercurial any day, and Git if you force me, but I will NOT attempt to sync two sets of churning code in anything else.  I can't put this in scientific terms. I could sit down with you and show you what a really wicked merge session looks like, and you would see that although Git and Mercurial will do some of the work for you, you have to make some really hard decisions, and you have to determine what some random change is that landed in your code, how it got there, and whether it was intentional, or a side effect, and if it was intentional, if it's necessary in the new place where it's landing.  If it all compiles, you could go ahead and commit it. If you have good unit test coverage, you might even keep your sanity and your remaining hair, and your customers.

9.  Mercurial and Git have "shelves" or "the stash".  This is worth the price of admission all by itself. Think of it as a way of "cleaning up my working copy without creating a branch or losing anything permanently". It's like that Memory button in your calculator, but it can hold various sets of working changes that are NOT ready to commit yet, without throwing them away either.

10.  Mercurial and Git are perfect for creating that tiny repo that is just for that tiny tool you just built.  Sometimes you want to do version control (at work) for your own temporary or just-started-and-not-sure-if-the-team-needs-this utility projects without inflicting them on anybody else. Maybe you can create your own side folder somewhere on your subversion server where it won't bother anybody, or maybe you can't do that.  Should you be forced to put every thing you want to commit and "bookmark" up on the server as a permanent thing for everybody to wonder "why is this in our subversion server?". I don't think so.

11.  Mercurial and Git are perfect for sending a tiny test project to your buddy at work in a controlled way.  You can basically run a tiny local server, send a url to your co-worker and they can pull your code sample across the network.   They can make a change, and then they can push their change back. This kind of collaboration can be useful for training, for validation of a concept you are considering trying in the code, or for any of dozens of other reasons.    When your buddy makes a
change and sends it back, you don't even have to ask "what did you change" because the tool tells you.


Bonus: Some Reasons to Use Mercurial Rather than Anything Else

TortoiseHG in particular is the most powerful version control GUI I have ever used, and it's so good that I would recommend switching just so you get to use TortoiseHG.  Compared to TortoiseSVN, it's not even close.  For example, TortoiseSVN's commit dialog lacks filter capabilities.    SVN lacks any multi-direction synchronization capabilities, and can not simplify your life in any way when you routinely need to merge changes up from 5.1-stable to to 6.0-trunk, it's the same old "find everything manually and do it all by hand and hope you do it right" thing everytime.

Secondly, the command line. The Mercurial (HG) command line kicks every other version control's command line's butt.  It's easy to use, it's safe, and it's sane.

SVN's command line is pathetic, it lacks even a proper "clean my working copy" command.  I need a little Perl script to do what the SVN 1.8 commandline still can't do. (TortoiseSVN's gui has a reasonable clean feature, but not svn.exe). Git's command-line features are truly impressive. That's great if you're a rocket scientist and a mathematician with a PhD in Graph and Set Theory, and less great if you're a mere human being.   The HG (mercurial) command line is clean, simple, easy to learn, and even pretty easy to master.  It does not leak the implementation details out and all you should need to evaluate this yourself is to read the "man pages" (command line help) for Git and Mercurial. Which one descends into internal implementation jargon at every possible turn? Git.
 I've already said why I prefer HG to GIT,   I could write more about that, but I must say I really respect almost everything about GIT.  Everything except the fact that it does allow you to totally destroy your repository if you make a mistake. That seems so wrong to me, that I absolutely refuse to forgive GIT for making it not only possible but pretty easy to destroy history. That's just broken. (Edit:  I think that this opinion in 2013 was based on inaccurate information, rewriting history is acceptable, and in fact, permitted in both Mercurial and Git, and the chances of a well meaning developer accidentally erasing his git repository remains a very small risk.)

Side Warning


Let me point out that you need a backup system that makes periodic full backups of your version control server, whether it is centralized or distributed.  Let me further point out that a version control system is not a backup system.  You have been warned. If you use Git, you may find that all your distributed working copies have been destroyed by pernicious Git misfeatures.  If you choose to use Git, be aware of its dark corners, and avoid the combinations of commands that cause catastrophic permanent data loss. Know what those commands are, and don't do those things.

 Get Started: Learn Mercurial

If you want to learn Mercurial,  I recommend this tutorial: http://hginit.com/

I really really recommend you learn the command line first, whether you choose to learn the Git or the Mercurial one. Make your first few commits with the command line.  Do a few clone commands, do a few "show my history" commands (hg log), and a few other things. If you don't know these, you will never ever master your chosen version control system.  GUIs are great for merges, but for just getting started, learn the command-line first.  You're a programmer, darn it.  This is a little "language", it will take you a day to learn it.  You will reap benefits from that learning time, forever.

And The Flip Side:  Reasons to stay with Subversion



(2016 update: Almost every team I know that hasn't moved to Git, has at least some people on the team who wish they could, and at least one stick in the mud in a position of power, overriding that team instinct.   I've changed my mind about Mercurial versus Git as well, and recognize that the programming world has chosen Git.  So it's time to learn how to fit in, instead of being the guy that sticks out because of his outlier opinion.   The tools and the community around Git are superior to the tools and community for Mercurial. )

Book Review: "Coding in Delphi" : A new Delphi book by Nick Hodges

Right now you can only get it by buying Delphi XE5.  I just finished reading it, and I love it.  I think that the title needs to be understood with the word "Modern" put in there somewhere. Maybe "Coding in a Modern Delphi Style" would fit the content well.  The title is fine, but I'm just putting that out there so you know what you're in for.

Did you want to know more about the proper use of Generics, not just the consumption of a Generic type (making containers using TList<TMyObject> ) but also enough to start really building your own Generics?  Do you want to learn how to dive into Spring4D (Spring for Delphi) and use its powerful capabilities?  Do you want to learn to use fakes (stubs and mocks) to isolate classes so they can be truly unit tested? Do you understand the real difference between unit testing and integration?  If you're not using isolation frameworks, or you don't know how to use them, or isolate your classes, is it possible that there's a whole new and better  level of test coverage?

Do you feel that you were late to the party, and nobody ever explained all the things that the cool up to speed delphi developers know?   This book is conversational, friendly, approachable,  code-centered, and although I already understand 90% of what is in the book, I still found there were some parts that really made my head bulge a little.

The book focuses on the raw coding aspects, and low level coding concerns, not on high level issues like user interface, or even architectural concerns like MVC.   The well known and beloved core Object-Oriented principles in the S.O.L.I.D. family, as preached by Uncle Bob Martin, are mentioned and given brief homage, but not expanded upon.  This is well and good, or Nick would still be writing this book and it would be eight times its current length.

Having finished reading the book, I'm going to go download all the add-ons and frameworks and libraries that Nick has gone over, and spend some time playing with them, and then I'll go read the book again.   I think that if you try to absorb everything in the book at once, you might get a bit of a headache. Instead I took it in over a couple days, let it wash over me and then I'll go back over it again in more detail.

I believe so strongly in the KISS and YAGNI principles, that I still don't think I'll be using all the shiny things that Nick talks about in this book, unless I can clearly see how using them benefits both the developers and the users of the code.  But I'm so clearly sold on the benefits of isolation, and testability, and design for quality and testability, that you don't have to sell me any further. It's just a matter of using these technologies in a way that can be done without substantially degrading debug capabilities.  (That is the secret down side of interfaces, and isolation, and dependency injection.  Your code can turn into a mysterious pile of mysterious semi-readable angle-bracket soup.)  

I am hopeful that there is a way to pacify both the desire in me to build testable, quality systems, and the desire to build readable simple systems that don't go down the Hammer-Factory-Factory road to hell.

If you want a copy, go buy XE5, or wait until some time next year when the book will have a wider release.  It's great though. You really should get it.  Nick joins my other favorite Delphi authors, like Marco Cantu, Bob Swart, Xavier Pacheco, and Ray Konopka on the virtual Delphi book authors shelf in my coding book collection, and it's a fine addition to the canon of Delphi books.

  

Five Code Upgrade Snares and How to Avoid Them

Here are five upgrade snares that have slowed progress or even stalled attempts by well meaning competent developers who have tried to migrate large code-bases up to new Delphi versions.  

1. The "I must not rewrite anything" snare.

The person who can not decide that something needs to be fixed will never upgrade anything. The person who values the patch he made 10 years ago to an ancient version of TMainMenu will never upgrade.  This is the first snare, it's a bear trap, and it's got your leg.

To avoid it, remember that your goal is to arrive at code that builds in both Delphi 6/7/2007 and XE5.
Don't panic.  Be calm.   Now continue.

2. The "I must rewrite everything" snare.

Having escaped the first snare, the hapless code-upgrader stumbles a little further along, is heartened by having survived the first trap, and decides to rewrite almost everything.   He was last seen stumbling into the Mojave desert.  His current whereabouts are unknown.

To avoid this snare, just as the point above, concentrate on making the code build, and pour your new code efforts into increasing unit test coverage.

3.  The "Failure to Recognize that We are in Boston, rather than Chicago" snare.

In Delphi 6, or Delphi 3 or Delphi 7, there were some features that are no longer current recommended Delphi features.  The BDE is one of them. If you think you can upgrade from Delphi 6 to Delphi XE5, and continue to use the BDE for another 10 years you're going to have a bad time.  Not just the BDE, but a whole host of things have gone to "should not use" status.  Also there are new VCL framework features including TForm.PopupParent which should be used to replace a lot of your Z-Order hacks that you invested huge time in in Delphi 6.  You can stage your changes, and you should stage them. Stage 1 might be "it compiles and passes unit tests, but uses BDE". Stage 2 might be "goodbye BDE". Stage 3 might be "it properly reads and writes the unicode data formats I want it to read, such as XML".

The second half of this trap is you assume that porting means a series of mechanical changes that you don't quite understand.  Don't make a change you do not precisely understand. Unicode porting usually leads to a lot of naieve things being done in your early days. For example, on your first day of porting, if you decide you'll change "All my Strings will be AnsiStrings" you're going to have a bad time. There is no "100% always do this" law, you must know what you're doing. But you should always choose AnsiString when you need a single byte string, and you should always use String (UnicodeString) everywhere else.  Read the excellent Unicode porting materials provided by Marco Cantu and Nick Hodges.  To avoid this trap, read these guides, and follow them.

4.  The "Hacks and Customizations" snare.

One day in 2001 you decided to modify TButton. Then in 2002, you decided to modify TForm, and in 2003, you decided to modify TDataset.   You had to, at the time there was no other sane choice.   But you will now have extra work to port your not-quite-VCL-or-RTL-based codebase up to your new Delphi version.  My suggestion is to make the code BUILD without VCL or RTL customizations, and function well enough to pass unit tests.  Then investigate new fixes for your previously required workarounds that are possible with the new framework.

I have also been trapped by this one, in the form of heavily customized Developer Express commercial third-party components that I then had to port changes from the 2004 version to a modern version.   Due to the nature and extent of my changes, this took me months.  I would have perhaps attempted to do more with subclassing, rather than modifying the component itself in the future.  However, the problem is that you can only do so much with subclassing. Sometimes you're really stuck and you do have to modify the original code.

Other than not getting into this trap in the first place, the way out is with care, with unit testing, and with proper version control tagging.  Attempts to remove customizations should be accompanied by tests, automated or manual.  Preferably both.

5.   The "Wandering in the Darkness, Surrounded by Grues" snare.

If you do not have unit tests, you will not know if you broke things.

When the code compiles and passes unit tests, you are on the path.

When the code compiles but passes no unit tests, you may still be on the path. But you don't know. The light is dim.

When you have spent several days in the dark, and continue making changes to code that does not compile, you are certainly not making progress.

You have been eaten by a grue. You have died.

Avoiding this one deserves more detail in a future blog post.  But it is mostly about unit tests, live use of your app after each change so you know if you broke the app's ability to run and do fundamental things, and the same sort of education and preparation steps as the previous point.  I have some more specific ideas on this one, that will be explored in a part 2 post.


If you've got another snare that we can all learn about then avoid, please post it in the comments!

Effectiveness - How to increase it.

Nick Hodges recently posted a link to a guy named Mike Hadlow's blog post about working hard versus working smart. I think it's brilliant.   I have often said that it's better to be lazy (while getting more done) than hard working (if by being hard working you are simply banging heads repeatedly into walls rather than walking around them), as a developer.  I believe you measure results, not hours that bums are in seats.

Anyways, rather than sticking on that point and belabouring it, I want to talk about the practices that have made a real difference in my effectiveness, in the order of their importance.  I promise to be biased, opinionated, and honest.   I do not promise to have scientific proof for any of this, only anecdotes and war stories. I use Science when appropriate, but I approach the whole business as a craft.

1.  Bisect and Isolate Problems to Determine Root Causes

Do not guess why something is not working, learn why it is not working.  When you know, you know, when you do not know, you can not be an effective developer.  If you do not understand and locate the exact source of a problem, you do not know if you fixed it.  This is the single most important skill a developer who gets things done must have.

2.  Learn New things, and learn Old things too

If you love what you do, you want to know the things that will make you more effective.  I do not know any highly effective developers who know only one programming language, one approach to problem solving, or who hate reading books about their craft.  Zero.   Do you realize how much amazing stuff is out there that you could download and play with? You haven't downloaded and tried Pharo?  Go? Python? Clojure?   None of those?  Sad, really sad.    You mean to tell me it's 2013 and you haven't read the Mythical Man Month?   Not just the one essay,  the whole book.  Oh, I weep for you.

3.  Clean code

This has always been a personal point for me. I hate leaving my code in a place where it emits warnings or hints.   I do not like code that has no test coverage.  When code is not exactly right, sometimes I say it has a "code smell".    I mean the same thing by Clean Code that Robert Martin means by it, but I like to stress that it's an ideal that you move towards, rather than a thing you achieve.  The perfectionist streak in me detests the idea that any code ever is clean. It never is.  It could always be cleaner. The standard is "what is clean enough that my co-workers, my employer's customers, and my own ability to work are not impeded". That's clean.

4.  Do Science

Admit it you thought I was going to harp on unit tests as a separate point, didn't you? No. I'm leaving it as a sub-point or reasonable practice within the general norms of Clean Code.   But clean code is not enough or even my main goal. I am a person who, like a mathematician in search of an amazing new result, is pleased by beautiful mathematical perfection in my work.  I find that the most effective way to find where those beautiful bits of software are inside your current working product is to be a scientist.  You want to know what a variable does? Take it out and find out.   You want to know why this application was designed with a global variable that makes no sense?  Rename it and find all the places where it's used instantly.  You have version control. Poke your code, prod it. Abuse it.  Run it on really slow computers so you can see the screen draw and figure out why it's so flickery and crappy, and make it better.   Be a scientist.   Do science.  Ask question. Form hypothesis.  Test hypothesis.  Revise hypothesis.      Remember that working copies should be cheap (you do use distributed version control right?) and easy and fast to revert any experimental changes you make.


5.  Use structure effectively

In the beginning was the machine.  Darkness moved on the face of the deep, it was formless and void. The programmer had a manual of opcodes.  The programmer built programs out of opcodes, which were data values stored in words (registers) in the machine. The programmer somehow flipped switches, or clipped bits of paper out of cards, and somehow got those 1s and 0s into the machine, and pressed a button or flipped a switch and the code ran. Or didn't.  Mostly it didn't run.  The programmer went back and tried more stuff until some lights turned on, and the early computer printed a character on the teletype.  Only by sheer brute will power was anything ever accomplished by this early programmer, with no Language tools.      The evolution and development of compilers and computer programming Languages was a response to the futility and low productivity of these early tools.   Practices, and ideas about how to use these languages grew and evolved along with them.    Today you have Object Oriented, Aspect Oriented,   Post-OOP, Dynamic and Statically Typed, Functional, and all other manner of attempts to provide Just the Right Structure for your program.  

The two cardinal sins against the effective use of structure are to put too much structure in,  so that the structure has no purpose or reason behind it, and not putting enough structure in.   It is my considered opinion that the Design Pattern movement began to love using Patterns so much that it overdid it.  I have seen a Factory for constructing Singletons that have a single method, that could have been implemented as a non-OOP procedure, and would have been better off for it.    Mostly Delphi code bases are too far the other way. They tend to have 50K line methods, 100K line units, and all the code is in the .pas file that Delphi created for you when you created your main form. Because making new units with reasonable cohesion, and minimal coupling,  was too hard.

When a codebase has not got an effective structure, the productivity of everyone working on it is sapped to nearly zero.

6.   Use tools effectively

If you do not learn how to use the Delphi IDE, command line compiler, build systems, continuous integration tools,  profilers,  static analysis, and any other tool that exists to help you, you are less effective than you could be. If you never learned how a distributed version control system works, even if you don't actually use one for your company's central repository, you're missing out on one of the single most powerful software tools ever invented.   Mercurial.   There's git, too, if you like accidental complexity and technical hurt.   (I never promised objectivity here, this post is 100% opinionated.)

7.   Build new tools or frameworks but ONLY when it is effective to do so.

Sometimes a new tool is a script written in 100 lines of Python (for fans of code that is zen like in its beauty) or Perl (if you like accidental complexity and revel in it). Sometimes that 100 lines takes 10 hours to write, and saves you 100 hours, and then saves you another 100 hours a year later when you need it again, or something almost like it.  This is something that people used to think was only possible if you were a LISP hacker.   Nope. You can build tools to accomplish anything you can imagine and then precisely formally define in code.   This meta-coding mentality can go TOO far.  See the famous "I hate frameworks / hammer factory factory" forum post  for more on that. You have to build tools when, and only when, they are justifiable.


What are your top effectiveness tips? Please share.

Modernize your codebase: Inspiration to Ditch your Ancient Delphi Version

I still see a lot of people running old versions of Delphi, the most common one people freeze at is Delphi 2007, because of the Unicode changes to String types, but I have also seen the odd one stick at Delphi 7, or even Delphi 6.

Here are some suggestions on getting out of those Don't-Upgrade-Ever Tar-Pits.

1.  You may lack motivation. You may not realize how much you need a modern Unicode String type.  You start by believing you can upgrade your AnsiString-forever, or shortstring-forever codebase. Yes, yes you can.  The internet is Unicode, and every computer is connected to the Internet 24/7.  WideString is not enough.  The modern String (UnicodeString) in Delphi is fast, flexible, and internet ready.   I've spoken to people who aren't upgrading because they assume the new String is slower. On some benchmark code (Sieve of Eratosthenes and some string crunching code), the latest Delphi XE5 is actually faster at UnicodeString (String) operations than Delphi 7 was at AnsiString (String) operations, and basic IntToStr and StrToInt functions.

2. Upgrade for Productivity boosts. The IDE INSIGHT feature is fantastic.  Once you get used to it, I find it hard to use a version of Delphi that lacks it.  It's google for your IDE.       I like to supplement it with the GExperts file open dialog, but the combination of the two is magic.

3.  You've been down the Ancient Delphi Codebase road, and you know what a pain things like Z-Order bugs are.  If it helps you decide to move up,  remind yourself that the VCL in 2007 and later supports proper Z-Order parenting, so if you're on Delphi 6, or 7, it's time to say goodbye to silly Z-Order bugs. (Window A behind Window B).  If your code is not Unicode ready yet, you can get XE5, and it will entitle you to a Delphi 2007 license (previous versions access), get your Z-Order bugs fixed, then continue to work towards proper modern Unicode readiness in your codebase.

4. Maybe you don't really need 64 bit Delphi, but having the possibility open sure is good, right?  Or maybe you would like to build something 64 bit? Running Windows 64 bit? Want to access lots of memory? Want to write a shell extension?  32 bit may be great for lots of things, but having 64 bit in your bag of tricks is worthwhile.  64 bit Windows will be so mainstream eventually that (like Unicode) the whole Delphi world will eventually move to it, and Win32 will eventually fade away. Yes. Yes, it will.  Do you still want to be building Win32 apps when Win32 goes the way of Win16? Remember NTVDM? WOW is the new NTVDM. It isn't where you want your apps to live, is it? Inside the backwards-compatibility box?  So be ready.

5.  Mac OS X support. You can write apps for OS X with Firemonkey.  Maybe this isn't a "need" thing, but it's nice to have in your tool-belt.    Being on a modern Delphi version means having this around when you want it.

6.  iOS support.  You can write native iPhone apps with either RAD Studio XE 5 or Delphi XE5 + the mobile add-on.   Your kids will think you're cool, if you make them a little App for Christmas. Right? Of course right.

7. Android support.  You can write Android apps using the Delphi compiler for native ARM and Android NDK.  

8.  If you're stuck on a version prior to 2010 you don't have proper platform support for Windows 7, and Windows 8.  Things like the file-open dialogs, etc, changed at the Vista level, and that support was never in Delphi 6,7 or 2007.  I think it's kind of funny that Delphi grew proper Glass (DWM) support, and then Microsoft suddenly yanked Glass out of windows 8.  But maybe it will come back in Windows 8.2.  You think?  Anyways, the core VCL on modern versions like Delphi XE5 is a lot less painful to support Windows 7, 8, and 8.1 on, than the creaky old version you're using, if your version is older than, say Delphi XE.

9. VCL styles.  This is an amazing way to add a "face lift" to your VCL application without rewriting it.  There is a bit of work involved, but a lot less than you'd think.   The amazing third-party add ons like RRUZ's awesome helper library make it easy to make your VCL app look great.  If you haven't got at least Delphi XE2, you don't have this great feature.  To go with that pretty Style support, add some VCL gesture support.  Support touch gestures on a  touch-screen laptop or Windows Surface Pro tablet, using VCL gestures. Swipe, pinch, or make your own custom touch gestures.  Combine this with the VCL styles and you can modernize your look, and functionality from "90s" to "up to date" pretty quickly for your core Win32 desktop apps.

10.  Delphi Language Evolution. If you're still on Delphi 6 or  7, and you don't have Generics, or modern RTTI, or the new module grouped-naming system (Unit Scopes), you might be surprised how much nicer and more readable these things make your code. For example, I love my Uses clauses to be grouped with all the System units first, then VCL units, then my third-party library units. The ones that have no dots in the namespace are either primitive third party libraries, or my own units. It's amazing how convenient and nice and "expressive" these longer names are. I love them.

I hope this list gives someone the motivation they need to kick themselves in their own butts and get their codebases out of Delphi versions that should be retired, and moved onto a modern Delphi version.  You'll thank me.

Oh. And how do you actually DO it when you get the nerve?  I'll write more about that later.   But for starters, I suggest you try to write some Unit Tests.  Remember that while you are in progress of modernizing your codebase, you can do it without ever breaking your application or introducing unknown behaviour.  I'll write more about that approach, in an upcoming post.


Update:  If someone is sceptical about the value of updating, I ask them, what does it hurt to download the trial and try it?   You can do almost anything you want to try with the demo version, get your code mostly ported, try to port your components up,  get stuck, and post questions on StackOverflow. If you really love Delphi 7, great. It was a great version.  Enjoy it until the heat death of the universe, or at least, until the sun goes super-nova. No problem.  But most of us moved on long ago, and I just wanted to let you know why we did.