Tuesday, September 12, 2017

Agile is Dead. Long Live Agility and Pragmatism.

The luminaries who signed the original Manifesto for Agile Software Development in the 1990s have come back more recently, including Dave Thomas who said that "Agile is dead" in 2015, as part of his reflection on what went well, and what didn't go well, in the Agile Software movement they started.

As often happens, practices which had an intent, a purpose, a meaning, and an effectiveness, can lose that purpose, meaning, and effectiveness. As Dave Thomas laments, the Values get lost behind the Implementation, and Agile Software Development, shortened ludicrously to just "Agile", has become that which it mocked, and sought to replace.

That being said,  I actually appreciate the core values that underpin the Agile manifesto, even while I  advocate a skeptical approach to all methodologies.  Don't drink the kool aid.  Question.  Gather data.  Use Data and your ability to Reason. You can be a skeptic of ALL methodologies and also be a bit of a proficient practitioner of one of them, Agile or another.

In my current team, we're actually doing a really light set of "Fairly Agile" practices.  I've been on this team nine months now, and I really like the way we work.  The things we do are things that give us lots of positive payback.

Here they are:

1. Code Reviews

Shipping code is reviewed (hopefully actually read) by more than one developer. We use a distributed version control system (git) and pull requests to do our code reviews. We use bitbucket.  In my previous job we used git with gitlab, which also worked great.

2. Light Daily Standups

Our daily standups are in Slack (text chat) 3 days a week, and are video two days a week. Our team is mostly remote and disbursed.     The video standups are much longer than the usual standup time.  But since the team is distributed, and there are relatively few other meetings, we are able to spend a lot of each work day building things.

3.Lightweight "Paperless" Audit Trails : Slack All The Things. Don't Email.

We don't ship changes to code that aren't tracked in an issue tracker.  We don't make discussions verbally, or in emails.  We default to public place accessible by all employees (slack chat) for all architecture and UI changes.  Emails are private and are thus problematic for the team, or else turn into an unsearchable tar pit, and may as well have never been sent, as you can never find anything.

 4. Pair up when you need it.

We have a practice of teaming up with another developer whenever that would help get us unstuck, or do something that one person alone couldn't do.  We don't do that frequently, but we don't do it NEVER. We do it when it works.

I think these practices are great.   I don't think we're doing everything perfectly.  I think a certain amount of discontent is actually a core value for pragmatic/agile/high-quality software developer work.  For example,  I wish our codebase let us write more tests. That's something we probably can get to when our codebases are decoupled enough to permit unit testing.

What is your core "Agility" and/or "Pragmatic" practices list? What do you think makes  your software team work effectively?

Cool Advanced Troubleshooting Technique - WinDbg

I was having some problems with my Thinkpad laptop hanging. It didn't blue-screen. It just stopped updating the screen, and did not reboot. It seemed either that the entire video layer had hung, or that the kernel itself had hung (such as waiting on a spinlock forever), or that a critical subsystem of my computer had failed (since windows uses virtual memory heavily, such a problem can also be commonly caused by a storage layer failure, such as your primary boot hard disk).

What's relevant to this delphi blog,. is that it was Delphi's own debugger that most commonly triggered the hang.  Exiting the debugger, terminating a debug session, was the most common way to hang the system.

I've watched a few of the videos on MSDN Channel 9 from the guys called Defrag Tools. This is some serious low level technical know-how that these folks are dishing out. If you're into it, go check them out.   

Anyways after a bit of troubleshooting, I decided to try switching out various driver versions to see if I can make the hang go away. After trying the OEM Intel HD 520 graphics drivers from Intel's website, I was able to work inside Delphi all day today with NO crashes.

This is not my first "Delphi + Video Driver" hell experience.   About two years ago, I had a crazy set of bugs in Delphi that only reproduced with certain ATI/AMD video cards, and only in parts of my Delphi application using Microsoft Direct2D/DirectX.   Direct2D surfaces would just stop rendering, and whatever parts of my delphi application were using Direct2D apis just stopped working.  A couple years before that, I saw similar problems with GDI+, also on ATI/AMD video cards.   And going way back about 20 years ago, in the WIndows 98 era, I remember there were bugs in ATI video cards of the era that broke the ability of the Win32 GDI layer to render transparent bitmaps.

But this one is the first time I've had a Delphi program, or Delphi itself, plus a bad video driver, actually hang my entire system, requiring a cold reboot to recover.

More details on my question on superuser.com here.

Some suggested ways to get started:

1. install the Windows 10 SDK to get WinDBG installed on your computer.
2. on a system which generates BSODs, enable memory dumps on your computer in your System -> Advanced Startup Options.  In systems which hang without a BSOD, learn how to enable the Scroll Lock key way to force a hang.
3. watch a few Defrag Tools episodes to get the flavor of how to use WinDBG.

Note that full memory dumps are huge. If you have 64 gigs of RAM a full dump will be more than 64 gigabytes in size.  On my 1 tb disk with about when my total free space is less than 16 gigs, a Helpful windows feature will automatically delete my dumps so that windows doesn't fail to boot.   Try to have at least twice your total memory free on your main hard drive before you try enabling full dumps.  If you have 64 gigs of ram, make sure your primary windows boot disk has at least 128 gigs of free space. For just a bit of learning with Windbg, just enable some non-full-memory dumps so you can get the flavor of the WinDBG tool, and try entering some commands.

There are two "sides" to WinDBG, the kernel side and the user space side.  In the kernel side, you can view call stacks, thread states, change which CPU you are looking at, list NT native kernel level processes, and walk back in time to various saved states. In the user space side, you can see the  Win32 level APIs, processes, and walk back to a time prior to the last saved exception to examine the state of your system before the last first chance exception.  Besides being able to see C/C++ symbols from non-managed DLL/exe images, you can also use some extensions to be able to see things from inside the CLR, so you can troubleshoot problems with .net code.

Unfortunately, the debug information format used by WinDBG is not the same debug information format that is produced by Delphi so this tool is not very helpful for debugging Delphi application crashes, unless you don't mind working with uninstrumented disassembled code with no debug symbols.

Thursday, April 27, 2017

Some Quick Delphi Opinions

A few quick bits of opinion, and a question for my readers:

1.  Be it resolved that third party tool companies that offer fantastic support are an essential part of any small shop (that uses Delphi or otherwise).  Be it resolved that RemObjects is about as awesome in the support department as it gets.    I've been a fan of DataAbstract and the RemObjects remoting SDK, for Delphi and for .NET, for a long time.   Recently I was trying to upgrade an old project to use the latest version of RemObjects DataAbstract.  The "remobjects connect" site where you get support is fantastic.  Unlike a lot of places, you get answers (and if there's a bug, fixes) really really fast.  Shout out to Marc Hoffman and the other excellent people at RemObjects here.  You guys are great.

2. It is essential to modern development to have a second editor, or two or three.  The more editors you know, the better, up to the point of diminishing returns, which is about six editors, as far as I can see.  I actually regularly use three editors even when working with a powerful IDE like Visual Studio or Delphi. There are things I can do with Notepad++ that I haven't learned to do with anything else. And for searching code, I just love Visual Studio Code, and it has a great Pascal plugin.    And there are things I can do with VIM that other editors just can't do.   If you haven't messed around with Visual Studio Code yet, please please download it and try it. You'll thank me.  And if you're bored some day when you should be coding, try learning VIM. You'll accidentally become more productive, especially when you need to large or very precise edits across a large set of files.  Agree with me here? Disagree? Discuss.

3. Question for readers: Have you tried Delphi 10.2 tokyo on Linux yet? I'm planning to try that out soon.  Will probably write about what I figure out when I have time to play with it.

Thursday, April 6, 2017

Nick Hodges Book Bundle

Just wanted to give a shout out to an Excellent Delphi Dude, Nick Hodges, who has recently made available a bundle of his three recent and very well done Delphi books, in a single bundle.

I have actually only read two out of the three, but based on the quality of those, I can unhesitatingly recommend that you buy yourself  a copy of these three excellent resources for Delphi developers.

Nick's book bundle is on leanpub here.

Tuesday, March 21, 2017

Not Just Code Monkeys - Martin Fowler

I recommend everyone watch Martin Fowler's 2014 talk, which was given to its original audience without a title, and the title was then explained and introduced at the end.

It's called Not Just Code Monkeys.

Watch it first, and I have only two things to add, after you've watched it:

1.   I think that Martin Fowler's idea that we as developers need more of the attitude that  formally licensed Engineering disciplines (Electrical Engineers, Mechanical Engineers, Civil Engineers) have, even though I am (like Fowler) a bit skeptical of formally licensing software developers.

2.  I think that software developers can be agents of social progress, and like Martin Fowler, think we have a duty to not co-operate in immoral and evil actions carried out with software.  Bob Martin, another software developer and conference speaker I greatly admire, recently blasted developers who would, for example, do what the developers in the Volkswagen emissions scandal clearly did;  To do something that they ought to have known was wrong. Sure, the bosses at the company should be held accountable. But it took technical skill to carry out that fraudulent activity, and engineers were complicit in the evil that was done.

Don't be evil. And don't put your head down and just think of yourself as someone who slings code. You can make the world better. Or worse.  Which will you do?

Update: A more recent example of evil in firmware:  John Deere tractors now contain firmware which effectively blocks farmers from repairing their own tractors.  Their license agreements also say that you don't really own your tractor, you just have a license to use it according to their terms and conditions.   This (rightly) angers a lot of farmers.  You wouldn't catch me implementing that kind of crap in an embedded system. It's evil.

Wednesday, March 15, 2017

Need a Delphi Job? Love and know Delphi? Ping me.

Just a friendly periodic reminder that I currently know about more empty delphi job positions than I know of available experienced Delphi talent. You need work? Remote or Local? You live in Canada or USA? Ping me at warren dot postma at gmail dot com.

I do not accept any referral fees, and I do not work with agents, and I don't pay or get paid for this.  I am just a geek like you, who knows how it feels to have mouths to feed and rent or a mortgage to pay, and if you need a job and you're passionate and have at least a couple years of Delphi on your resume, you should be able to find something, and you can't, and you're in Canada or the USA, please do ping me. Mostly the people I know are looking for people who are willing to work in-office in various cities in Canada and the USA.

I accept emails at the above address if you're trying to fill a position in Canada or the USA as well. I will not talk to you if you work for any kind of agency.

Update March 17: Well I've now heard from more of my wonderful readers than I actually know about Delphi Jobs, so if anyone is feeling that they can't find people in the continental US, or in Europe, and want to hear about a few senior people, and you're an employer, and you're someone with direct hiring authority, not a headhunter/agent, or a go between of any kind, you can email me, and I'll pass your emails along to folks.

Update March 22:  There's an interesting job for Toronto area developers (no remote) posted on the Toronto Delphi User Group. Software Development Manager, delphi, local in-office job (Toronto) only. Check it out.

Friday, January 27, 2017

One month after Diving Back into Delphi: Hey I still Really Like Delphi!

I spent most of 2016 not writing Delphi code.  Yeah, I fixed the odd bug here and there, but mostly I was focusing on C#, ASP.Net core, and lots of other stuff.   Starting January 2017, I've gotten back into Delphi in a big way, because I'm working at a new job.  So far I have to say, I like it.  Delphi, and the new job.

I was kind of dreading going back to Delphi after having escaped it for almost a year.  But I have to say there's a few things I really really like in Delphi that I miss when I'm working in the .net and visual studio world:

1.  I like that the language isn't as complex as C# and there isn't fifteen ways to do things.

2. I absolutely love CodeSite and haven't found anything as useful for C#, although Serilog+Seq, comes close, and is web based, which is a bit more modern than CodeSite.

3. I absolutely love Nexus Quality Suite and haven't found any other tool for profiling and watching apps run, and seeing what they're doing inside, as much as NQS.  Now there are a lot of tools for .Net but actually NQS is pretty awesome.

4. I actually just like how pascal syntax works, it's clean, it's just the right level of verbosity. Begin and End are better than Curly Brackets, dammit!  And Colon-Equals is objectively the right way to do assignment. And strings should be one based! Fight me!

Sunday, January 22, 2017

Giving up again on Windows 7, and on Windows as my main operating system for work machines

I recently switched back to Windows 7, for work purposes, for a couple of weeks. Here's what I've learned, and why I'm switching back to Windows 10, but only inside a VM:

1. Suspend and resume, and sleep/wake functions in Windows 7 are horrible, even on decent hardware.  Windows 10 is vastly superior, faster and more stable.

2. Windows update on windows 7 is horrible and broken, and even the ability to supposedly "pause" these updates, is actually only a partial pause. There are still times when TrustedInstaller.exe is going to chew up all your I/O bandwidth and pwn your machine. 

There really is no stable way to run modern 32 bit or 64 bit Windows applications on your computer, and maintain control of your computer, unless you fight Windows by boxing it in a VM.

I'm switching back to "Windows 10 in a VM", on a very fast Solid State Drive with a huge amount of RAM. I can pause Windows 10 updates by one simple trick. Shut windows 10 off, and only use Windows for what I actually need it for, which is when I'm getting paid to write and repair Delphi code.

If anyone has read any formal write-up on what sorts of invisible installs Microsoft slides out to users, even in Windows 7, and what they do, and why, I'd like to know about it.

Other surprising things in Windows include:

* The ready boost features, which like most attempts at smart caching, show that caching and cache invalidation are truly hard problems, NP hard, nearly impossible problems.  You can speed some users up, deploy your solution, and actually end up decreasing their net system performance for THEIR workloads.  ReadyBoost in Vista was a complete disaster.   It has been scaled back and restricted in its application and it rarely hurts you on Windows 7, and on Windows 10, it has been coralled and controlled enough that it rarely has a downside.

* The windows search indexing feature, and the windows defender anti-virus active mode, can radically cripple your system performance by adding I/O latency, stealing I/O bandwidth and CPU resources, and of course, all of it steals RAM.

When I have performance problems on Windows, I am used to switching to the I/O monitoring features of the Windows Resource Manager.  Most often, my computer, when slow is I/O bound, it is seldom ram/swap bound, and most often its programs that are doing things I literally do not care about, taking away the resources that should be doing work I care about.

I do not see a solution here, since Microsoft does not permit you low level control of your own system. At one time, in the Windows 2000, NT, XP era, Microsoft permitted you to turn off ANY service in Windows, and the list of services that ran in the background was quite manageable. The list of processes on a freshly booted Windows system is so large, and so much stuff runs now, that you just better realize that about 8 GB of RAM, and about 4 MB/s of your total I/O bandwidth, and at least one full core of CPU, is off the table, that's reserved for Microsoft, and whatever it decides to use your computer for.   If sometimes it finishes and you can actually use your WHOLE computer for a few minutes, you should really be thanking Microsoft and not so annoyed that, on the whole, Microsoft thinks your computer should be doing something other than compiling your code or serving up a database to your users, that 75% of your computer is yours,  most of the time, and that 25% of it is Microsoft's most of the time.

Windows 10 then introduces this "active hours" configuration, which just might be the critical factor in deciding whether to use Windows 10 or Windows 7 for serious professional development work. Microsoft has at least decided that, for now, during the hours of 8 AM to 7 PM, I am permitted a modicum of control over what happens on my computer.  Thanks, Satya.

Saturday, January 7, 2017

Delphi 10.1 Berlin Pro Tip: Renaming CommunityToolbar bpl and good riddance to it.

In Delphi 10.1, without all the latest updates, if you get Delphi startup crashes, you can get around them in some cases, by removing optional parts of Delphi that are taking delphi down.   One such piece I usually rename so that it can't get re-enabled again, is the community toolbar IDE plugin bpl,

I go into the Program Files (x86), folder, then Embarcadero\Studio\18.0\bin, then rename CommunityToolbar240.bpl, by putting a tilde (~) on the front of the name.

Not only does Delphi 10.1 berlin RTM  actually start up without access violations and crashes, an unsightly and useless toolbar is gone from the IDE. I consider that win-win.

I believe the above issue must have been  fixed in Update 1 or Update 2.  But given with how much of a pain installing so called updates that are really not updates but full reinstalls is you can't really blame me, can you? I work in a variety of VMs and  it's not a one time reinstall for me, but a series of VMs with a series of Delphi versions in each.    I have had crashes in the community toolbar ever since whatever version it was first introduced in, and it remains one of the ugliest and most pointless IDE plugins ever.

There are other IDE plugin bits which are optional in Delphi, and if you're having a startup crash, one of the other things i sometimes have tried disabling is the debugger plugins for the C++ builder (if you have the full "RAD Studio" SKU, you have both), and a few other things which have historically been bad.

If you're running out of memory or you find the IDE is slow, it's also helpful to remove bits that might be causing that, and see if the problem goes away, even if only as a technique to find and isolate, and kindly report your findings in the Quality Portal.

If renaming the dll/bpl doesn't make things better, you can always change your environment back to its original state, by putting it back to its original name.

I also find that if my registry settings are to blame, renaming the current-user embarcadero BDS registry hive to "BDS_old" and then starting with a fresh settings, and see if the crash/problem goes away is also a helpful step in bisecting and understanding bad Delphi IDE states.  Clearly the rename is going to nuke all your local configuration settings and get you back to install-time defaults.  If your environment is self contained and easily set up again by just running some package-install and library-path setup tool, this technique can be doubly useful.