Friday, November 5, 2021

Windows 11, and Delphi 11 : How is it for everybody? You better move to Delphi 11 if you're going to update to Windows 11, and here's why...

 So the future is here, Windows 11, and Delphi 11. Do you think the "everybody moved to 10 everything" and "everybody moved to 11 everything" all the same time is a coincidence?

Here's how it actually works:

1.  Something in the industry creates a buzz. What is it this week? Bitcoin? NFT? NFC? GUIDs?  N-Tier Design? Message Queueing?  The Internet of Things?

2. Everybody rushes to be "part of that". 

If you get hired at Embarcadero, or anywhere really, to sell anything, especially in the tech sector you'll get hauled into your bosses office regularly and someone who looks like they might belong in a Dilbert cartoon will tell you what the new thing is, that they read in their BossManagerMonthly magazine or blog, and tell you you need to get on top of it.

What is Windows 11?  For microsoft it's a chance to make a set of controlled breaking changes to the Windows platform ecosystem. It's neither purely a "good" thing, nor purely a "bad/mediocre" thing. It's a necessary thing. 

While Apple feels free to break everything all the time, every day, Microsoft has comitted to, and maintained a history of not breaking everything every day.   And so there comes a time, when breaking changes and "flag days" have to happen.     In true microsoft fashion, there are devices that will continue to run Windows 10, for the next few years, or longer.     And also in true microsoft fashion, there are devices that will refuse to auto-upgrade for you but which you can force to update to Windows 11, via the media creation tool, with Microsoft's full blessing.  You try it, you get the results and you get to see for yourself what Windows 11 does on your random pre-2020 consumer hardware.

So what is this post about? It's just a request for discussion.  What has been your experience? Are you running Windows 11 and Delphi 11 and how is it?

I have one anecdote to report; It seems Delphi 10.4 debugger hangs a lot on Windows 11.  Any attempt to set breakpoints and single step, the whole IDE (bds.exe) process hangs forever at 0% CPU. There do not appear to be wait chain analysis tools built into Windows 11 yet, and I'm not sure if I could do a wait chain analysis on BDS.exe from visual studio but I don't have visual studio on this box.   

The issue is  reproducible with a trivial helloworld-from-delphi app in 10.4, but is more frequently reproducible on the large 150+ megabyte exe apps I get paid to code, some of which load large non-trivial amounts of redistributables with, including openssl,  chromium embedded (google chrome browser), and a lot of other stuff.  These large apps are not fully portable to Delphi 11 without a large investment of time, and so I can't say if the same app will hang the debugger kernel in our large Delphi 11 apps.  The main app being debug above is a 32 bit 169 megabyte exe that loads another 180 megs of 32 bit DLLs (Chromium "libcef" is over 100 megs of stuff).   

In a trivial hello world app, Delphi 10.4.2 freezes about 50% of  the time after I hit a breakpoint and then continue.  Other 50% of time, it may recover and then when you end the debug session it will say "fatal debugger error" and ask that you shut down the IDE. THere's basically something fundamentally different in the Windows Kernel environment in Windows 11, and the Delphi 10.4 debugger is not up to speed with the changes in the platform.    My tests are with Delphi 10.4 build 27.0.40680.4203 (delphi 10.4.2, all of Update2 installed) on Windows 11 pro build 21H2, osbuild 22000.258 (windows 11 initial release).

Tuesday, July 20, 2021

Did you know that the Raize Components (Rz*) by Ray Konopka are now in Getit under a completely inscrutable new moniker?

 Would you have guessed what "Bonus KSVC 7" is, if you saw it in getit.  No? Neither would I.

It turns out that if you are looking for Raize Components they're now called Bonus KSVC, and available as version 6.5 and 7 in GetIt.

Raize Components, as it was for over a decade, got renamed to Konopka Signature Controls in 2020. But one rename was not enough for the bright sparks at EmbarcIdera.  Now it's been renamed again, this time it's called KSVC. Sure.   And the word bonus was necessary to put in front in the title, because "raisins".

Naming things is one of the hard problems in Computer Science, I guess. 

The components remain quite awesome,  and since they're in GetIt, I can heartily recommend everyone check them out if you hadn't heard about them.  There's still a discussion forum dedicated to them on the site, but the product appears to have been 100% bought out by Embarcadero and is basically "part of Delphi now", which was a good move by Embarcadero.  

Wednesday, April 14, 2021

Rant: Git is not a version control system. But you're going to use it as one anyways.

I can hear you saying it.

" You're wrong, Delphi Code Monkey, Git is a version control system.".

Hear me out.

Git is not a version control system, it's a directed acyclical graph management system that happens to be possible (sometimes) to do version control with.

Every time you find something Git can't do, that it ought to do, for all of the billion people who probably use it and know it well,  if you keep searching for ways, however torturous, to handle your issue, there is probably a way.

There is stashing, and submodules, branching, and merging, there are cherry picks, and there is rebasing, there are branching models and workflows like Git Flow.        There's recursive submodule hell.  There's local git cached state hell.  There's ignore hell.   There's Git's unix roots are showing on windows hell.
There's git config hell.    There's unable to check out a branch because of local changes, and yet git status shows no files have changed, hell.   If I sat here and thought back through the years of horrible things Git has done to me, one by one, I would probably scream.

There are a thousand fresh hells waiting for you.  A thousand inscrutable (but usually googleable) error messages.

Here's the worst of the Git hells.  Git lockin hell.  Once you choose Git,  well, it's Hotel California.  You can "check out" but you can never leave.   The world of software development is now, and probably always will be locked into some version of Git or other.

Git is not actually (much) of a version control system, but you're going to use it like it was, and you're going to learn to like it.

Update:  The point of this rant (and it mostly is just a ranty rant) isn't that you SHOULD NOT use Git, it's that the nature of Git is to expose all the guts to you, and to blow up with errors that are because the internals are in a bad place and Git leaves it for you to put Humpty Dumpty back together again,  and that you are going to do that a LOT when you live and work all day, especially in a team of 5+ people, with Git.    Perhaps if you are saying to yourself, "I don't have all these problems that other people talk about", it could be because (a) you're not using submodules, (b) you're not working with teams of 5+ comitters, and (c) you don't use a lot of branching, merging, rebasing, cherry-picking nor complex workflows like git-flow.

Friday, October 23, 2020

A few quick tips about Downloading Delphi 10.4.1 (sydney) and all future Delphi Releases for current active customers.

Probably this is old news but I didn't see the news whenever it hit, and so I'm posting this here in hopes that other people who hadn't realized that EDN/CodeCentral is no longer where you get your Delphi downloads from, will Get The Memo.   Did you Get the Memo?

 1.  Don't bother trying to find 10.4.1 Sydney in the old EDN products portal, the one with the url starting  10.4.1 isn't in there.  I wouldn't be surprised if this site went away at some point. 

2. If you have an active subscription the place to get 10.4.1 Sydney is from the My Embarcadero Portal using the same login credentials as your active subscriptions and your licenses are associated with.

3.  Even very old stuff like Delphi 7 isos is now on the new site. It does seem that the same file pile of your purchases from the Borland, or CodeGear, or the pre-Idera Embarcadero era, are all on there, and there's no loss of continuity, and as far as I can see, there's no reasons to complain about the change, it seems good.

It's nice to see that the ancient EDN stuff is finally being rebuilt.  I still have some complaints about the download and install experience. For example, why isn't TeeChart available both from GetIt instead of only as a one-shot-miss-it-and-it's-gone a checkbox during initial install?  

Wednesday, July 22, 2020

Git with Delphi: The Basics

I wrote this blog post as I ran into a Delphi coder who was using some non-verson-control tool like Zip or DropBox to store their code.  Now I'm not saying you shouldn't ever store code in zip files or upload code to dropbox, but that should not be your source of truth.  This is a highly opinionated fly through of the basics that a non-git user should be aware of.


What is source control?  Source control means having a single  SOURCE OF TRUTH for what your code is.    Ideally that single source of truth should be on a private or public repository hosted on a git server, like bitbucket, or gitlab.

Why GIT?  There is literally nobody policing this, if you're a solo developer, pick whatever version control system you want, but if you pick anything other than Git, you're going to eventually probably  regret it.   There are network effects (an economics term, really) here. I'm not going to explain network effects. If you have to google that, go ahead.

What is a source of truth?  Git provides a source of truth because all git commits have a "revision hash", a fingerprint that marks them uniquely.  Git provides a way to say, for sure that if you have git commit A939B9.... (more digits omitted), you know exactly what content every single file in your produce source code contained.   In order to be really sure about it you should actually also be using Continuous Integration instead of building things on your developer PC.

What is CI (continuous integration)?   Another reason to use git is so that you can have your code built automatically whenever you change it and have that build happen in a controlled way, in a controlled and repeatable environment so that you actually know what you sent your customers, if you have customers.  If you are working for a company that sells software, and the software binaries you put on customer PCs were built on your own desktop computer, and not on a build server, and your company is more than just one person, you also should be using Git, so you can get ready to be building your software from an automated build (CI) environment.

What are these unit and integration tests things people talk about? You have those right.  If you have Git, and CI, and unit tests, you can be running those things every time your code builds. Are you beginning to see possibilities unfolding like flowers?  If you are a single developer, having CI and unit testing, and continuous unit testing on every git commit, can really help you.

How do I get started? The rest of this post lays out a basic plan for getting started with Git.

Wait I have objections...    Post them in the comments. I prefer subversion is an opinion, not an objection, but feel free to tell me that if you like. 

Step 1.  Sign up for an account on bitbucket.

Step 2. Download Source Tree. (The easiest Git client to learn, not the fastest, or most powerful, and not one you will stay with forever, SourceTree is good at holding new user's hands and getting them some critical early experience with version control, especially if you haven't used version control before and are coming from a background of using Zip or Dropbox to "back up"  your code.)

Step 3.  Create a dummy repository in bitbucket and clone it with source tree.  Follow their tutorials.

Step 4.  Create a delphi application with File -> New Application and commit that source code for the brand new main form right after the first time you save the project and main form and give them the NAME they are going to have for a while.   

Step 5. Using your  git GUI, Add and Commit the files to the local git repository. You have only now been working with your own files on your own hard drive so far.

Step 6. Push your commits up to the repo you created in bitbucket.  Log into the bitbucket website and find the code on your bitbucket account in the repositories page.  

Step 7.  Go to a different computer (if you can) or a different folder on the same computer (if you only have one) and clone the repo to that location.  Observe for yourself that Bitbucket is like a Transporter for Code.  The code on my computer is now code on another person's computer.  Yet another use for Git. Code transporter.  Team coding enabler. Backup system.   Historical log of what your code was over time.  And a thousand other things.  Once you have internalized and understood the value of even 5% of what a good version control tool can do for you, you will never work without one.

Step 8.   Make the application do something.  Fibonacci sequences.  Sorting interview question. Anything.   Once it works, Commit the working code, explain the code you wrote in your commit messages always. Get into good habits to start with so you don't have to learn the hard way.  Now mess the code up on purpose. Now using your Git Client (SourceTree or others) find the change and DO NOT COMMIT it, instead revert it. You have now observed another use for Git.  The Time Machine.  Any time you move your code in a way that makes it worse, you can just revert the change before you commit it.     

Step 9.   Make a bad change and commit it even though you know you just broke your app. Now find the way to revert a commit (get your code back to the way it was before the bad commit).   I'm not going to tell you what the command is called, because I want you to figure it out for yourself.   So git can not only be a time machine BEFORE BAD COMMITS it can also be a time machine to get you back code after bad commits.

Step 10. Now learn about .gitignore.  Do this before you import any real projects into git.  Find out what .gitignore is and how to write expressions into it like *.dcu and *.local.   Do not commit binaries (exes, dcus, dcps, bpls) into Git, unless you really really have to.  In general, you should NEVER EVER commit a DCU for a .PAS file that is in the repo as you should be BUILDING that .DCU file from source.  If your projects are badly configured (DCU Output folder is NOT set) then get used to setting the DCU Output folder option in Delphi to a sensible default for all your projects. Mine is .\dcu\$(Platform)\$(Config)   so that I can have win32, win64, and even other non windows platforms all build and not have my dcus get mixed up.     .gitignore is the place to code these dcu and exe output folders in so that when you add your source code into a git repo you do not commit the exes.   If you really really want to store your ssl dlls in git, nobody is going to come and arrest you, but please for the love of the flying spaghetti monster, learn what .gitignore is and use it. Don't commit DCUs. Sermon endeth.

Step 11. After you understand git ignore then take your source code folder and do a git init there, from source tree.  Add things in small chunks. Not 10K files in one commit. Add one top level folder that contains less than 50-80 megabytes of content, at a time, if you can.  Trust me you'll be glad you did this, if you have a 1 gigabyte project, please do not add 1 gigabyte of files in a single commit.  After each commit, add more files until your repo is built.   If you were previously using mercurial or subversion there are tools to keep your subversion and mercurial history and convert over to git.  I'm not going to cover them but I know they exist and that they work as I've used a variety of them.  I generally don't bother. I just start over with new history in git, and I go back and look at the old version control systems to see old versions.

Step 12.  Get used to comitting all your code after you write it and writing GOOD COMMIT MESSAGES. A good commit message is a message that explains the purpose of your changes.   If you are lazy here, your future self will hate you. You don't need that kind of rejection in your life. WRITE GOOD COMMIT MESSAGES NOW.

Step 13. Push after every commit until you have made it a habit.

Step 14. Delay using and learning about submodules, branching, tagging, and other topics in Git until everything I have said above is well ingrained and is part of your habits.  Learn to use Diff tools, learn to merge, before you learn to branch.     Imagine you learn to drive a car but you concentrated on learning how to drive fast before you're very good at steering and braking.     Steering and braking are the ways you stay out of trouble.  Merging, and diffing are ways to get out of trouble. Submodules, and branching are ways to get into it.  Until you have the tools to sort yourself, just don't make the crazy advanced messes. Don't try to deploy and manage a Gitlab instance at cluster scale when you don't even know how to Diff two files, compare them, and solve merge conflicts.

I believe the above will be helpful as I've written it, but I reserve the right to amend the advice above if anyone can point out ways in which the above might lead a new Git user astray. The happy path, of a contented Git user, is a gradual growth in competence with a powerful tool.

Can git be frustrating? Yes. Is it complicated?  Yes.  Is it worth it to learn it? Also Yes.

Are most of us going to do things wrong, and learn everything the hard way? Yes, because you are human. It's okay. 

Friday, May 29, 2020

Please DO NOT use the Jedi JVCL JvCsvDataSet component. I wrote it and it's not safe.

Once upon a time I was a Jedi JVCL developer. I contributed JvCsvDataSet, to the Jedi JVCL project.

Unfortunately, I believe it has several internal design flaws and needs a total rewrite, and also may be incompatible with modern delphi Dataset field management.

I do not have time to go debug the horrible memory and pointer bugs that are lurking deep inside it, but I believe the component should be deprecated.

I wrote some of this code, some of it is based on very old code from a book written in 1998, and it's also been altered, both by me, and by other people, over its long history.   I think it should be moved into a legacy folder and removed from a future JVCL version if no longer maintained.

Like I am saying above, and I'm saying again, I don't have time to fix it, and I believe it's broken in Delphi 10.0, 10.1, 10.2, 10.3, and 10.4.

If it's being used in a little utility program fine, but if you're building your app around this component, please rethink that.  Please rip it out and use FireDac TFDMemData and write some other CSV import and export functions.


Tuesday, March 24, 2020

Delphi Seattle becoming almost unusable on Windows Build 1909

I have previously posted about fun adventures running insider preview builds of Windows 10, with Delphi.    Now, I no longer run insider preview builds because the problems are just not worth it.

However, as Microsoft believes your computer belongs to them once you put Windows 10 on it, you now have to deal with your "windows 10" (pronounced "Windows is like a box of chocolates, you never know what you're gonna get") system will daily, or weekly, break stuff and die randomly on you.

In the midst of a global panic over a real virus, I would like to add that I feel that Windows 10 is a virus.

I can't rely on Windows 10 not to :

1. Ship new stuff I don't want.

2. Turn back on options that I turn off, like windows defender realtime protection.

3. Break production binary applications, for which I have no source code, most importantly Delphi itself, which I need to do my job.

4. Work the same way on all the hardware I own.  One build of windows will be a disaster and crash repeatedly, but work fine on other hardware I own. Build 1903 worked on one of my PCs but not the other, and now build 1909 works on one of my PCs and not the other.

Most recently, I can no longer operate Delphi 10 seattle which is the version my employer's production codebases are relying on. (When you work in a team, by the way, one person can't just move versions, so if you suggest that I should move to a new delphi version, I'll just ignore you as we're going to do that when we all do that, as I belong to a team.)

Here are three of the fun problems that I'm seeing:

1. If ever you had to set up a new PC on a new windows build and get it going with Seattle, and Windows build 1909, good luck.  To get a preview of the fun you'll have, you could rename (god forbid you should ever ever delete) the registry BDS\17 registry key, Delphi can not create a valid new setup and start up without crashing. You get an access violation in coreide230.bpl.  If you have a valid backup of a working BDS\17 registry key, you can recover it.  Anyone using Seattle on Windows 10, please keep backups of your BDS\17 key under your Current User registry settings.

2.  (UPDATE:Fixed in recent Win10 quality update) Delphi is unable to build our projects without dying during the write DCU phase randomly. This was always bad on some peoples computers, but usually not on mine, and now I can no longer operate the IDE and compile or build the main projects I need to compile for my day job.

3. (UPDATE: Happens less in build  1909 than build 1903, but still a problem.) Delphi's debugger when reset, is hanging the Windows 10 1903 and 1909 kernel hard, on some systems, for some applications, mostly ones that seem to have certain DLLs loading and unloading in them.

I'll post solutions if I find them, but if I don't, I'm going to move into a Win7 VM for all development, because Windows 10 is not something you can trust.  What's most odd is I have a Win10 VM running build 1909 and this issue is NOT happening in the VM.

UPDATE:  On the latest quality update of Windows 10 build 1909, on the main machine that was having these issues, the problems with the bds.exe executing delphi compiler within the IDE  (point 2) is resolved.

UPDATE2: Still no workaround known for Point 1 other than to keep your working backups of registry in current user.  

UPDATE3: A new issue that is less grievous but still annoying has appeared.  The USB User Mode Driver Framework has started hanging. The main symptom is that all USB attached peripherals including external mouse and keyboard stop working for about 30 seconds while some core Windows user mode driver framework bits restart themselves.  I can trigger it by resetting the delphi debugger. Something about terminating the process being debugged, which may be holding onto a handle for some device driver related object somehow, seems to trigger a crash in a usb driver, which somehow then crashes the host process. 


UPDATE4: Although my main work machine is unaffected, build 1903 and build 1909 are "bricking themselves" (failing to boot) after something in the windows update process corrupts the boot time data storage area (BCD) of the Windows boot volume (Drive C).  This happens repeatably on certain Lenovo Thinkpad hardware. I suspect a kernel-mode panic during boot due to a driver bug in the chipset drivers.