Sivv Open Source Projects by Jason Southwell

Just wanted to point out these open source projects written by Jason Southwell, who in addition to being a well known Delphi developer out there is also my boss! Hi!

Anyways, I use these libraries where we both work, and I think you should all check them out.


SIVV Open source is on bitbucket ...

A few example projects:

Chimera  Networking and JSON Library

Chimera project includes chimera.json, a very fast JSON library, and a bayeux (pubsub) network protocol  client, and other things.

CocinAsync

Networking/threading/async library. It recently grew some new "flux" like capabilities for event driven programming.

DuckDuckDelphi

A nifty duck-typing facility built over delphi RTTI facilities.  As the comments state in the project source:

//  Instead of:
//    if obj is TControl then
//      TControl(obj).Visible := True
//
//  You can simply call
//    obj.duck.setTo('Visible',True);

The call will do nothing silently instead of blowing up if you had done a bad runtime cast.  It's a nice pattern.



Discussion and questions on these components is at this site:

https://sivv.com/

Source Control, Version Control, Change Control, and the Underlying Disciplines Thereof

So, out there in the world of developers, Delphi and others, there are many schools of professionalism, around:

• Version Control Practices
• Continuous Integration Practices
• Release Management and Change Controls

Let's imagine there's a dial from 0 to 10, and at each number on the dial, there's a developer or team that is functioning at that Level of Versioning/CI/Release/Change practices.

0.  Dude with laptop, ships code from his laptop to customer pcs. 

1.  Dude had a bad day and lost data. Decides to up his game to using Zip files.  (For most of us old timers, I hope this day was back around 1991 for you.)

2. Dude sees the beauty of being able to check in, revert, commit.  Dude is saved and has seen the Version Control System light.  (This happened to you in about 1999, hopefully and your first version control system was Visual Source Safe, hopefully you are not still using that.)

3.  Dude realizes that checking in frequently is great.  Dude has not realized that Branching is useful, or is using a tool in which branching is not very useful (like Subversion).   Dude needs to learn new tools and practices, but doesn't want to.

4. Dude is member of a team of Developers.  This team works well together, so they have to know what branches are, and merges. None of them know what Continuous Integration is.  The team builds and ship from a designated lead developer PC.


5. Dude & Team had a bad day, and learned to love CI.    The CI system is a bit sad but hey, we've got one, and that makes our Kung Fu better than teams that don't.   No longer does The Dude ship code from his/her laptop to customers.

6. The Dude and Team, do CI, and honestly tried to start Unit Testing, but that never really worked.  There are millions of warnings and hints in the code.    When the code somehow builds without an error, we hand it to QA then ship it.   If you don't have dedicated QA people, you are still level 5, hire a QA, and have CI, and you're level 6. Congrats, you're probably above average in the Delphi world.

7.  The Dude and the Whole Team are conscientious about cleaning up hints and warnings and is working on getting code under unit test.   This is now better than about 60% of delphi shops.

8.   Dude and Whole Team Always know what was shipped to customers, as each version like Version 1.6.7.85080 is  known to correspond to a certain CI build which in turn corresponds to a known revision in version control.   All the executables are re-tagged with versioninfo updating tools during builds.  This is now better than about 70% of Delphi shops.

9. All the prior best practices are followed and also, All code that we build also passes tests or it doesn't go to human QA. These tests cover 10% or more of real bugs that QA would find. You are probably in the top 20 best Delphi shops in the whole world.


10. All the prior best practices are followed and also,All code that we build also passes tests or it doesn't go to human QA. These tests cover 80% or more of real bugs that QA would find.  You are a unicorn, and you probably don't exist, or you're working in Go or C++, not Delphi. 

So far, I'm not aware of ANY delphi shop that can claim to be in category 10, for even the top four largest codebases at your company.   Typically a few products are in good shape, and the others are pretty rough. From comparing notes with other Delphi professionals, most best of breed Delphi shops are probably in categories 6-9 above, all of which are Pretty Good to Really Good, in my opinion. 

But Pretty Good and Really Good is still basically much lamer than many Go, Ruby, and Java open source projects out there, that you can go inspect and see, are being built with levels of professional practices that far exceed what I am personally aware of within Delphi based teams.   If your Delphi teams blow away all the rest, please please write a blog post and explain how you do it.

Some truly impressive open source projects you can observe a whole higher level of practices going on include the C# roslyn compiler and the .net core open source project, the gitlab version control system (built in go, and ruby), Google Chrome,  Firefox, and SQLITE, which has a truly impressive set of unit tests and change control practices around its C/C++ core. 

I want to move on to the question of WHY the best practices matter.

 A lot of the principles that underly the best practices above should be obvious to most professional software developers, but sometimes these principles are left implicit,  and not spelled out.

Here are some principles that lie beneath the practices above.  I have seldom worked on a team that couldn't do something a bit better.  The best teams were always asking, "how can we improve?". The worst teams were always trying not to think about it. 

1.   Change Control Disciplines : We want to not make accidental changes, or even have people intentionally commit things, that were not wanted.   How are we going to minimize risk of accidental or just random developer "felt like doing something" and that ending up in our product without anyone being aware, or testing for it?

2.  Quality Control Disciplines:  How do we ensure we know the risk and the proper testing scope, for any release of software we work on?

3. Version Tagging Disciplines:   Does everything contain what it says on the tin?  Is version 1.0.3.10231 enough information to recover the exact source code set that was used? (If you used CI, and build 106 in jenkins, became installer-1.0.3.10231 , and you can see which git or svn revision that was, you can recover sources to any version in the field, and accurately patch it.)

4. Code Hygiene and Code Quality Disciplines : Do we fix warnings (especially the ones that lead to direct bugs, like uninitialized local variables) before we ship to customers?   Do we keep warnings and hints very close to zero so that really bad stuff doesn't hide "in the weeds" in plain sight?  Once all the real ugliness is out of the way, some teams graduate to Code Smells, and Refactoring, to get the architecture in a better place so the product can grow and change and not collapse under its own weight.  Code Hygiene and Code Quality disciplines are a never-ending asymptotic series of goals and ideals, but within those disciplines is a path out of drudgery and fire-fighting leading towards coding quality software that your customers will love. 

There are more details and things to say about the kinds of disciplines involved in the points above, that are not even talked about, there's a lot to be said about how to move from ball of mud (classic delphi) to TDD and well-factored code that is not a ball of mud.   Those are topics for another day.

New Idera/Embarcadero Forums

Folks who might have still been on the ancient Newsgroups might not have ever used the reboot of the Embarcadero Support Forums, and those are gone now, and have been replaced by some very modern and nice looking Embarcadero/Idera support forums which are located here:

https://community.idera.com/developer-tools/general-development/


My old community login did not appear to get moved over but it only took a second to get registered, and the site appears to be fast, responsive and easy to use.   I like it.

Windows 10 insider builds may interfere with Delphi 10 seattle IDE operation

Other IDE versions may be affected. I will update this blog post if I figure it out.

Here's what I think is happening:

1. Something about environment variable handling has changed.

2. The PLATFORM environment variable during an IDE build is somehow wrong.

3. The IDE fails to build some or all projects after this Win10 feature update. (Currently in insider preview).

4. One or more units contains a precompiler directive in the form {$I  filename}.  This is being included at some place where it is causing a problem. For example:

{$I versionchecks.inc}

MANY many third party components (open source and commercial) contain checks of this kind, and some may suck the includes in in a certain standard place, like the very top of the unit.

5. Putting a precompiler include before the Unit name appears to be problematic.

5. Putting a precompiler $I directive after Uses keyword, in the middle of a Uses clause, appears to be problematic.


Insider preview build 1903, build 18348.1 appears affected.

Typical compiler error example:

[dcc32 Fatal Error] XUNIT.pas(408): F2039 Could not create output file '.\dcu\Win32\Debug\XUNIT.dcu'

or

[dcc32 Fatal Error] F1026 File not found: 'YourProject.dpr'

Workaround:  Use the DPROJ option to use MSBUILD to compile externally. 

(Helps if MSBUILD from command line remains unaffected.)

This is not reproducing on all windows machines for me.

GetIt is broken, please fix it Embarcadero

Various people, including myself are having problems with GetIt returning license errors instead of installing the thing it should install. Even on free items like the Jedi JVCL library, you can get this error:

It's not a blocker when you can install Jedi yourself from source (clone the git repo, download a zip from sourceforge, but where it really affects the product is when you build an Android firemonkey app the first time. You get a prompt to install the tools with getit, and getit having failed, your only choice is to uninstall delphi and reinstall from the ISO, instead of from the web installer.

I hope Embarcadero fixes this soon. If anyone knows a workaround to get Getit to function, I will edit this post, and I will update it once a fix is released.  Above is with Delphi 10 seattle, which many projects I work on are using still.

UPDATE: I have heard from an Embarcadero person that a fix for this is unlikely.  You can get the JEDI JCL without using GetIt, but some things like CodeSite (Embarcadero edition) are NOT available anywhere else, not even in the EDN Registered Users Downloads section.   If anyone has a list of Seattle accessories/tool-installation downloads that are missing from EDN and no longer can be fetched from GetIt, I'll post links for workarounds here, as any appear.

Delphi starter is back as "Delphi Community". Good news for all Delphi users.

I am one of those codgers old enough to remember Delphi 6 personal, and the Turbo Delphi era (the next free edition name for delphi), and Delphi Starter, and each time the pattern is the same:

1. Announce it.

2. Hope it works.

3. Wander away and lose interest.

Let's hope that DOES NOT happen this time. Delphi needs this "free" starter edition.  I wholeheartedly agree with the wiser and saner heads who will from time to time argue that this needs to come back if it's currently in a hiatus. Not dead, you understand, just resting, like that parrot in Monty Python.

With or without a free edition DELPHI IS NOT DEAD, but with a free edition it's healthier. We need a tools ecosystem, that's an essential part of modern software development. A healthy ecosystem needs us to be able to build free stuff for Delphi.  With a free edition of Delphi, I hope it will always be possible to run continuous integration servers, for example, for open source projects written in and for Delphi development, and not need to worry "how do I build and run its unit tests and deliver the installers for our product/package". 

We need encryption libraries. We need REST server and rest client frameworks and add-ons thereto.  We need something approaching the breadth and depth of the ecosystems of Java and .Net, if Delphi is to continue to exist as a sensible way to work in a web-everywhere connected world.

So delphi Starter, ahem, Community, is back. I for one am glad.

Windows 10 Is Terrible

You would think that Microsoft, and Apple, having some of the highest paid software engineers in the world, would generally deliver higher quality software, that makes its users happy.

You will notice I didn't say Best Software Engineers.  I think that such a term is meaningless. We are all human beings.  The things we produce have bugs. And worse than that, because bugs were accidents, we hope, there are misfeatures. There are misbegotten elements in Windows, and Mac OS, that you cannot alter. They are part of choosing to use these products.   There are serious problems in Linux, but there are literally no misbegotten elements in Linux that cannot be removed. You can literally do anything to Linux.

But since most of the professional world runs on Windows,  and most of the Creative World (including most of the music workstations, and pro digital photography world) lives on Mac OS, both mac and windows users have a common problem.  You probably don't know, you even probably don't want to know, what your operating software does,  and whether you do, or you don't, you can't fundamentally control it.

I will write a separate post on my Linux focused blog on why Linux is terrible, but since most of my Delphi blog readers are, like me, stuck using Windows, either on your PC, or in a VM, this rant is mostly about why Windows 10 is terrible, and why staying on Windows 7 forever isn't an option either.


1.  You don't actually control your computer anymore.    

1.1  You have no way to stop system reboots.  Hello, Microsoft Here. We've decided to reboot your machine.  You can't say no.  This is acceptable?    Microsoft will of course counter that they give you some control now, you can say, well I'm working from 9 to 5, don't reboot then.  And you can say I use my computer actively these days. You can't say, no, let me decide when to reboot.   It kills me that even Microsoft employees get Windows 10 reboots while they are giving presentations on how great Windows 10 is.

1.2  You can not disable Windows Defender, unless you want to install something worse.   You can temporarily disable the Windows Defender Realtime feature, but it is automatically re-enabled after an unspecified amount  of time. From what I can see that time is about 1-2 hours.  The service that you could have disabled in Windows 7 to prevent Windows Defender is no longer possible to disable unless you take extraordinary steps to grab permissions of registry keys, and do some low level hacking that could easily damage your Windows install.

1.3  You can not keep permissions on NTFS files anymore.  If you, like me, think that the C: drive root can and should contain folders that are writeable by a non-admin user, Microsoft  disagress. They reset all folders that start with the root of the boot drive as part of the regular security sweeps of your system, and you can not disable this. This breaks several applications that I maintain and develop.    It seems these sweeps are associated with windows updates, so when Windows 10 updates to a new kernel, it does these NTFS permission updates.

1.4 You can not be sure you'll have 100% of your CPU and disk bandwidth available, at any time. Who is using all my computer's resources right now? Oh yeah, Microsoft. There are many privileged tasks that  you can not fully control or stop, including superfetch, the trustedinstaller, and there are many that do not tell you they feel like using your whole computer that at least can be paused.  The windows search feature, which is useful, is not configurable so that it doesn't use your valuable computer resources when you're trying to, say, write code and compile your applications in Delphi and then debug them.




2.  Windows 10 is gathering information about your computer, and you don't know what it is.

Some information has been revealed about what kind of data Microsoft collects, but you still can't opt out of this, and you don't know what data it sent from your computer. In the latest Creators Update you have some control over the level of data gathering, but Off is not an option.  This is frankly, unacceptable.  On my main Windows 10 box, the Privacy page in the Settings app, has some new options, but "opt out from all telemetry and data transmitted to Microsoft" is not one of them.    I have to say, it is nice that there is a web page where you can view the data they have that is associated with any email address you may choose to log into that site with.  But even they (and you) may not actually be aware of, and may not, via this web site, be able to say "we're not monitoring you".   They can only say, "here's data we have for that microsoft account, associated with that email".   

They do not claim that the data you clear is everything.  In fact, the EULA makes it clear that it's not.  You can clear out your browsing history but not all your telemetry data.  Why do I even care? Because it's a principle.  Surveillance that is used to send your CPU load history, that can be used to see performance regressions, arguably helps Windows users. But it should be my choice to opt into even the most benign data collection programs.   That your browsing history (for Microsoft Edge only) is clearable, is good.  All they give you there, is good, assuming you trust them. It's just not enough. It's sort of a smokescreen.  I'm not saying that they just pretend to delete data, but I am suggesting that the real picture might be a lot more complex than they want to admit publically. They don't actually know who had access to personally identifiable data, that may do something with it, that is an abuse.   They can't because that's a human problem.   The best procedure for privacy is to not collect data in the first place. Microsoft has become addicted to your data.  It's their lifeblood, they admit that publically.  They say it's the "lifeblood of Windows". No, it's the lifeblood of the new Microsoft.  Your life.

So why not stay on Windows 7? The ecosystem will continue to rot, and eventually it will become difficult to run Windows 7 on modern hardware. Microsoft has even started actively looking for ways to make Windows 7 not runnable on new hardware.    Windows 7 is almost as bad as Windows 10, but at least, when you disable the Realtime mode in Windows Defender it stays disabled.  Windows 7 is not ultimately a good solution.    If you have Windows 10 Pro, you might find that the Group Policy editor provides a "close enough for rock and roll" method for disabling Windows Defender, via group policy. If you have Windows 10 home, that method is not available.  Windows 7 has many of the same flaws (superfetch, boot time problems, update problems, windows defender problems) as Windows 10, but overall the symptoms are not as bad.  The most egregious problems in Windows 10 are somewhat less of a problem in Windows 7.  

What is the solution? I don't have one.  I'm just mad.