If you follow one of the usual tutorials for Go programming they will start by trying to dump a load of things you have to do on you. This is perhaps something that you, as a long-time-Delphi geek have become inured to in your own environment. Let us imagine a developer who has been given access to a source code repository or server, perhaps a big subversion server, and has no familiarity with the Delphi codebase at CompanyX, where CompanyX is basically every delphi-using company ever. Let's make a quick list of the first tasks our developer would face:
Has any of that ever seemed insane to you? It does to me. And so when I look at new languages one of the things that I look for is if the problems above have been thought about and resolved in that language and its related tools, including its build system, if it has one, and its module system.
Go has been known to me for some years as a famously opinionated language, characterized by the removal of features that its designers felt were problematic in C++, and were thus removed from GO:
I think the above has the benefit of being about as nice a structure as I could imagine. The folder names above tell me even where on Github I might find this project. Source code is now globally unique and mapped by these conventions so that I know where to find any public code I want. If I want to use gitlab.com or bitbucket, or if it was stored on a private server inside my company named gitlab.mycompany.com, I would move my code into different folders to make that choice clear. For a language which is intended to be used in large systems, it's an appropriate design choice. Let's contrast this with Perl or Python where the intended use starts with one to ten line scripts that are basically used for any kind of little task you can imagine, and where this kind of ceremony would be stupid.
I have worked in enough large code-bases in Delphi, and where any form of organization is accepted, it will inevitably become horribly complicated.
Let's briefly discuss the forms of folder/directory organization one might try to do in delphi:
* Ball Of Mud: Everything for one application is in one or some small number of directories - It is extremely common that one directory contains 90% of the files that are not third party components and that directories are only used to hold files not written here. No sensible use of directories is made. The source should all be in one directory with 10K files in it. Usually ball of mud folder structure goes nicely with ball of mud source code organization inside your code files. That form with 5K lines of untestable business logic mashed into the form? That "controller" object that is more of a "god" object that direct references everything else tightly via USES statements? Ball of mud.
* MVC or MVVM: Views in their own folder. Models in their own folder. Controllers in their own folder. Additional folders for shared areas, and per application area. I've heard that this is possible, but I've never seen a Delphi codebase coded according to MVC. Ideally, if you're going to do this, you also don't have your Views reference your controllers or even have access to the source folder where the controllers live. Your models ideally don't reference anything, they're pure model classes and don't take dependencies on anything.
* Aspirational: This is the most common condition of Delphi codebases. There is some desire to be organized and some effort has been made, but it is fighting an uphill battle that may be unwinnable, because the barn door was already opened, and that cow of accidental complexity is already out and munching happily in your oat field. You have a desired modular approach and it's expressed in your code, but every unit USES 100 other units, and your dependency graph looks like something a cat coughed up.
So given that I have seen large systems get like the above, I have a lot of sympathy for languages like C# where at least you can get your IDE and tools to complain when you break the rules, and I even have more sympathy for Java where namespaces are required on classes, and where the classes must live in directories which are named and hierarchically ordered. In Go we have named modules, and the modules contain functions and can define interfaces, but they're not really Classes like in Java. But the idea of order and organization has been preserved as important. In Delphi we have the ability to use unit prefixes which are weaker than true Namespaces but still potentially useful. In Delphi but most code I have seen does not attempt to adopt them. It seems to me that having a codebase that uses unit prefixes, and that has source organized into these folders is a worthy future goal for a Clean Delphi Codebase, but existing legacy codebases are all we have, and so getting there is not something I'm going to hold my breath on. One has to have practical achievable plans, and not tilt at windmills.
My first reaction to Go's requirement to use a fixed structure was predictably the same reaction that I had when I first realized the horrors of Forced Organization that java was forcing on me when I first tried Java in 1996. Now, it's 20 years later, and I think we can say that Java's designers were right. Java has proven to be especially useful in constructing extremely large systems.
The go package dependency fetching system (go get X) works precisely because of this forced organization, and it's all very well thought out. There's NO reason that a clever Delphi programmer couldn't learn the lessons of how GOPATH and how go get and go install work and use that to fashion a guaranteed well organized and maintainable and clean delphi codebase, incrementally, by a phased approach.
You don't gain much if you close the barn door after the cow's got out, and you can't stop everything and rewrite, but if you can build some tools to help you tame accidental complexity gradually, you can restore order, over time, while you work on a system.
What goals might you start with? I'm not going to tell you. All I'm going to do is say that if your brain lives in a box that has a begin at the beginning, and an end at the end, and you can't read and think outside that box, you're sadly limiting yourself as a developer. Becoming a better developer (in any language) requires what the old timers even older than me called a "Systems Approach". A view of what you build, and your project and its goals that is larger than your daily work, longer in scope than whatever form of agile "sprints" you're doing, and which has a sustainable high quality engineering methodology behind it.
You can't build that kind of mentality in at a language level (in Go or Java or Pascal), but I think it does help to have the bar set progressively higher as you can, so that once code becomes cleaner and more maintainable, there is at least the potential to detect when someone has made things worse.
Thus far we have seen many programmers throw up their hands at the 5 million line big-ball-of-mud projects and consider rewriting it from the beginning. My feeling is that the bad patterns in your brain are still there, and if you rewrite it all in the same language or a different one, you're going to make all those mistakes again, and some new ones, unless you start learning some ways to approach system design that promotes clean decoupled programming. Studying and research phases are required. Do not race to reimplement anything, either in Pascal or any other language. Spend time and sharpen your sword. And remember Don Quixote.
- Setting up a working copy of source code so it builds, and so the forms open up without errors due to missing components.
- Associating package set X required to build version Y of product Z.
- Setting up library paths that might be completely undocumented.
- Individual things done at company X like mapping a fake drive letter with SUBST, or setting up environment variable COMPANYX to point at some shared folder location.
- At some companies they will just look at you blankly if you ask "can you build this entire product and every part of it from the command line on every developer's PC"? Other companies have exactly ONE computer (MysticalUnicorn.yourcompany.com) on which this feat is frequently possible. Still others (the sane ones) have made the process so unspectacular, and merely reliable that they think the ones who gave you the blank look just haven't realized how insane they are yet.
- At some companies it might be considered acceptable if the build scripts and projects and sources ASSUME you will always check your code out to C:\COMPANYX. When you want to have a second branch you simply clone and copy a tiny little 120 gigabyte VM and fire that up.
Has any of that ever seemed insane to you? It does to me. And so when I look at new languages one of the things that I look for is if the problems above have been thought about and resolved in that language and its related tools, including its build system, if it has one, and its module system.
Go has been known to me for some years as a famously opinionated language, characterized by the removal of features that its designers felt were problematic in C++, and were thus removed from GO:
- There are no exceptions in Go, only error returns, and panics.
- There are no generics in Go.
- There is no classic object oriented programming with Inheritance, there is only composition, and there are only Interfaces, there are no base classes (because no inheritance).
- The module structure is pretty much mandatory. Here's me starting a brand new Go project from a command line, what is happening should be pretty clear to most geeks:
What is the thinking process that goes into designing the module system, with the following structure:
~/work> export GOPATH=~/work
~/work> export PATH=$PATH:$GOPATH/bin
~/work> mkdir -p $GOPATH/src/github.com/wpostma/hello
~/work> cd $GOPATH/src/github.com/wpostma/hello
~/work/src/github.com/wpostma/hello> vi hello.go
package main
import "fmt"
func main() {
fmt.Printf("Hello, world.\n")
}
~/work/src/github.com/> go install github.com/wpostma/hello
~/work/src/github.com/> hello
Hello, world.
I think the above has the benefit of being about as nice a structure as I could imagine. The folder names above tell me even where on Github I might find this project. Source code is now globally unique and mapped by these conventions so that I know where to find any public code I want. If I want to use gitlab.com or bitbucket, or if it was stored on a private server inside my company named gitlab.mycompany.com, I would move my code into different folders to make that choice clear. For a language which is intended to be used in large systems, it's an appropriate design choice. Let's contrast this with Perl or Python where the intended use starts with one to ten line scripts that are basically used for any kind of little task you can imagine, and where this kind of ceremony would be stupid.
I have worked in enough large code-bases in Delphi, and where any form of organization is accepted, it will inevitably become horribly complicated.
Let's briefly discuss the forms of folder/directory organization one might try to do in delphi:
* Ball Of Mud: Everything for one application is in one or some small number of directories - It is extremely common that one directory contains 90% of the files that are not third party components and that directories are only used to hold files not written here. No sensible use of directories is made. The source should all be in one directory with 10K files in it. Usually ball of mud folder structure goes nicely with ball of mud source code organization inside your code files. That form with 5K lines of untestable business logic mashed into the form? That "controller" object that is more of a "god" object that direct references everything else tightly via USES statements? Ball of mud.
* MVC or MVVM: Views in their own folder. Models in their own folder. Controllers in their own folder. Additional folders for shared areas, and per application area. I've heard that this is possible, but I've never seen a Delphi codebase coded according to MVC. Ideally, if you're going to do this, you also don't have your Views reference your controllers or even have access to the source folder where the controllers live. Your models ideally don't reference anything, they're pure model classes and don't take dependencies on anything.
* Aspirational: This is the most common condition of Delphi codebases. There is some desire to be organized and some effort has been made, but it is fighting an uphill battle that may be unwinnable, because the barn door was already opened, and that cow of accidental complexity is already out and munching happily in your oat field. You have a desired modular approach and it's expressed in your code, but every unit USES 100 other units, and your dependency graph looks like something a cat coughed up.
So given that I have seen large systems get like the above, I have a lot of sympathy for languages like C# where at least you can get your IDE and tools to complain when you break the rules, and I even have more sympathy for Java where namespaces are required on classes, and where the classes must live in directories which are named and hierarchically ordered. In Go we have named modules, and the modules contain functions and can define interfaces, but they're not really Classes like in Java. But the idea of order and organization has been preserved as important. In Delphi we have the ability to use unit prefixes which are weaker than true Namespaces but still potentially useful. In Delphi but most code I have seen does not attempt to adopt them. It seems to me that having a codebase that uses unit prefixes, and that has source organized into these folders is a worthy future goal for a Clean Delphi Codebase, but existing legacy codebases are all we have, and so getting there is not something I'm going to hold my breath on. One has to have practical achievable plans, and not tilt at windmills.
My first reaction to Go's requirement to use a fixed structure was predictably the same reaction that I had when I first realized the horrors of Forced Organization that java was forcing on me when I first tried Java in 1996. Now, it's 20 years later, and I think we can say that Java's designers were right. Java has proven to be especially useful in constructing extremely large systems.
The go package dependency fetching system (go get X) works precisely because of this forced organization, and it's all very well thought out. There's NO reason that a clever Delphi programmer couldn't learn the lessons of how GOPATH and how go get and go install work and use that to fashion a guaranteed well organized and maintainable and clean delphi codebase, incrementally, by a phased approach.
You don't gain much if you close the barn door after the cow's got out, and you can't stop everything and rewrite, but if you can build some tools to help you tame accidental complexity gradually, you can restore order, over time, while you work on a system.
What goals might you start with? I'm not going to tell you. All I'm going to do is say that if your brain lives in a box that has a begin at the beginning, and an end at the end, and you can't read and think outside that box, you're sadly limiting yourself as a developer. Becoming a better developer (in any language) requires what the old timers even older than me called a "Systems Approach". A view of what you build, and your project and its goals that is larger than your daily work, longer in scope than whatever form of agile "sprints" you're doing, and which has a sustainable high quality engineering methodology behind it.
You can't build that kind of mentality in at a language level (in Go or Java or Pascal), but I think it does help to have the bar set progressively higher as you can, so that once code becomes cleaner and more maintainable, there is at least the potential to detect when someone has made things worse.
Thus far we have seen many programmers throw up their hands at the 5 million line big-ball-of-mud projects and consider rewriting it from the beginning. My feeling is that the bad patterns in your brain are still there, and if you rewrite it all in the same language or a different one, you're going to make all those mistakes again, and some new ones, unless you start learning some ways to approach system design that promotes clean decoupled programming. Studying and research phases are required. Do not race to reimplement anything, either in Pascal or any other language. Spend time and sharpen your sword. And remember Don Quixote.
