Hey Marco, Here's how to fix Delphi: Make it fast.

The title of this blog post is a bit provocative.  I think that Embarcadero is doing well with Delphi and I am excited about XE4.  I have been using it. But I need to complain today.

 I get paid to write Windows applications, and Delphi is still the best tool for that job.  But it needs some improvements. Here are the improvements that I believe would benefit everybody using Delphi.  Delphi's speed issues slowed me down today, and that's not acceptable.  I work with large applications in Delphi, some of which approach 1.5 million lines of code.  Delphi just doesn't handle them well.

1.  Ditch the Microsoft Document Explorer.  Even Microsoft has ditched it.    

2. Give me F1 context sensitive help that is faster than Googling or searching Stack Overflow.

3. Get rid of the 3-5 second delay popping down the Edit menu when you have a large project open.

4.  Fix the useless Error-Insight compiler.  Get rid of all the various parsers in your product and replace them all with one new one that can do all the parsing you need done.   The differences in the parser's support for various Delphi language features added between Delphi 7 and XE4 has resulted in the broken mess of code-insight, error-insight, and code-completion, not to mention code-formatting glitches.

5.  Learn from all the Java and C++ IDEs out there about using a background process to do do scans of the edit buffer so that it does not freeze up the editor, ever. It's never okay to block the user from inputting text while you have a big fat think. Never. Fix anything that freezes the editor long enough to cause users to lose data because the IDE is frozen while they were typing code.  This slowed me down and caused me to lose a good chunk of my productive working time today.


6. Some of my forms and data modules take 8 to 10 seconds to  open and become editable on a Xeon computer with 8 cores. This is just not cool.  What kind of stupid algorithms are you doing at DFM load time that could take 10 seconds to parse a DFM?  I could compile 100 megabytes of Delphi code in that time from the command-line compiler.

7. Make one checkbox in the preferences that says "Make it fast". Turn off all the slow stuff with that one checkbox, so that when what I need is an editor that has Delphi syntax highlighting, and a compiler and error list, it gets that right, and forgets everything else.

As cool as the Mac stuff is, and I will blog about how cool it is soon, what I want is a fast Delphi.

Delphi Prism is dead, long live Oxygene.

I just learned that Rad Studio XE4 will not include the "Prism"  (delphi compiler for .Net) product in the bundle anymore.

I for one am not surprised.  RemObject's delphi-flavored "Oxygene" language that powered the "Delphi Prism" product (a rebranding of Oxygene, since day one) has been more of a threat to Embarcadero than an asset, since the move towards iOS ("Nougat") and Java.

Embarcadero has a really nice plan, a cross-platform-Delphi go-to-market strategy, and I believe that their plans and RemObject's plans have been on a collision course for some time. As a consumer who purchases products from both, I am happy.  As I should be.  Because that means there will be competition.  I believe that the perception from inside the Embarcadero camp might be that they are a serious producer of compilers and native-code-generation tools, and the idea that .Net based "JIT" is "native development" is laughed at inside the hallowed halls of Delphi's developers, the idea being that  Delphi's core compiler and IDE technology is of an order of magnitude more importance and sophistication than the tiny little product known as "Prism" inside the RAD Studio box, and Oxygene everywhere else.

I don't actively use Oxygene/Prism, but I'm definitely interested in both it, and in XE4's cross-platform abilities when they ship.  As I have already learned iOS and Objective-C I don't need anyone to spare me that learning curve. It's done.  But I could use a tool that will let me build one app and target more than one mobile platform at once.

RemObjects' products, including Oxygene, are powerful precisely because they are built on top of a big open source ecosystem (Mono on non-Windows platforms) and by being built atop .Net are also leveraging powerful .Net platform capabilities, on Windows.

Ironically, the era of doing your own compiler from scratch is drawing to a close as Embarcadero prepares for a future where a new llvm-based Delphi compiler allows for native-compilation on iOS, and Android.  This is an era of collaboration and an era of competition, at the same time.  While Embarcadero may be "big" compared to RemObjects, both are small compared to the real juggernauts in the software industry.  Microsoft has the Windows platform just about sewn up, and VisualStudio has a commanding presence on Windows.   As Embarcadero and RemObjects set sail on the "blue ocean" of cross-platform development I wish them both the best of luck.

My personal bet is on the Embarcadero horse, because I like the idea of an llvm-based compiler and no runtime overhead and full static optimizing compiler tech, on mobile, windows, and mac.  But if that falls short of its promise for some reason, I am glad that there is another horse in this race.  Having Embarcadero and RemObjects as competitors is better for me, the intended target market for both company's products.

I am not on any Delphi XE4 field tests or betas and the information in this post is based purely on what I have read in public websites.

Update:I was not aware that the ios apps emitted by nougat were native compiled binaries... However I do not consider the dalvik stuff any less of a jit based java tech, dalvik does do preJit compilation, just a change of degree not of nature. Optimizers happen at device load time and are less effective. There is still an overhead in dalvik but it is part of android and you don't redistribute any runtimes. So when I say "native" I mean non jvm and non jit. If I am truly mistaken here I ask to be corrected.
Update 2: I just read the docs for Xe4. Zero based immutable strings? What the hell were you thinking, Alan? I have reversed my bet and am now betting on remobjects. I guess oxygene strings are like .net strings and are thus zero based and immutable but if I have to bear that kind of semantic seismic shift I think I also would like it named something other than Delphi when you're quite finished turning delphi into objective c with begin and end keywords.oh well. I must admit that Arc is pretty damn cool, I forgive you Alan, I am still super excited to try this out when it ships! This is probably the price for using a string type in the new compiler which participates fully in objective c classes as a toll free bridged equivalent to nsstring.
Update 3: I plan a followup that covers the reality better as soon as I get hands on time eith both xe4 Rtm and Nougat RTM. Color me a possible Oxygene customer.

Programming For Non-Programmers : No Silver Bullets, No Free Lunches.

Programmers tend to be fond of these aphorisms:

• There Ain't No Such Thing As A Free Lunch (TANSTAAFL) -- Robert Heinlein
• There Are No Silver Bullets -- Fred Brooks

But leaving aside our cynicism for a while, we also like to imagine local violations of these general-relativity-principles, and one of those areas of perennial optimism is the Programming Language for Non-Programmers.   That was, if you remember far enough back, the impetus for COBOL,  for the Fourth Generation Language (4GL) idea, for tools like PowerBuilder, and Visual Basic, and even, to a certain extent, for Delphi itself.  It resulted in some horrific syntactical nightmares, like AppleScript.

Where Delphi differs from the typical "tools used by people who write code who are not primarily programmers" is that Delphi offers a programming language with elegance, sophistication, and expressive power, that is not broken or hobbled by its syntax.  Not so with just about everything else in that list.   

However, it's time for people to try again, and the newest attempt I've see at "programming for everybody", is called LiveCode, from RunRev.  Go download it now, I'll wait.

What's great about LiveCode? A few things:

• The core product is free, and is supported by revenues from their tutorial videos (LiveCode Academy) as well as from their sales and support for their premium product.
• It's cross platform, on Windows, Linux, mobile, and Mac, in fact, it claims to support all the platforms that Delphi still has on the distant future roadmap.

What's not so great about LiveCode?   Well, if you remember back in the good old Borland TurboPascal days, one of the things they did was give you a language introduction in a book, like this:

Inside was a systematic, carefully laid out, and orderly presentation of core concepts, like this:

After a few pages, you begin to grasp a few concepts which you can use over and over again throughout your working life with Pascal.  That's another way of saying that Pascal syntax is orderly, does not try to be english, but is nevertheless readable, and the language is orthogonal to the task it performs.  Now let's look at how coding works in liveCode.  You drop a button and an edit box on a form.  Score for LiveCode is good so far.  You don't need a PhD or a tutorial video to figure out how to use it.  So far, it's as friendly as Visual Basic, C#, and Delphi, and far friendlier than Java.

Double clicking on the Button does not do what it should do. I invoke Visual Basic, Delphi, and C# as three common environments that know what's what. When you double click a button, you should immediately be given the coding context where you will write what happens when the button is pressed.  Instead you have to locate the properties-inspector analog, and find a button that looks like a CD player Play button, and click that and then click Edit Script:

Now we get the code window:

I'm rather proud of myself. It only took me 20 minutes to figure out that the way to set A = B is:

   put "Test2" into Field "Field1"

Let's unpack the concepts here for pedagogical types:

1.  LiveCode owes a debt most of all to 4GLs, AppleScript and HyperCard, and borrows some good ideas from all three.

2.  Put is one of the ways of writing assignments.

3.  The value that is put into something else is the first thing you write. This is backwards to most of us, who learned BASIC,  which has LET A = 5, which has always made perfect sense to me.  Instead we PUT 5 INTO A in this language.  Already, that's too much typing for me.

4. Next let's note that a field has a name "Field1" and that it is a Field, and that unlike a Delphi edit box named Edit1,  we must refer to it always as Field "Field1", not as just Field1.  

I managed to type a few malformed versions of that above line of code that froze the IDE completely, requiring me to Force Quit (end task), like this one:

put "Test" into Field "Field" to "thing"

The above statement passes the grammar-checker in LiveCode, and then goes into the internal works, and does something that wreaks havoc with the internals.  That, my friends, is the steep cliff that all "friendly" languages have lurking at all times.  Designing a grammar is a difficult task, and designing your own language leads to all manner of quirks.   

The lifecycle of these technologies, it seems to me is, well known, and is documented in various places and by various names, most famously as the Hype Cycle:

Am I saying that LiveCode is unimportant? No, far from it. I'm deeply impressed by the author's goal, and I, like them, dream of being able to teach 8 year olds to make video games, or do whatever else they want to do, in a language free of accidental complexity. But I disagree that emulating english with all its ambiguities is the way to do it.  I think that something more like Python is close to ideal for teaching programming. Nevertheless,  Python lacks a true Delphi-style RAD IDE.

There is a very delphi-like free and open source IDE out there that has enough of the basic Delphi IDE features, and enough power in the backing compiler to be most of what I want, but I still think Pascal is not as friendly as it could be, and so I have some hope still for LiveCode.  Since it's open source, I've been wondering if I could hack a bit on their language choices, and try to do things like this:

• Let users pick items and combine them using menus and toolbars that will result in code templates being generated, for common tasks like variable declarations and assignment, looping, and conditional checks.
• Let users write in a simple declarative style without line-noise or overhead:
     Field1 = "test"
• When users type something that is not understood, provide some kind of interactive help that helps them navigate their way out of the mess or confusion they are in.

Anyways, as an inveterate tinkerer, this gets top marks from me just for existing, and for being open source. (The code is all on github.) Check it out.

Update: I found a pretty nice open courseware (training) program here.  I have to admit, that what I said above about the concepts of the Delphi language being pretty, while that may not apply to the LiveCode language, which is an ugly duckling in the same family as HyperCard/AppleScript, but its parts/components style of visual composition of elements to build software, like the images of "lego blocks" in the training course I linked to, is a compelling way to introduce the construction of software systems to non-programmers.  Score another point for LiveCode.

Update 2: I forgot to mention two of LiveCode's ugliest warts.  The first ugly wart is that live-code stores its applications in a binary ".livecode"  file that you can't use version control with.  That means anybody who works with it is basically working without version control.   The second ugly fact is that it is not completely working with Unicode and UTF8.  There are many open issues with people using languages like Arabic (Right to Left Input, etc), and who cannot get dialogs to contain unicode data.
I think the most important change that LiveCode needs to make is to move to using a .livecode named folder, containing auto-named content files which are JSON format, and can thus be cleanly version controlled.  Even image-based development environments like Smalltalk have their own version control systems that work inside the image (letting you commit and revert, and check in and out right from inside smalltalk) but LiveCode does not. No version control in 2013? That's crappy.