Saturday, June 29, 2013

ARC is awesome: My comments on Allen Bauer 's blog

Allen is a pretty talented guy, and he has a long history and deep insights into the history of Delphi.  I can confirm (from outside the Embarcadero/CodeGear fold) that everything he says is exactly the way it is, even the motivation for .Net being that Microsoft planned to "yank" the Win32 platform out from under us Win32/VCL programmers, and force everybody onto new "managed" .Net runtime APIs, to the point that existing Win32 APIs would be a "penalty box" environment, a sandboxed subworld inside the real windows, the equivalent of the NTVDM, the virtualization layer that kept 16 bit DOS mode executable compatibility around in Windows even as recently as Windows 7 32 bit.   You may remember that NTVDM only went away when we got to the 64 bit Windows world, because Win32 is a virtual layer inside the main windows environment, now known as "WOW" (Windows-on-Windows).    As being at the root of the platform's APIs is key to Delphi being Delphi,   the now mostly dead and buried "Delphi for .net" product seemed necessary to Delphi's continuing health and it having a future on a .Net-only Windows world, that never materialized.

If Embarcadero could ship a WinRT targeting version of Delphi, they would have already. But Microsoft is holding all the keys, as they have meticulously crafted a sandboxed, signed, Apple-App-Store rip-off mode inside Windows 8.   But WinRT is just a "penalty box" inside Windows 8.  Rather than enabling new features, it is primarily notable for what it takes away.   In Windows 8 AppStore/WinRT applications, your application does not get to decide when it runs, and is more heavily "managed" by a power-consumption-optimizing user shell that is more similar to Windows Phone,  iOS, and Android than it is to classical Windows programming. It's also heavily asynchronous, and if you haven't noticed code-samples for Windows 8 WinRT applications in C# make heavy use of the "magic" await keyword in C#.   I am not slavering for such magic to appear in Delphi, because such magic has a dark underside.     Just as .Net garbage collection has a dark underside.

Does ARC have a difficult and dark underside?  Not really.  But there are a few things to be aware of in reference-counting based memory management, things you probably already should know if you are a competent Windows COM/DCOM programmer, things like:

  • Reference counting cycles are something that happen when object A holds a reference to B and object B either directly holds a reference to A, or holds a reference to something else that holds a reference to A.    A "directed acyclic graph" means "a bunch of dots with lines in between them, and each line has an arrow, sort of line a one-way street sign, and you could follow the arrows any way you like, but never end up back at a node you had previously visited."  If your references are like that, they could in some way be considered a "tree" instead of a "spiderweb".  In your "tree" of objects there are no cycles.  If however your application has more of a spiderweb structure, then you will need to learn how to break these cycles.

  • Breaking reference counting cycles is easy to do in a CLANG/LLVM based ARC scenario, such as Objective-C in XCode, and the same is true in Delphi on iOS.  Weak references are the answer.  Weak references are really cool.  They are like a C++ smart pointer on steroids.   The compiler and its code generation back ends provide some really strong guarantees that enable you to work with a really easy to use programming model.  If the object pointed to by a weak reference still exists, you get that object. If it does not, you get a null (or nil if you like) object reference.  If you were curious how this worked when you first heard about it, you might want to read up on the Apple Clang/LLVM documentation on Weak references, because the Delphi implementation and mechanics are almost 100% guaranteed to be the same, since they're based on the same LLVM code.    Weak references are expensive.  They slow your application down, but remember that correctness (not crashing) is always more important than performance.  It would be a real mistake to pathologically avoid weak references where they are the correct solution just because they can introduce performance issues if abused or used too much.  On the other hand, if you can avoid circular references completely in your design, then you avoid the problem of refcount cycles for free.  Use your head, and choose on a case-by-case basis, and optimize after profiling and running and testing your code, not before.
Having used Objective-C with manual memory management (retain and release calls), and with garbage collection (which was crap, and is going away out of Objective-C), and now having used Objective-C with ARC, I am very happy to see ARC coming to Delphi.

I predict that when ARC comes to delphi on VCL Windows Desktop, a lot of old crap in your codebases is going to have to go away including:

  • Length delimited string syntax like  string[30] which is really the ancient shortstring type from TurboPascal
  • A lot of old compiler flags to turn on dos-mode real48 compatibility,  alias string to shortstring ({$H-}) 
  • UPDATED When I first looked at ARC, I thought Variant Records might be incompatible with it, but actually upon reflection, I'm guessing that there will just be continued restrictions on the combination of Variant Records with Managed Types, which is already the case today. You can't put an AnsiString in a Variant Record for example. So you won't be able to put an ARC object reference into a variant record.
I will be really surprised though, if the trial-balloon that Embarcadero has floated about non-mutable zero-based strings will fly.  I expect that is a deal breaker, as there are hundreds of millions of lines of otherwise clean code that still depend on Strings starting at position 1.   Parsing strings with Pos() and Copy()  and the lack of requirement to deal with a separate String and StringBuilder type are classic differentiators between a Delphi managed-heap-of-strings and a Java or .Net managed string environment.   Changing these semantics is not a simple matter of adding or subtracting 1 from a string position, it is a symptom of a fundamental difference between the classic mutable Delphi AnsiString and UnicodeString copy-on-write behaviours, and the behaviour of the type that it seems is being chosen to replace it, which is in the end,  something homeomorphic to the Cocoa/Apple immutable NSString type.   

  Homeomorphic means "same shape", or in this case, "if it walks like a duck, quacks like a duck, and plays poker as badly as a duck, then it is a duck".  It looks like the easiest way to move Delphi onto an LLVM compiler backend and ARC is to adopt immutable NSString as part of Delphi's base way of doing strings. I think that's a mistake.  I believe it is possible to preserve the appearance and semantic model that Delphi has always provided.  Strings are (to a user of the language, not to the implementor) as simple to work with as integers and characters. This differs from most compiled languages.  Strings in Delphi are as simple to work with as strings in Python or C#.   I am not aware of any other language that is strictly statically typed, and runs without an interpreter, JIT or VM, and yet provides as simple a string model as Delphi.   Don't even try to tell me that C++ has it, because while you could probably write a string class in C++ that only you even use, real world C/C++ codebases routinely mix 10 or 20 string-like types per codebase.  Erasing that advantage would not only break how Pascal/Delphi string coding has worked since 1984, it would prevent people from moving their existing codebases up, and would slow Delphi adoption.  

I expect that the new compiler and the old compiler will be co-hosted in a single Delphi IDE for  the next four to eight releases of Delphi, and that eventually the classic non-LLVM compiler will go away. During that time, it's critical that all Delphi codebases get transferred into a form that allows them to move up.  That means that ad-hoc hacks like "obj.Free" doing nothing, and, if this is going to fly, it also means that String semantics stay exactly where they are, with no attempt to even offer immutability or zero-based offsets is made to users.  The last thing we need is a new {$H+} and {$H-} compiler switch to replace the old one.

A simplified, rational coding model that can be easily applied and used without any possibility of bad things happening silently in the background, and without requiring users to rewrite around immutability or zero-based strings is required for this move towards ARC to be a success.

Wednesday, June 26, 2013

Build 2013 : Quo Vadis, Microsoft

After it became obvious to the Powers that Be at Microsoft that Windows 8 was bombing spectacularly, they went back to the drawing board, and said, "what do we need to add to windows to make it a commercial success?".

I believe Microsoft did a good job of that discussion, because what I heard today in the Build 2013 discussion on the new APIs and capabilities in Windows 8.1 they have thought about what kinds of tasks people might want to accomplish using Windows, for business, and at home, and what kinds of capabilities Microsoft could make available that are better than what other people are offering, and more importantly, they're making a bold "double down" on their investment in what can only be described as a mixed bag of successes and failures.  Or the worst version of Windows, ever.

With Windows 8.0 a public failure, and Windows 8.1 the plan to move past that, and with an incremental updates in Visual Studio 2013 to help people build apps for the app store, can Microsoft really turn around Windows 8's abysmal adoption figures and make something people will want to build on top of?    What? 100,000 apps in the Windows 8 app store? I wouldn't be publishing those numbers, that's closer to the Blackberry App store catalog size than it is to Android or Apple App Store sizes.

What do they have in Windows 8.1 that I think matters?  First, a new set of "Point Of Sale" APIs (They're calling it Point of Service), that look pretty slick.  Secondly, they've got an AirPlay clone baked into Windows 8.1, and a multi-monitor system including a "Projector" API that lets you make an application that has a presenter view and a public customer facing view. Previously WinRT apps had one and only one window, on one and only one monitor.  Which sucked.  So WinRT is growing.  And some of the RT APIs can be accessed from the desktop, including notifications.  All of this sounds good.  But it's not enough.  The Bing platform is cool too. Also, not enough.

What do they not have in Windows 8.1?  That's an even longer list:

  • NO complete and cohesive strategy to unify the disparate worlds of Windows Desktop and Windows RT/Store.  Imagine I am a software vendor with an existing Windows product.  How am I to feel about the split-personality disorder that is Windows 8 desktop (powerful business applications that run the world) and the Windows 8 store (Touch Screen Games  for Toddlers).
  • NO support for other programming languages or tools within the Windows RT/Store world. It's Visual Studio or nothing, C#, JavaScript, C++, or nothing. I want to build Windows RT apps using Delphi. Yes I could use Oxygene, but I want to use Delphi.  Embarcadero wants me to be able to do that.  But it would take a Microsoft effort to extend a hand to Embarcadero and get this going.  Until Microsoft cares about developers building stuff outside Visual Studio, I remain sceptical about the long-term health of the Windows Store and the Windows RT ecosystem.
  • NO changes to Windows RT that would make it compelling enough that I would want to buy a Windows RT device like the Surface RT, which Microsoft recently placed on an educational "Fire Sale" pricing, which tells me they've got a warehouse full of 'em and can't get them to sell. They're also basically giving them away to Build attendees. Maybe they should sell them to anybody who wants one, along with a 1 year WinRT Store developer account, at $99 each. Hey, that just might work. Are you listening Microsoft? Developers Developers Developers.
  • NO changes to Windows 8.1 that will make people who were previously negative about Windows 8.0 want to run out and buy a Windows 8.0 device like the Surface pro.
However, what they have not announced, I am meant to believe will be coming soon, using Microsoft's agile rolling release system.  If I believe in the message that Ballmer delivered, then the future of Windows looks very bright.  But on stage, what I see is a tired man, a tired angry, angry man.  A man who should step down and let somebody else rebuild this place.  But that's not what's going to happen. What's going to happen is Ballmer is going to radically reorganize Microsoft, as a part of his legacy.  Within 2 years I predict Ballmer will be gone, of his own volition.
I don't think he wants to leave on a low note.  Surface RT is a failure. Windows 8.0 is a failure.

The keynote did discuss a bit of start-menu/desktop integration.  A background image from Windows desktop is also now, if you like, the background image in the new "this time it's nice, no really we mean it" Start screen in Windows 8.1.  Wallpaper? Is that the best we could do? The fact is that jumping from one screen (your desktop) to another is against the grain, and against the entire history of Windows that has been baked into Windows since Windows first appeared on most people's PCs, in the Windows 3.1 era.    In the pre-start-button era of Windows 3.1, a Program Manager and a File Manager provided you with a way to launch programs.  Windows 95 brought the original version of the start-menu, the one that has hardly changed since that date.   A change was overdue.  But Microsoft clearly did not understand their customers, and went where the customers were not willing to follow.  Windows 8.1 still represents a leap that most users will not willingly make.

Quo Vadis Microsoft? Where the hell are you going? And are you sure your customers will follow you?

P.S.  The Channel 9 Video Host People are so awkward that I feel sorry for them.  Does anybody else think that they would be better off just letting the geeks and Microsoft staff just talk directly to the customers, and skip the whole Channel 9 Host "Personality" and "Color Man" bits?

Wednesday, June 19, 2013

Windows 8 user interface is useless on tablets. And useless on desktops. So it's just useless.

As a case in point for how useless Windows 8 is on a tablet, consider the common source of confusion since version 3.5 of Windows NT, when a printer queue is paused.  First users don't think about printer queues and so the idea of them as paused, is unnatural to users.  Secondly, Windows has never improved its print queuing.  First the queue itself (the list of documents awaiting printing) is hidden away in a sub-window, even in Windows 8, and secondly, the means of resuming is hidden in a pull down menu in a sub-window.

I can tell you how this happened. Incompetence. I'm not talking about the developers here.

I'm talking about the incompetence of Sinofsky, who has now been fired because he did too much too fast, rather than being fired for the reason he who should have been fired, which is for not rebuilding Microsoft more boldly than he attempted to.

Sinofksy had to fight the inertia of a giant organization, a giant codebase, and a lot of internally warring stack-ranked factions at Microsoft, and thus, in the end, I blame the incompetence of Steve Balmer for not giving Synofsky carte-blanche to rebuild the Windows organization as he saw fit..

Again, let me be clear. The incompetents working on Windows 8 that I am referring to are the now-fired Synofsky, and the still-CEO Ballmer, who created the toxic stack-ranked corporate culture and the internecine warfare style of company management  that in turn gave us the worst version of Windows ever, Windows 8.

P.S. If you're curious, the new Settings UI in the "Modern" (Metro) part of Windows just dumps you out to the desktop Printers and Devices pane, and there is no "modern" UI for working with printers yet. 

Saturday, June 1, 2013

The most awesome thing about Delphi is that it is not C++.

Recently, I was trying to debug some code on my weekend spare time, to help  out a former employer who needs a bug fixed in a C++ application.   I removed an ad-hoc shared memory implementation previously, and substituted a well known and trusted shared memory implementation, called Boost::Interprocess.

I really don't see a reasonable alternative to Boost::Interprocess for C++ shared memory, but I really haven't gained any love for C++ during this exercise.  In fact, my antipathy towards C++ has grown towards what you might call full blown hatred of Modern Boost::style::C++.  I think that I would like to make a T-Shirt that says....

Is that a ...

   boost::intrusive::rbtree_node_traits<boost::interprocess::offset_ptr<void,int,unsigned int,0>,1> >::find<
            int,boost::intrusive::detail::key_nodeptr_comp<boost::container::container_detail::rbtree<int,std::pair<int const ,DATAPOINT>,
            boost::container::container_detail::select1st<std::pair<int const ,DATAPOINT> >,std::less<int>,boost::interprocess::allocator<std::pair<int const ,DATAPOINT>,
            boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,int,unsigned int,0>,0>,
            boost::interprocess::iset_index> > >::key_node_compare<boost::container::container_detail::tree_value_compare<int,std::pair<int const ,DATAPOINT>,std::less<int>,
            boost::container::container_detail::select1st<std::pair<int const ,DATAPOINT> > > >,
            boost::intrusive::rbtree_impl<boost::intrusive::setopt<boost::intrusive::detail::base_hook_traits<boost::container::container_detail::rbtree_node<std::pair<int const ,DATAPOINT>,
            boost::interprocess::offset_ptr<void,int,unsigned int,0> >,boost::intrusive::rbtree_node_traits<boost::interprocess::offset_ptr<void,int,unsigned int,0>,1>,0,
            boost::intrusive::default_tag,3>,boost::container::container_detail::node_compare<boost::container::container_detail::tree_value_compare<int,std::pair<int const ,DATAPOINT>,
            std::less<int>,boost::container::container_detail::select1st<std::pair<int const ,DATAPOINT> > >,boost::container::container_detail::rbtree_node<std::pair<int const ,DATAPOINT>,
            boost::interprocess::offset_ptr<void,int,unsigned int,0> > >,unsigned int,1> > > >(const boost::interprocess::offset_ptr<boost::intrusive::compact_rbtree_node<boost::interprocess::offset_ptr<void,int,unsigned int,0> > const ,
            int,unsigned int,0> & header={...}, const int & key=3, boost::intrusive::detail::key_nodeptr_comp<boost::container::container_detail::rbtree<int,std::pair<int const ,DATAPOINT>,
            boost::container::container_detail::select1st<std::pair<int const ,DATAPOINT> >,std::less<int>,boost::interprocess::allocator<std::pair<int const ,DATAPOINT>,
            boost::interprocess::segment_manager<char,boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,int,unsigned int,0>,0>,
            boost::interprocess::iset_index> > >::key_node_compare<boost::container::container_detail::tree_value_compare<int,std::pair<int const ,DATAPOINT>,std::less<int>,
            boost::container::container_detail::select1st<std::pair<int const ,DATAPOINT> > > >,
            boost::intrusive::rbtree_impl<boost::intrusive::setopt<boost::intrusive::detail::base_hook_traits<boost::container::container_detail::rbtree_node<std::pair<int const ,DATAPOINT>,
            boost::interprocess::offset_ptr<void,int,unsigned int,0> >,boost::intrusive::rbtree_node_traits<boost::interprocess::offset_ptr<void,int,unsigned i comp={...}) 

... in your call stack or are you just happy to see me?

How would you feel debugging a line of call-stack items of the above level of complexity, where the above is simply 1 of about 20 similar levels of function calls. Why even print that crap, we may as well go back to debugging via memory addreses in the EXE file, and naming these things checkpoint Fred, checkpoint Charlie and checkpoint Dave.

Anyways, before you go absolutely crazy with Spring4D and try to create an equivalent mess in Delphi now that we have generics which are a lot less powerful than modern C++ templates, but still powerful enough to create a lot of trouble, ask yourself, do you really really want to debug a call stack that has a few dozen   TFactoryProxy<
expressions when you break it?