Seven Days using the Surface RT: The Good, the Bad, the Ugly, and the Inexplicable

I'll cut to the chase right away. I'm typing this blog post on the Surface RT's Type cover, the one with real keys that move when you touch them and a small trackpad device built in, which is as nice a small keyboard as I've ever used.   The inexplicable thing, from the title of my post, is that with all its flaws, I simply love the Surface RT. 

Last weekend, when I heard about the sudden price drop on the 32 gigabyte Surface RT running the ARM variant of Windows 8, known as Windows RT, I decided I wanted one. So I went to FutureShop and bought it.  I also grabbed the Type Cover but instead of paying full price, I got one used from an online classified advertisement.

This is the limited (Windows RT) Surface model that you cannot install any regular Windows applications on, because it doesn't have an Intel-compatible (classic PC) CPU.  It does however run a version of Windows that you can not tell apart from the version that runs on my regular desktop PC. 

What is most surprising about that is that they have left the full Windows 8 desktop environment on there.  You can even (surprisingly) install various setup.exe or installer.msi files, but here are the two catches, and they're big ones:

- The only setup.exe or installer.msi files you could run on your surface RT are the ones built and signed by Microsoft.

- There is no known legal way to permanently rootkit your Windows RT device, at least not yet, which is the only way you can turn the code signing security features of Windows RT off or reduce them to sane levels.

So, while Microsoft clearly can and does and has written Desktop software for Windows RT, including the Visual Studio 2012 remote-debugger toolset, and other stuff, you cannot do the same. This seems to me to be the big injustice of Windows RT, and is the one thing I wish Microsoft would seriously reconsider. Sure, make the Windows store app side always require a signing, and require side-loading to use your Enterprise tools and enterprise signing system. Fine. But free the Windows RT desktop up to hobby-users. 

The flaws in Windows RT are so many, but one of them is that the desktop environment is not, and cannot easily be made touch friendly, and yet, there are so many things that you simply cannot do without using the desktop. In a previous blog post I complained about Windows 8's inability to pause and resume a printer queue without finding a pulldown menu that is more or less hidden, and almost impossible to tap with your finger on a touchscreen.    Hopefully Windows 8.1 will make the use of a keyboard and mouse, and the use of the classic desktop less frequently necessary.

The second most annoying flaw is that this portable device has all the beauty of Windows 8's desktop, and all its warts.   Do you love waiting for 36 updates to download and install on Patch Tuesdays?  You know I love that.   You can about double the installation times and that gives you the flavor for how much Windows RT updates are just like their Windows 8 sibling's updates.    There are more updates for RT than Win8, it seems, or the Surface RT simply takes a long time to install them.

But at the top I said I like this device, let me point out five reasons why I think this hardware form-factor and this software approach are interesting and offer people value. Some of these points are conditional on things that may never happen, because Microsoft has a history of snatching defeat from the jaws of victory, and long experience makes me less than hopeful. Still, there is hope, however far off, that this could morph into a brilliant product line.  Right now, it's flawed, but it shows promise.

1.  A device you can hold in one hand or fit in a small daily-carry satchel, for $349, that runs MS office, and has a usable keyboard.  ($349 for Surface, and $99 or $120 for the keyboards).    For people who need Word, Excel, and PowerPoint to go, this is a steal.  When Windows 8.1 comes out, Outlook will be included, which is where I think this will get really interesting.   The other cool thing in Office is "OneNote" which is perhaps not well known, but is a pretty fabulous application. Think of it as a hyper-textual notebook for all your personal or corporate internal scribbles.   What would a programmer keep in there? I might keep a list of SQL queries that helped me out of a jam at one time or another,  List of Books I Plan to Read in the Next Five Years,  A Place to Store My Library Card Numbers,  Utility Company Account Numbers, and stuff like that.   I currently keep all that stuff on Google Documents, but the thing is that I would rather keep them available for even when I'm offline, and OneNote is probably an even better home for them than Google Docs, as this information is information I might like to keep with me wherever I go, even when there is no internet.

2. A device with remote desktop that does a pretty good job of remoting into Win8 and Win7 and other Windows versions.   IT people, programmers, and people who have to use a dozen or two dozen cloud computers will find this device indispensable.

3.  A very nice screen for watching 720p video while on an airplane, bus, or train, or for entertaining your kids while you're on a drive. I plan to use this thing, hanging on the back of our car's front seat, to entertain my 2 year old on road trips, with episodes of Blue's Clues.  Exactly like many people are using the iPad and Android tablets.  I happen to really like the screen size (10.2), and think it fills a nice niche that makes it better for video in the car than the Google Nexus 7" we own.  I used an iPad long enough that I borrowed from my employer that I have a healthy respect for the large iPad screen, but now that the Surface RT price is cheaper by $150, I think that the number of people who would buy it as a video device has increased.  I know the Surface RT is not a high-DPI display, but the color quality and even brightness of the screen are really impressive.  I have used this device for watching Netflix while at home, and I really enjoy using it much more than I liked the iPad or my MacBook or my PC laptop. It's the best portable Netflix device I have yet used.

4. I intend to write an app for this thing, it's going to embed a scripting language, and let you mess around with code and mathematically oriented libraries.  You could say it will be something like WxMaxima, or Mathematica when it's finished.   I am also going to write a photo-app to help you with offloading pictures from your camera. Oh, I forgot to mention the best thing so far that makes the Surface RT better than any other tablet.

5.  The Surface RT gets USB the right way around. Instead of one USB device port, it has a USB host port.  Why oh why does everyone else get it wrong?   You can use the Surface RT to download pictures from your camera at full USB speeds, or you can plug a USB card reader into the Surface RT to read your SD or CF cards.  You can also plug in a USB hard disk, a usb mouse, or any of thousands of other USB peripherals, as long as the drivers are built-in to Windows RT already.  Any device that requires custom drivers to function will not be a good fit with Windows RT, sadly.  I can't think of too many devices that you would want to plug into a Surface RT that do require extra drivers, other than Printers.  We'll have to see how the Printing situation evolves on Windows RT.


The printer and device driver dilemma may in fact be the reason why Microsoft has to open up the desktop development side of Windows RT.   Without the ability to plug in some Drivers, and some Desktop Accessories that live outside the WinRT sandbox, the platform is going to remain artificially sterile and artificially useless.  Other things that I already wish I could load on WinRT that I can't include CCleaner, or something like it to monitor and clean up disk space on this "Tiny" 32 gigabyte flash drive, and a decent text editor like Notepad++.  No I do not want some stupid Windows Store App text editor, Microsoft, you idiots.  Today, for example, I tried to print to my HP LaserJet 3030 multifunction laserjet printer/scanner/fax device in my home office, that works with Windows 8, Windows 7, Mac, and everything else, but does not work with Windows RT.   That HP laserjet model didn't get into the list of pre-installed drivers in Windows RT, and while Windows RT has an "Insert Disk" button (ha, remember that, when computers had floppy disk drives?) it doesn't have any insertable drivers.  It also doesn't work with the Windows Update feature, it just gives you an error if you try to update the list of available printers from Windows Update.  With all that cruft, it shouldn't be long till someone jailbreaks RT and we can begin the work of fixing it so it actually functions.


Anyways,  I love the Surface RT,  but it is flawed and limiting in some ways that it didn't have to be.
Here's the final thing to consider when looking at RT versus Surface Pro:  If you're going to spend over a thousand dollars on a Surface Pro, ask yourself, do you really want to be burning a Core i5 CPU and a full Win32 OS services complement, on a mobile device? Because I sure don't.   Yes, maybe it would be cool to have Delphi on a Surface device but actually, I doubt it. Desktop Win32 and its pathetic ability to deal with high-DPI screens on classic Win32 API apps makes the Surface Pro something that I would find ugly and useless for Windows desktop apps.    Maybe the Surface RT is limiting, but it's well designed, and far far less irritating than a 150% DPI-virtualized bitmap-scaled blurry Delphi IDE would be. 




 

Inprise Reprise

I was reminded recently that Delphi was Borland Delphi first, then Inprise Delphi briefly, then back to Borland Delphi, and then more recently, CodeGear Delphi, and then after the Borland/Codegear split, it became Embarcadero Delphi.

I would love it if anybody who knows more of the real details would come along and fill me in, but from where the Delphi user sat, this is what the whole Inprise thing looked like to us:

Borland in the early days was Philipp Kahn's company.  It was a software company that sold general utilities for microcomputer users.   For DOS, there was  SideKick, an early "PIM" (personal information management) program for DOS, launched in 1983, the same year that TurboPascal was first launched.  You may remember that TurboPascal first was a CP/M tool.  Originally written for Z80 by Anders Hejlsberg, and running on a single board Zilog Z80 CPU based hobbyist computer called the NasCom, that few of us ever heard of,  what became TurboPascal later was originally known as BLS Pascal. When  this code was ported from Z80 to 8086, and the incredibly successful TurboPascal for DOS was born.  That same compiler codebase was used in the creation of Delphi 1.0 for Windows 3.0 (16 bit), and Delphi 2.0 for Windows 95 (32 bit).    I started using TurboPascal around version 5.0, and remember being very excited when TurboPascal 5.5 came out, which included this new Object Oriented stuff.   I believe I paid less than $100 for TurboPascal 5.5 from my campus computer store, and it came in a very large box, with a lot of very nice manuals.    In the DOS era, paper manuals were an essential programming tool.  Online help on an 80x25 or 80x43 EGA DOS text screen was not much of a "help" to programmers, and I relied heavily on the paper manuals while learning and using the system.

 Borland sold millions of copies of TurboPascal in the DOS era.  So from a humble hobby-computing market, they got big.  What happens when companies succeed financially and start getting bigger?  I guess they hire a lot of people, promote some of them to middle management, and generally bloat out from their humble startup days and into their days of fat.  At microsoft, famously at one point, there were more than 10 levels of middle and product and senior management between each developer and the CEO.   I have no inside information on life at Borland circa 1995 (Delphi 1), or when the Inprise debacle started, during the Del Yocam era, in 1998.  I have heard some tales but I will not repeat them.

Instead I wish to reflect on the simple fact that by renaming yourself and abandoning your constituency, you alienate all your current customers.  There were actions by Borland staffers and corporate leadership prior to 1998 that alienated some DOS and early-Windows-era customers, but the Inprise era is spectacular for the sheer ineptitude and bad timing of the move. As a person who often wonders why Delphi did not achieve more corporate traction than it did,  the later JBuilder-is-the-future fiasco is still second in the list of disasters to the "Abandonment of Borland brand" inspired under Del Yocam's time leading Borland.   On the internet newsgroups,  and at Delphi user groups, the move to the "Inprise" name was met with shock.   Why did the community even care? It's just a name, after all.

No, it wasn't. The "Inprise" name change was basically a message that "We're not that DOS era tools company anymore, and we think we're ready to move into big ticket Enterprise software".   The enterprise software enterprise failed.  Inprise failed. Borland failed.  Later on Borland went chasing after ALM (application lifecycle management) as a new Enterprise era, and abandoned codegear, and Embarcadero picked it up for a song.

So, what is this Enterprise level tools software that every software developer who wants to build enterprise software needs? This is very much still an open question.   Enterprise software development tools, if you will forgive me for answering my own question, are "Snake Oil", pure and simple. Large companies in general need all the same software development tools and functionality that medium and small companies have.  Yes they have the money to buy a high end edition, if that higher edition really helps, and those higher editions exist, and yes people buy them.  I am not saying that nobody buys it.   I'm saying that the tools don't deliver a measurable Enterprise productivity boost that they promise.  In short, I am saying that no high end $5000 development tool provides quite as much inside the box as it promises on the outside of the box and that the claims of "Enterprise" software products often outstrip what they can really deliver.    

 What they have that other companies do not have is (or so we hope) a large pile of cash that they are willing to part with, and additional fear and anxiety that salespeople can trade on, and receive money for for intangible deliverables like "Enterprise readiness" and "Enterprise level functionality", or best of all "Five-Nines" reliability.  

 It's quality sauce, performance and speed as an extra cheese topping, on your mid-market pizza, that makes your Enterprise Pizza.    The word "Enterprise" is still with us, in CodeGear/Embarcadero Delphi land, as the name of an edition of Delphi.

What do the high-SKU Enterprise and Architect editions of Delphi offer me that I would pay for?  Actually, very little. Now in XE4, the new database layer, AnyDAC FireDAC is actually great, and I would almost buy Enterprise or Architect to get it. Almost.

What else?  I don't currently like DataSnap much, but it is definitely the most important and most valuable thing that comes with Delphi Enterprise and Architect.    If Embarcadero could improve DataSnap to the point that people like me want to buy it, that would probably make it worth buying.

What else, else?  Well there's the blobby-gram builder tools, and the code metrics.  Guess what. I think blobby grams are useless, and the code metrics feature is inferior to the $99 Peganza Pascal Analyzer. You've got to do better to earn my dollars for those features.

I really do want to see the Enterprise and Architect editions succeed in the free market and sell lots of copies.  That will do as much to keep Delphi going and growing into the future, as having it become more popular would.   But Delphi is facing a clear and present danger;  New user adoption levels are so low, and the number of publicly findable Delphi jobs is at a deadly near-zero level.  Developers cannot find Delphi jobs, and Delphi employers cannot find Delphi developers.  As the software development manager and team implementation leader at a very small Delphi company, I am painfully aware of both sides of this equation.   I am a developer, and I try to lead a small software team through releases and bug fixing.

The way out of Delphi's current "No Jobs/No Developers" bind, when competing against a free Visual Studio express edition has got to be the opposite direction to the direction that the leaders during the Inprise period at Borland chose.

 I see a considerable pressure coming out of Delphi's product management to drive the dwindling number of Delphi faithful up the curve towards Enterprise and Architect.  I say, most of us are not budging.   Taking away features from Delphi Pro will only hurt Delphi and Embarcadero, terminate Software Assurance agreements and alienate the small shops that still use Delphi.

 What needs to happen is a return to the Borland/DOS era product market and reach.   Somehow Embarcadero needs to find a way to sell 30 million copies of $99 to $399 products, either in one lump sum, or even as an annual $99 rental.  The future of Delphi depends on it.

Was Delphi Starter the right move? I can't say as I don't know the sales and revenue from it, but I can say that it was not enough to make a dent in the NoJobs/NoDevelopers issue.  There needs to be something else done to address the perception that Delphi is a dead end. It's still the best technology out there for building applications.  Unfortunately, it's perceived as a a career-limiting and company-growth -limiting move. That perception needs to be addressed, and changed, with main force.  The cross-platform focus (iOS and then Android) is definitely going to add something to the Delphi toolbox that will attract new people, but something needs to be done to get more developers onto the platform.  A new generation of Delphi developers, these ones should be under 30 instead of graybeards over 40 like me.

An Unusual Open Source Project for Delphi, by Me.

You might not know that second to Delphi, my favorite programming language is Python.     Also, my favorite version control system, Mercurial, is written in Python, and I keep all my Delphi projects in Mercurial repositories.   I use Git when I have to, such as when contributing to Jedi JCL and JVCL, but when given the choice, I use Mercurial.

As an interpreted  dynamically typed language, Python is wonderful, and it fits in easily when you need a general purpose programming language, or a script, or a system level tool like Mercurial,  build-system tool, or some kind of quick parsing or text manipulation system, or when building a web-application.   When you need something that compiles to optimized non-interpreted code, you usually write that extension in C.  Python also makes it easy to write extensions.   No I do not recommend trying to write extensions in Pascal.

Having said all this about Python, you can even use my new little open source project if you are not a Python programmer.

The project hasn't got a fancy name yet, it's just a mercurial repository with a few python scripts in it yet, but I have plans for all of these tools, and these plans will eventually converge into a bit of a swiss-army knife tool. I am building this tool on my own time, outside my current job's work hours, but I plan to use all of these little scripts while I work.

The project is here:  https://bitbucket.org/wpostma/wppythonscripts

Now I will introduce the scripts, what they do, and what they might do later, when I have time to improve them. The reason for open sourcing them is that I hope that other people who need what I'm trying to provide here will also contribute their own features to these scripts.


fixdfm.py   : Fix invalid .DFM files that won't open in Delphi


Have you ever seen this error: OBJECT expected on line ....


If you are like me, and it happens to you at 5:30 PM on a Friday after you just did a giant merge, you probably are not very happy to see it. It means that Delphi's DFM parser doesn't like your DFM.  If the line number is 23, like here, perhaps you can open the .dfm and find the problem quickly. What if the error is at line 5223?  How fun is that?  And what if the DFM opens just fine, but all or most of the the controls are invisible, and appear to be gone.  How do you like it when that happens at 5:30 PM on a Friday? Well if you're me, it does happen, and you need a tool to sort it out.

Let's run it on the broken .dfm. It's a command line tool, so I run it like this:

C:\dev\myapp>fixdfm.py myform.dfm

Scanning but not saving changes. Provide -save parameter to make changes
1 : inherited Form1: TForm1
testInheritUnit1.dfm : 23  : Unexpected Property Located After Child Object
    Tag = 3

myform.dfm lines could be removed: 1
Changes made to myform.dfm not saved.
---
1 files scanned.

Now let's fix the DFM automatically:


C:\dev\myapp>fixdfm.py -save myform.dfm
Changes will be saved.
1 : inherited Form1: TForm1
myform.dfm : 23  : Unexpected Property Located After Child Object
    Tag = 3

myform.dfm lines removed: 1
---
1 files scanned.


So, for some common cases, like when Merging just inserts some random property that should have belonged to a DFM object declaration in a place where no property declarations are allowed, Delphi does not offer to remove it for you, it just fails to open the DFM at all.   What about if your merge tool removed an end keyword, or added an extra one?   This tool cannot (yet) fix that, but it can detect it, and it can give you a map of where the dangling "scope" in the DFM is, so you can hopefully repair the damage.

A future version of this tool, will use Mercurial's version control system to permit you to scan DFMs, find broken ones, and then revert back through mercurial's log, one revision at a time, until it finds a non-broken DFM.  Then it will repair the dfm, and rename the broken one to myform.dfm.orig  so you can use a diff/merge tool to find any extra objects that should be added to the form, and add them yourself. And if I can, I will even make that part automatic. (Take DFM one,  find objects that are in it that are not in DFM two, and shove them into the correct places in DFM two.)


dailyzip.py

This script is just to make a zip file containing the binary output of a build, or alternatively, a snapshot of source code of a project so you can have a snapshot that corresponds with your releases. It uses Mercurial to find the current repository hexadecimal ID, so you can always associate a daily zip back to the exact Repository revision that created it.  The binary zip of all executables, DLLs and other outputs of a full build could be useful to your Beta testers or QA people. For any software product, a downloadable binary build snapshot showing the latest source still builds, and still works, can be very useful.   Knowing what exact source code state corresponds to those binaries makes it even better.  Since those hex ID codes from Mercurial are guaranteed unique, you will always know what goes with what.

  Note that there is a flaw in my statement above. Knowing the tip revision is enough for most people, but some people might wonder "what bugs were fixed or features added between this daily build and the last".  Such an auto-readme function could easily be added to dailyzip.py. Of course first I have to build a readme-builder.py script, which will extract a bug list like this:

    BUG 1234 :  WP : Fixed the thing that was
    broken in the Thing. By the other thing. 
    Not that thing.  The other one, in Unit3.pas.


As you can see, I am a fan of clear and unambiguous commit comments, which correspond to one and only one bug fix or atomic buildable change per commit.  Always leave your repository buildable.  Never commit unrelated changes together.  And everybody will win.

getrepohex.py

This script is for grabbing the current repository revision (if the current folder is inside a Mercurial repository only)  and doing something with it, like putting it into a file "ver.pas" or "ver.inc" that you could compile into or include into your Delphi app.

md5sum.py

I know there are a million MD5 hash calculation utilities out there, but I prefer to have mine in script form so I can use it to fingerprint things, and maybe even do a little further magic. By having most of what I need in a script instead of a binary, I find that when I need to do something like build a snapshot of a whole bunch of MD5 fingerprints, this script can serve as a basis for whatever task I need to do.   Usually it involves knowing if the files I uploaded to FTP and down onto a client computer are binary equal (and not corrupted) from the time they left my original location, to wherever I'm getting them down to.

vertool.py

This little tool lets you extract version information from a  Delphi .dproj file, or modify the version information
in a delphi .dproj file.  If you ever find keeping a set of related executables and their projects all consistently at the same product revision level, you might find this useful.


Where is all this going?


A. Eventually I'll come up with a better name than wppythonscripts. The name I need would mean "These are mercurial/delphi code workflow and build automation helper utilities".

B.  Eventually I will have a bit of a build and test automation framework put together that can build programs, unit test them, report pass/fail, build the installer MSI, and if it passes, it would be either upload it to an FTP site, or copied to a folder, for further QA/Testing or release processes to occur.

C. It will all play nice with existing tools like Hudson, and CruiseControl, and help take the output of running MSBUILD and make both nice build logs, and error reports, as well as help with statistics calculations so I can see the trends of error and warning counts, commits, builds, etc, from either Hudson or CruiseControl.

D. Because it is written in Python and targets Mercurial users, it is going to support some commit-hooks, which will prevent checkin of broken DFMs, prevent checkins of worthless noise DFM changes -- the Delphi form designer modifies and regenerates with meaningless changes to DFMs even when you just open and modify the .pas file. A meaningless change on two branches now means a meaningless and time-wasting merge conflict later, or even possibly, a corrupted .DFM.  So the dfmfix tool will stop the insanity before it starts, it will soon be able to detect and revert such changes and auto-revert them before you can even commit something like this:

What is this not going to be?

A. Not a complete build tool. Use DANT or TRAIN or FinalBuilder.

B. Not a continuous integration tool. Use Jenkins/Hudson, CruiseControl, or FinalBuilder

C. Not something with a GUI, these are command line tools.

D.  While I would love to build an equivalent to NuGet (Visual Studio), Maven (java), or CPAN (perl),  I doubt I can accomplish that in this project.  

If someone wants  a GUI for the fixdfm.py, I'll accept a contribution of one, if it's coded in Python using the built in python tkinter GUI toolkit.  Or I'll build you one, if you send me some money or a flat of nice Belgian beer.

This is an open source project, and it aims to make your life, if you are a Delphi developer who users Mercurial, a little nicer.


Please report bugs, especially please send me .DFM files mangled by your version control system.  Any version control system that has to merge .DFMs is going to mangle them, if you have more than one developer opening and changing DFMs.  Not only does Delphi make random looking order and object persistence property set changes, it also tends to reorder, remove, and add a lot of noise, especially if you have a codebase that started out in Delphi 5 through 7 and has survived a move up through several versions after that, perhaps up to XE2 or XE4.   Please file bugs on the BitBucket site, and feature requests.  I particularly find that the TMS components with their backwards-compatibility property-upgrade features, can cause some strange surprising things when opening and saving a DFM to make a change in a completely different area.

I really must recommend that all Delphi developers on ALL version control systems review and revert all .DFM changes that they did not intentionally make, to keep their version control history clean. But since asking all Delphi developers to do that is placing a burden on them, I think that a little help mapping Delphi's crazy .DFM changing tendencies to a stable version control system would be good.