Here at RemObjects, we’ve always been great fans of so-called “dogfooding” – the act of using your own products early on in the beta cycle. As you know, we create developer products, and nothing helps us drive and test those more than actually using the products ourselves actively. For example, creating “Bugs”, the Mac (and later iOS) version of the client for our internal bug tracking system (which I plan to write more about soon, as well) is what largely drove the development of Data Abstract for Xcode from “early alpha” up to way past its initial release.
As I’ve talked about in the past few blog posts, we’re gearing up towards a major new release of our compiler, with the upcoming Prism XE2 release (Soon™) and Project “Cooper” (later this fall). Both of these product releases mark the availability of a brand new compiler, the “Oxygene 5” compiler.
Oxygene 5 is not just an incremental update to the old compiler code base (not that there is anything wrong with that), but for various reasons, we actually went and rewrote the entire thing from scratch. Literally, there’s not a single line of code in the core compiler that’s shared with version 4.0 and earlier. This has been a project that’s been over a year in the making, and over the past couple of weeks, we made some important steps in terms of getting ready to really dogfood the new compiler ourselves, all across the company.
#1: We’ve updated all our build machines to use the new compiler. Being a distributed development team, we rely heavily on our automated build system (another topic I will write more about soon), so putting the new compiler on there is a big step. But we did, and indeed it showed us a few more bugs here and there, and we fixed those, but all in all, everything worked fine. So all our Oxygene-based products were now being build with the new compiler. That includes most parts of Oxygene itself (including the core compiler, which we migrated from C# to Pascal in the rewrite), Hydra, Data Abstract for .NET, Relativity Server, and a range of other smaller tools.
#2: So our new products were being build with Oxygene 5 now (and reasonably passed our automated testing), but did they work, too? Only one way to find out, so the next step was to update our build systems with a version of Oxygene 5 that was created on the new build system. This is called a “generation 2” compiler (generation 0 being the old, trusted 4.0 compiler, and generation 1 the new compiler, built using the old, trusted 4.0. generation 2 is the new compiler, built with (a few hours or minutes older version of) the same, new compiler).
So at this stage, our build system was relying on the generation 1 compiler having been stable enough to build a generation 2 compiler that in turn was stable enough to build all our products (including, of course, itself). If that works (and it did), you know you have something fairly solid.
Since then, we never had to go back to gen0 or gen1 on our build system. So as we improve the compiler, fix more bugs and add more features, we regularly update our build system to newer versions of the compiler, but all of these are post-gen2.
#3: But why stop there? Our build infrastructure is a sophisticated distributed system that makes a lot of use of Data Abstract, including DA LINQ (and LINQ is one of the trickiest parts of the compiler, from what Carlo tells me ;)). So our next step was to update all the servers that make up our build system and replace them with versions that (a) have been built using the latest compiler and (b) use the latest build of Data Abstract for .NET, in turn generated on our build system, with that latest compiler.
So by now our build system is completely driven by tools created with gen2+ versions of the Oxygene 5 compiler. And everything is running fine across our various build servers (which include services running under Windows, Linux and Mac OS X) – so that’s pretty good coverage.
Of course these exercises did not all go completely without a hitch. For example, we ran into some nasty-to-debug issues where our servers would run fine on Windows, but fail badly on Mono. Turns out Mono did not allow for some corner case (but within spec) IL constructs that the new compiler generated. We fixed the compiler to generate “more conventional” code – and logged bugs with the Mono guys, as well.
#4: So what else can we try to break? Our main website [remobjects.com](http://remobjects.com] is written completely in ASP.NET with Oxygene and Data Abstract. We run two copies of that site, one that is live and one that acts as a staging platform to test new things (but most of the time is identical to the live site, short of some minor differences, such as showing debug information or non-live versions of texts from the database). The staging version is now running with a gen2+ Oxygene 5 compiler and using the latest Data Abstract, as well – and all is functioning. Switching the production site over would be a technicality, and something we’ll probably do any day now.
#5: And last but not least, all of our internal developers that develop in Oxygene (even those not involved with the Oxygene product itself, such as our Data Abstract for .NET team, or people like me working on internal tools) are now using daily Oxygene 5 builds for their day-to-day work.
So. It goes without saying that even the scenarios I described above don’t provide complete coverage, and of course we still heavily rely on thousands of automated tests, as well as feedback from beta testers on their real-life projects. But I think we’re in pretty good shape with the new compiler – and I’m looking forward to ship it to you guys soon.
Stay Tuned for More
Read more about Delphi Prism at remobjects.com/prism.
Read more about Project Cooper, currently in beta, at remobjects.com/cooper.
