Happy New Year!
We're starting of 2019 with the first new Elements build of the year, out today. Elements .2359 introduces a great new feature that will provide significant speed improvements to your build times and, thus your overall productivity: Improved Dependency Caching for References.
What does that mean?
When you repeatedly build a solution with Elements projects as part of your ongoing development, our build chain smartly evaluates what parts have actually changed since the last build, and only re-does the steps that need to be done. For example, a project will recompile if any of its inputs, such as source files or references, have changed.
This, of course, has always happened — but we've made the system much smarter.
You see, part of a project's inputs are its references, i.e. all the libraries or other projects it depends on. If a reference changes, the project needs to be rebuild. And in a multi-project solution, this escalates, and trickles through the entire solution – you might be working on a small fix to a shared library project, and suddenly your entire solution with half a dozen projects needs to be rebuild for each change.
No more!
As of today, Elements (or rather, its underlying EBuild build chain) knows a lot more about what changed in a reference, and makes smarter decisions on whether projects actually need to be re-compiled, based on that.
For example, if you make changes to the internal implementation of a class in one library, this change does not really affect any of the other projects that use this class. EBuild knows this, so when this happens, it will recompile only the library that changed, and keep the existing binary of any projects that depend on it, even though the date stamp of the library changed.
This use case alone, in our experience, makes up over 90% of all changes in libraries. Once a library's API has settled, most work goes on under the hood. And you benefit each time you hit "Build" or "Run".
Even when the public-facing API of a project changes, there are benefits to be had. Say Project A uses Library B, and that one uses Library C internally. When you make a public change to Library C, it and Library B will compile, but Project A might not need to, if it is unaffected by the change.
Of course the core compile is only one of many steps of the build, and EBuild knows exactly which steps it can skip, and which ones it can't. For example, if Project A from the scenario above is a Cocoa or Island application, and the libraries are static, then even though Project A was not re-compiled, it still gets linked again, to get the new copies of the changed libraries.
Real Life Use
I've been "living" with this new feature for about a week now, and it has already saved me a tremendous amount of not just time but also, more importantly, hassle, when working on Fire.
As you can imagine, Fire is a huge project; it's split over half a dozen libraries (and one no-code-but-the-main()-entry point app) and, on a good day, the whole thing takes about four to five minutes to compile. The dependencies between the libraries are very linear – there's a base library with shared low-level code, there's the code editor, there's the project system (the largest project), and finally the UI layer.
Because the build times (especially for Cocoa) are long for a solution of this scope, I used to be constantly managing the "Enabled" state of each project manually, to make sure only those project(s) relevant to the current task got rebuilt. Doing that manually is error prone, of course. If I forgot to enable the root app project, I ended up testing an excutable that wasn't relinked with my changes. If I left one too many projects active, there went an extra minute I had to wait before I could test my change...
But no more! With the new EBuild logic in place, I can now simply leave all projects enabled all the time! For 90% of my code changes, just the one library that's affected will re-compile, Fire.app gets re-linked (but not re-compiled; an optimization I could not force manually before), and I'm good to go!
Only with EBuild
As you know if you have been keeping track of our progress, we've been slowly rolling out our own build chain, EBuild over the course of 2018. It became the default chain for Cocoa, Java and Island in Q1; Fire and Water switched to using it exclusively in Q2, and in Q3 it also became the default for .NET projects, and in Visual Studio.
While the switch to EBuild went great, and there are countless benefits, internally and externally, a big part of the work has of course been replicating existing functionality, and implementing things that MSBuild quote-unquote "gave us for free". And many Elements users may have wondered what the point of the exercise was.
One reason is that EBuild's implementation of the tool chain just is a lot more approachable and maintainable. It's a single, clean codebase that can easily be debugged, improved and worked on to support all of Elememts' needs.
But another reason is that only with EBuild, a build chain that is fully under our control, it is possible that we can implement advanced features like this.
Improved Dependency Caching for References as a feature would not have been possible under MSBuild – and it's only the first of many good things you will see come out of EBuild.
I hope you enjoy it!
