Island/Delphi: Using Delphi APIs from Elements

marc hoffman -

With the just-announced Elements Twelve, we have added a major and long-requested new feature to our native-code "Island" platform: support for the Embarcadero Delphi class/object model and – with that – the ability to consume just about any* Delphi APIs from your Elements projects, including the Visual Component Library (VCL), FireMonkey, dbGo, DBX, and so on.

You can also use third-party component libraries and, of course, your own Delphi-compiled code – directly and seamlessly from Oxygene, C#, Mercury, Java, Swift or even Go.

The possibilities here are endless, from merely (re)using few Delphi APIs you really like and are familiar with, to building entire new application on top of VCL or FireMonkey, but with the full power and flexibility of the  Elements compiler, for the code you write. Always wanted to create a VCL form in C#? Now you can!

Inter-operability between Elements and Delphi scales to your needs. You might just reuse some code in your new Island app – or you might just bring a full VCL app over into Fire or Water, and start using Oxygene's more advanced Object Pascal language all across your code.

But lets step a bit back and ask:

How Does This Work?

To start with, the Elements compiler and the Delphi compiler use very different object models.

When you create a class in Elements (we're focusing on using the Island platform for native Windows, Linux or macOS projects, for the purpose of this article), these objects are laid out using a system defined by our own compiler, and descending from a type called System.Object that's defined in the Island RTL base library. All Island objects descend (directly or indirectly) from this type.

When you compile an Island projects, these objects (and other stuff) get compiled into an executable (an .exe, on Windows) or a library (a .dll or .lib, on Windows). For libraries, the Elements compiler also emits a .fx file that contains meta-data about your code. Essentially it describes the types you created, so that when you use your library from a different project, the compiler knows what types there are and how to find them in the .dll binary. This is, of course, a highly oversimplified description.

Delphi uses its very own object format for classes you create in your Delphi project, and these classes descend from a type called TObject, and they work very differently from Island objects. Delphi also has its own ways of dealing with simple types such as Strings (Island strings are Objects, Delphi strings are not), and so on.

When you compile a library (we'll ignore executables for this post) in Delphi, your code gets compiled into a binary and a .dcu file for each unit or – when you are building a package, a .bpl binary (essentially .dll), and a .dcp file that combines all the units in your project into one.

How Shall The Two Meet?

As you can see, these are two similar but very incompatible paradigms, as Elements and Delphi both do their own thing. How, you may ask, can we enable the Elements' Island compiler (and, by extension, code written to compile with it) to access Delphi stuff?

While the implementation of this answer was very tricky, and took us the better part of a year to get right, conceptually the answer to this question is very simple, and was solved in two steps:

1) We enabled the compiler to understand and be able to create classes that, instead of using it's own "Island" object model" use the Delphi object model. In other words, the compiler knows how Delphi objects work, and it can create classes that, when looking at their binary representation and functionality in memory, look indistinguishable from those created by Delphi. Which is more complex than it sounds.

2) We created a tool (integrated Water, our IDE for Windows, and into our EBuild command line tool) that can take a set od .dcp files created by Delphi, read and understand them, and emit a set of .fx files that describe the classes (and other stuff) defined in the package in the standard way the Elements compiler wants in order to use a library.

It sounds almost too easy and, yes, there is a lot more to it behind the scenes, but with those two things out of the way, you can now use any* Delphi package from Elements.

Let's Get Started

As mentioned before, you can import core Delphi packages, as well as third-party libraries and your own. But the process starts by importing a core set of Delphi packages — what we call a "Delphi SDK".

In Water, with any project open, select the "Import Delphi SDKs" items from the "Tools" menu. Water will go and find all the (supported) versions of Delphi it finds on your system, and show you a dialog where can choose which version(s) to import. Just select the right one(s) and click "Import". This might take a while, but it only needs to be done once.

You can read more about this import and its prerequisites here, and this help topic also has steps for when you don't want to or can not use the IDE-integrated import in Water.

With this out of the way, you're ready to create your first project.

Create a new (or open an existing) Island project for Windows, Linux or macOS via the New Project dialog, and go to the Settings node in the project tree. You will see a new setting called Delphi SDK, which is empty. The drop-down will list all the Delphi versions that you imported earlier, and by selecting one, you are telling the Elements that you want to use Delphi APIs in your project, and which version to use. In this example, let's set it to Delphi 11.

Note that from this point on, everything described should work similar in Visual Studio, and in Fire, as well. If you're working on a Mac, you will want to manually copy the imported Delphi SDKs from your Windows machine, or manually run an imort from the command line. Both options are described at the bottom of the before-mentioned help topic.

References

Once the Delphi SDK setting is set, you will see that Elements automatically added two new "implicit" references to your project: "Delphi.rtl" and "Island.DelphiSupport".

The first reference is the .fx that was imported from Delphi's rtl.dcp package, and it includes all the core Delphi APIs you're used to. We will reference additional packages in a second. All imported Delphi packages re prefixed with "Delphi." to keep them apart form regular non[-Delphi references (which also have an "rtl").

The second is a small helper library that provides utilities and bridging support to make Island and Delphi objects (and non-objects) work together. For example, it provides helper types that make Delphi strings behave as expected (and not just like a dumb PChar), from your code. More on Strings later.

If you open the "Manage References" sheet for your project, you will see a whole bunch of extra Delphi.* references available. For example Delphi.vcl (if you're in a Windows project), Delphi.DBX, and whatnot else. Just reference all the packages you need.

Let's Write Some Code

Let's create a new class. For fun, I'm using C# in this example, mainly because it looks more different to Delphi than if we were using Oxygene 🙃. 

public class TFoo : TObject
{
        
}

And with that, you have created the first-ever Delphi class in C#. How can we be sure it's a Delphi class? Let's invoke CC to override a method, and you will see CC offers the methods you're familiar with from Delphi:

We could also call the modelOf() System Function:

writeLn(modelOf(TFoo));

and it would print out "Delphi".

Why did this class become a Delphi class? There are two ways this happens:

  1. As mentioned before, all Delphi classes descend from a single root class, TObject. As such, whenever you declare a new class with an ancestor that is part of that class hierarchy (in this case, TObject itself, but it might also have been, say TStrings or whatever else floats your boat), the compiler will automatically mark it as a Delphi object, and generate its implementation to match.
  2. Island also allows you to explicitly specify a class's object model using an Aspect attached to the class. If you add the [Delphi] aspect/attribute to a class declaration, it will be a Delphi class, if you add [Island] instead, it will be an Island class. (Do note that you cannot specify an object model that conflicts with the class's ancestor, of course).

Because TFoo is a Delphi class, this means we can use it any way we can use Delphi classes. For example, store it in a TStringList:

Delphi.System.Classes.TStringList list = new();
list.AddObject("My Foo", new TFoo());

Note how the TStringList class, which comes from Delphi's rtl package, is just available as if it were nothing. Just like package names, all types imported from Delphi get "Delphi." prefixed to their namespace.

Like with all types (Delphi or not), you can access them by their full name, or you can add a uses clause (using, in C#) to the top of your file to importthe whole namespace.

using Delphi.System.Classes;

...

TStringList list = new();
list.AddObject("My Foo", new TFoo());

What About Strings?

Glad you asked!

In the snippet above, we're passing a string literal to a Delphi API ( AddObject). This API of course expected a Delphi-style string – that is, a pointer to a reference-counted memory structure that looks a bit like a PChar, with some extra magic.

But on Island (and all other Elements platforms) strings are objects, instances of a class called System.String defined, once again in Island RTL. So what gives?

The Island compiler (aided by the Island.DelphiSupport library which by the way is open source and can be seen and contributed to here, has sophisticated support for seamlessly converting string types back and forth, where needed.

In the above case, the compiler sees a literal being passed straight to a Delphi string, so it just does the right thing immediately. But even if we had an explicit Island string instance, it would be converted back (and forth) seamlessly, if needed: 

System.String s = "Hello";
writeLn(typeOf(s)) // yep, its an Island String!
list.Add(s);

VCL, FireMonkey, and All the Rest

Of course, TStringList is not where the fun ends. You can build (from scratch, or import) entire VCL or FMX applications to Elements, and/or use other more advanced and unique Delphi-specific APIs in your apps.

For example, add a Delphi.vcl reference to your project, and copy one of your TForm classes (.pas and .dfm) as well as the Application start-up code, and build and run your first VCL app:

You can also use the "Import Delphi Project" item in the "File" menu, to bring a whole project over into Elements and using Oxygene, as well. Though don't expect that build without any tweaks, for all but the most trivial projects. Our goal for v1 of Island/Delphi is to enable the use of Delphi APIs from new/upgraded code, not 100% source compatibility and reusability of existing code in Oxygene.

As this article is already getting long, we will have another blog post in the coming weeks to look at importing and using third-party libraries, and your own packages. In the meantime, we hope you enjoyed this, and that the new Island/Delphi platform will bring you next-level abilities for re-using and/or migrating your Delphi code to Elements.

Let us know what you think!

P.S.: Elements Twelve build .2867 is out now. As always, it is a free update to everyone with an active subscription.

If you are not developing with Elements yet, grab a free 30-day trial version here, or purchase your license now, for only $999 including support for all six languages, all platforms, and all product features.

You can learn more about Elements at remobjects.com/elements.

* When we say "any", we don't, for now, promise one hundred percent compatibility. We're sure there are some weird APIs out there that expose things in a way that does not quite work (yet, or maybe ever), or that might need a workaround. But for 99% of cases, you should be able to "just use" the types from the packages you imported, "as they are", without giving it much thought.

Oh, and this feature should support Delphi 7 through 11.3, for Windows, Linux & macOS. Although we mostly tested with newer versions (10 and 11), and we recommend using those.

** Island/Delphi is built on the "Bring Your Own Delphi" model. A paid license for Delphi and a licensed copy of Delphi's library are required in order to import and use a Delphi SDK with Elements. Delphi's RTL, VCL, and other libraries are the intellectual property of Embarcadero.