Avatar of marc

by

New Expression Types in Delphi Prism

February 20, 2010 in .NET, Mono, Oxygene, Prism

It seems the sneak peek at “for expressions” in my previous post caused quite a bit of stir, so i figured i would spend a few minutes talking about the new expression types in general.

As you probably know, if you ever stopped to think about Pascal grammar, the most basic component that makes up a method body is a statement. A method body is pretty much defined as a list of statements, separated by semicolons. x := 5 is a statement. MyObject.DoSomething is a statement. Even a begin/end block is a statement (containing, nested within itself, another list of statements).

There’s a special kind of statements that seen frequently, and that are expressions. An expression is a statement that has a value, and as such, can be used in any code construct that expects a value – such as on the right side of an assignment, within arithmetic formulas, or as parameter to another method, for example. 5 is an expression. so is MyObject.ToString.

(In early Pascal, expressions did not use to be full statements. For example, functions (ie, procedures that return a value) could not be called without making use of that return value. In other words, sqrt(9) was not a valid statement, although x := sqrt(9) was. But that’s a thing of the past).

Functional Programming

Where am i going with all of this? Well, with the upcoming Spring 2011 release of Delphi Prism, we have extended the language. We took three commonly used statements, and we turned them into expressions.

This was done as part of an effort to bring more functional programing concepts to the language. Functional programing has been a niche in commercial software development for a long time, with most of the limelight going to (first) procedural and (later) Object Oriented programming, the latter of which is sort of a natural evolution of the first.

Functional programming uses a quite different paradigm, but it turns out that it is a very good fit for writing efficient multi-threaded code, and that’s why functional programming has been seeing a big revival over the past year or so, with developers looking at Erlang or Haskell. Microsoft has also stepped up its work on F#, a functional language for .NET, and is including that as a first-tier language in Visual Studio 2010.

For Prism, we decided to take a slightly different approach, and see if we could – over time – extend it with elements from functional programming languages, but, as has been the goal with Prism from day one, keeping them in line with traditional Pascal syntax and feel. (F#, for example, has many interesting concepts that make for a powerful language – but a god-awful and near-incomprehensible syntax to go along with them).

Our new expression types are a first step in that direction, so let’s have a look at them:

“if” Expressions

“if” statements are the bread and butter of just about every pascal method, allowing the conditional execution of one of two statements, depending on a boolean condition:

  if Condition then
    ThenStatement
  else
   ElseStatement;

The key word here statement. Both the then and the optional else case contain a statement (not an expression!) they execute, and as result, the entire if clause is a statement (but on an expression).

We have extended this syntax to allow the entire “if” clause to become an expression, if the statements it contains are expressions, as well. in other words, you can now write

  if Condition then
    ThenExpression
  else
   ElseExpression;

and the entire construct is an expression, representing the value of either ThenExpression or ElseExpression. To give a real life example, you could write:

  s := 'The Condition is ' + if Condition then 'true' else 'false';

The type of the “if” expression (in this case, a String) is determined by the lowest common denominator of the two expressions. If the optional else clause is missing, the result for a false condition will be nil, and if the type of the then expression is a value type, the “if” expression will be a nullable type. In the following example:

  var x := if Condition then 3;

x will be a nullable Int32, and contain a value of either 3 or nil.

“case” Expressions

The same extension applies symmetrically to “case” statements. A case statement where each individual case is an expression becomes an expression itself. As with the “if” expression, the type will be the determined by lowest common denominator of all the contained expressions, and if no else statement is provided, it will be nullable.

For example:

  s := case Number of
         1: 'One';
         2: 'Two';
         else 'Many';
       end;

I hope looking back, it makes sense why went into such detail with regard to statements vs expressions, at the start of this post. The point, really, is that expressions are just a sub-type of statements that have a value. And if you think about it, there’s really no good reason why the above statements/expressions should not have a value, and thus be expressions.

“for” Expressions

The last new type of expressions i want to talk about are “for” expressions. As with the other two, we have taken the regular “for” loop statement, and enabled it to be an expression:

  for each x in SomeSequence yield Expression;

and

  for x := StartValue to EndValue yield Expression;

As you can see, the syntax for this uses a slight variation on that of regular “for” statements, by replacing the do keyword with yield. This is mainly because yield makes more sense in the context of an expression. do implies execution, but we’re really yielding the value for each iteration of the loop.

If this sounds familiar from another feature in Delphi Prism called iterators, then that is no accident. A “for” expression has very much the same attributes and behavior of an iterator – in fact, you could think of them as “anonymous iterators”.

So what is the value of a “for” expression? The mentioning of iterators and the use of the yield keyword may have given it away: the value of a “for” expression is a sequence containing each of the individual expressions.

For example:

  var SomeNumbers := sequence of Int32;
  SomeNumbers := for i: Int32 := 0 to 100 yield i*i;
  // SomeNumbers contains the squares of 0 thru 100

And, just like an iterator, this sequence will be generated on the fly, as it gets enumerated. In the example above, no code runs to actually “calculate” the numbers 0 thru 10000 – the “for” loop itself is an O(1) operation. It’s not until we access the sequence – for example thru another for each loop, that the actual squares will be calculated.

To give an extreme example it’s entirely fine to declare the following

  var AllInts := for i: Int64 := 0 to Int64.MaxValue yield i;

to get a sequence of all (positive) Int64 values. We could then loop over that, using say

  for each x in AllInts index i do begin
    // only handle first thousand items. else we'd be at it forever.
    if i < 1000 then break;
    Console.WriteLine(x);
  end;

Summary

And that’s the new expression types, one of many new features in the upcoming Spring 20112010 release of Delphi Prism. If you’re part of the field test (and if you’re an active Delphi Prism customer, you really should be!), you can see these in action in the current beta drop.

Let me know what you think, and stay tuned for more!

16 responses to New Expression Types in Delphi Prism

  1. Spring 2011? *g*

    Apart from that: I like it, espescially the if-expression is much more readable than “iif”.

    • oops. goes to show how confusing per-year version numbers are. CodeGear uses 2010 for what shipped in 2009, while we use the actual dates. So of course this should be “Spring 2010″, in RemObjects terms.

  2. Only in Delphi Prism? It’s nice to see the language kept alive, but please spread the love to the Win32 compiler!

    • i hear you. Delphi Prism is the only compiler we have direct influence over. But i know the Delphi/Win32 (we’ll need to find a new moniker for that, soon?) team is looking into getting more language features of Prism over to the native side.

      The big problem is – i guess – that there’s just so many good ones to choose from ;).There’s five years of fast paced language innovation to catch up with!

      It will certainly help if you “vote” for your favorite Delphi Prism features by logging QC feature requests (or rating any exiting QC items), so the team knows what features to put priority on!

  3. This stuff is great, Marc. Kudos to all involved here.
    You guys are really giving C# a run for it’s money.

  4. Hi Marc! I/we would like to ask you a question. What optimizations does the Prism compiler perform when you use the above “for” expression? For example, it seems to me:

    Prism: SomeNumbers := for i: Int32 := 0 to 100 yield i * i;
    = ???
    C#: var someNumbers = Enumearble.Range(0, 100).Select(n => n * n);

    The only problem is that it is ugly because the C# compiler generates unnecessary IL overhead for this code.

    I publish your articles on Microsoft Hungary DevPortal and we are curious. Thanks.

    • i’ll have to defer to Carlo to comment about the exact code and optimizations that will happen here, but in general the code that gets generated should be similar (if not identical) to that of an real iterator defined as method. It will not use the Standard Query Operators, afaik.

      looking forward to your article & ndash; let me know when it’s posted, so i can plug it here.

    • This is the translated version of your article:
      http://devportal.hu/blogs/jankajanos/archive/2010/02/21/delphi-prism-218-j-kifejez-233-s-t-237-pusok.aspx

      “but in general the code that gets generated should be similar (if not identical) to that of an real iterator defined as method”

      Can you show me the generated code using Lutz Roeder’s reflector? I’m very curious what is the difference between the C# version and the “for” expression. :D It is for sure that the C# version is not an optimal solution.

    • Indeed it’s just like an iterator:
      class method __Global.Main;
      begin
      var x:= Enumerable.Select( .ForHelper(0, 10, 1, false), method (i: Int32)
      begin
      result := (i * i);
      exit
      end
      )
      end;

      The ForHelper looks like a plain for loop like:
      if aBackward then
      for i: Integer := aStart downto aEnd do yield i
      else
      for i: Integer := aStart to aEnd do yield i

      (iterator)

    • So it is worth avoiding it if you want to write efficient/performance-critical code. :D

  5. Really like it.
    However, with this, you are introducing duplicates for already existing syntaxes.
    Most importantly “iif” seems to be succeeded by the more elegant if-expression. You might want to express that by mentioning something in the wiki about “iif”.
    Maybe even an optional flag in the legacy options to deprecate the old one, if one really wants to.
    To minimize ambiguity or force the new syntax in team projects.

    Do these for-expressions allow for compound statements?

    var q := for item insomeList yield
    begin
    var outputValue : Integer;
    DomSomething(item.Value, out outputValue);
    exit new class(Item := item, Output := outputValue);
    end;

    • yes, we’ll definitely need to update the docs on iif().

      as for compound statements: no. the for statement needs to be an expression. compound statements are not expressions. For the above, you’d need to write a full iterator, ie

      method foo: sequence of whatever; iterator;
      begin
        for item in someList do begin
          var outputValue : Integer;
          DomSomething(item.Value, out outputValue);
          yield new class(Item := item, Output := outputValue);
        end;
      end;
      
    • Yeah, in this case, an ordinary LINQ query wold do it just fine, I guess. It’s not that we are going to lose those any time soon, are we? ;-)

      While I love the memory-friendly “streaming” of iterators, I was never a fan of iterators that should have been nested and locally scoped to where they are used and where they make sense.

  6. This feature is great but I have been using this all the time in VB6. I don’t know as to what is so great about this.

    Here is the code:
    ——————————
    Dim i As String

    i = “Hello From Yogesh ” ‘& IIf(i = “”, “It is empty”, “It is full”)
    i = IIf(i = “”, “It is empty”, “It is full”)
    MsgBox i
    ——————————

    I have to admit that you people are taking this feature a step ahead of what is currently available in VB6.

    • Delphi Prism has had

      iif()

      for years, as well. the new thing really is the integration with the regular “if” statement type and enabling that as an expression.