Archive for the ‘Mac’ Category

# Episode 39 of the Podcast at Delphi.org

It’s been a long while, but Jim and i finally found the time again to sit down and record a new episode of The Podcast at Delphi.org, talking about the upcoming Delphi Prism 2011 release, software development for the mac using Mono, and a lot more. It’s been great to be back on air, and we’ll hope to make this a more regular thing once again. In the mean time, i hope you’ll enjoy this episode!

See the official post for this episode at Delphi.org. Or subscribe in iTunes or via the RemObjects on Air feed.

(Via The Podcast at Delphi.org.)

# RemObjects Script for .NET

A while ago, I started planning a new Data Abstract feature: we wanted to support scripting in the .NET parts of the product, for validating business rules on both the client and the server, and for writing server logic, like we already have in Data Abstract for Delphi. There were several options we considered to us to use:

  • IronPython/Ruby
  • JScript.NET
  • Active Scripting JScript COM objects
  • CodeDom (C#, VB.NET)

But each came with it’s own problems. CodeDom, IronPython & IronRuby wouldn’t work for the Delphi and OS X side of things, and we wanted the new solution to be compatible across platforms. JScript.NET was deprecated a while ago and isn’t updated anymore, and the JScript COM objects aren’t very easy to manage from .NET, and wouldn’t work on Mono.

So what we did instead was write our own JavaScript (or as it’s properly called ECMAScript) engine. Microsoft had just introduced the new DLR, so we decided to base it on that, to have a good framework that would let us have an implementation with good performance and interoperability with other DLR and non-DLR based languages.

Initially, the development of this went rather slow, as the DLR at the time was very new, not even a 1.0 version was available, there were few docs and the samples online were hopelessly out of date. Adding to that was the problem that every update of the DLR code changed something. But now the DLR is becoming quite stable and parts of it will be in the soon-to-be released .NET 4.0 framework.

Introducing Script for .NET

Script for .NET is the end result of this work, and it’s a free-with-source implementation of ECMAScript (edition 3). It has all the features expected from JavaScript and the default (non-browser) objects like Object, Array, Function, Math, Number, Date, String, Boolean and RegExp. As mentioned above, the engine is based on the Dynamic Language Runtime, but it works on version 2.0 and later of .NET, as well as on Mono.

Debugging Support

The engine includes support for in-process debugging by the host, allowing to step through, step into and step out of the current function, and inspect all the locals and their current state. The following screenshot of one of the included sample applications shows this i action:

Usage

To make the engine easier to use, the class library includes an EcmaScriptComponent class which wraps the DLR and Scripting engine, lets the application run and debug arbitrary scripts, as can be seen below.

 
private void ShowMsg(string s)
{
  MessageBox.Show(s);
}
 
 
  ScriptEngine = new EcmaScriptComponent();
  ScriptEngine.Globals.SetVariable("ShowMessage", new Action<string>(MessageBox.Show));
  ScriptEngine.Source = @"
    var pi = Math.PI;
    ShowMessage('Todays value of PI is : '+pi);";
  ScriptEngine.Run();

It is also possible to and expose classes or complete .NET assemblies to the engine for use from within the script:

  ScriptEngine.ExposeAssembly(typeof(System.Object).Assembly); 
  // exposes all namespaces in mscorlib.

Where to get it

You can read more about RemObjects Script on its product homepage. The project itself is maintained on our code.remobjects.com open source repository, with full access to the SVN and bug tracker. The Spring 2010 release is available for download, now.

With the script engine itself out of the way, we are now working o the integration with Data Abstract, for our Summer releases.

# Everything is Relative

Today i want to talk a bit about “Relativity”, a new feature we’re shipping with Data Abstract this month (and which is actually available as “tech preview”in the current Winter 2009 releases).

“Relativity” is, simply put, a standalone server application that can be used to host Data Abstract schemas and serve data. In essence, it provides the complete Data Abstract middle tier, in a precompiled box and ready run. This offers several benefits, not the least of which are:

  1. No need to build your own. Relativity provides all the power and abstraction of a true Data Abstract middle tier server, without you actually having to create, build, and deploy your own executable. This allows you to concentrate on the client development. It also is really handy for prototyping, if you just want to have database server to talk to, without much work, even if you plan to eventually implementing your own custom server.
  2. Build on Data Abstract for .NET. .NET is platform with the most advanced version of DA – it supports our powerful DA SQL execution engine, and other features not available on all platforms. “Relativity” is being made available to all users of Data Abstract, so whether you’re a developer using .NET, Delphi, or Xcode, you can now write clients against Relativity that take full advantage of these features.

But let’s go and take a more detailed technical look at Relativity.

Relativity is a server application, so it does not provide any user interface. You can deploy a Relativity server in several ways, including:

  • Running it as a regular user application, on Windows.
  • Installing it as a Windows service.
  • Running it as a Unix daemon with the mono-service2 tool, on Mac and Linux.
  • Running it as a menubar item, on Mac.

Regardless of how you deploy and run it, Relativity will start making it’s Data Abstract services available on (by default) port 7099. But of course before it can start serving actual data, it will need to be configured with what data to serve. That is done thru the Relativity Server Admin tool, which is currently available for Windows (but versions for both Mac and iPhone are in the works).

When you first start it, the Admin tool will look something like this:

Screen shot 2010-02-03 at 2.24.03 PM.png

On the left hand side, you will see a tree view with all known instances of “Relativity” servers. This includes any servers (local or remote) that you connected to in the past, as well as any servers discovered on the local network using ROZeroConf.

Select any of the servers (here, “VEMMY” is the one instance running in my Windows VM), to connect and get more options.

Screen shot 2010-02-03 at 2.30.01 PM.png

The first two nodes allow you to configure how this “Relativity” server is accessed. This includes options such as what ports to listen on, what RemObjects SDK channels and messages to use, etc. Here you have the full choice of all the options provided by RO, you can choose the familiar HTTP, TCP, Super HTTP or other channels, BinMessage, SOAP Message, and so on.

Where it gets more interesting is the third node – Domains. This is where you configure the meat of the services Relativity is providing.

Each Domain corresponds to what typically would be a separate Data Abstract middle tier server. Domains are completely abstracted from one another and Relativity allows you to host as many separate domains as needed. With that, a single Relativity server can replace multiple DA servers, be it for related to completely independent applications or projects.

As you can see, i have three domains set up in my server, one for “BC7″, our new bug tracking system (using a PostgreSQL database running on our Xserve build server), one for “RemObjectsCom”, the database that drives our website (Microsoft SQL Server, running on the Windows server that hosts our website), and finally “OneSpace”, the database for a personal iPad project i am working on.

As we drill into a specific domain, you start seeing more things that will be familiar to you as a Data Abstract developer. Each domain can contain one or more Schemas – which correspond directly to .daSchema files or TDASchema components as you know them. You can manage schemas right there in the Admin tool, or directly launch Schema Modeler to edit the schema in question:

Screen shot 2010-02-03 at 2.39.38 PM.png

The domain also maintains a list of Connections to back-end databases available to it’s schemas. Once again, this directly corresponds to the .daConnections file or TDAConnectionManager component. Since Relativity is based on DA/.NET, these connections are based on the available DA/.NET drivers – regardless of whether your regular development platform is .NET, Delphi or Cocoa.

Another interesting part is the Login node. Relativity provides several options for handling authentication from client applications for data access (something you would normally need to manually implement, in a standalone DA server): in addition to relying on a static list of allowed username/password combinations, you can also choose to use a data table or command defined in one of the domain’s schemas to verify users (as i do in the example shown below) or query an external LDAP server for login.

The LDAP option is very handy if you are running a variety of services – some using Relativity, and some not – that you want to share user data among. We recently switched to LDAP for all our employee resources, and it has made administration so much easier.

The LDAP support in Relativity is based on a new 100% managed LDAP client implementation we’ve added to Internet Pack for this upcoming release. Carlo will blog more about this, soon.

Screen shot 2010-02-03 at 3.05.53 PM.png

The Login Provider architecture is extensible, so you can implement your own providers, for example if you need to do user authentication with a different method.

Client Access

Once you have Relativity set up to serve your data, your client application(s) can access it like any other Data Abstract server.

The only slight difference is that, since a Relativity server can host more than one domain, you need to tell the server which domain to access, by simply passing an additional Domain parameter as your login string, as in

User=foo;Password=bar;Domain=DomainName

You will also want to select which Schema to query. in regular DA applications, each schema usually has its own DataService, so keeping in line with that, Relativity allows you to append a schema name to the ServiceName property of your Remote Service. Instead of using DataService, simply use DataService.DomainName. (Alternatively, you can also pass a Schema to LoginEx).

In Closing…

So, this has been a brief introduction to Relativity – i hope you found it useful.

We think of Relativity as a strategic piece of Data Abstract, and it will play an increasing part in what we do for DA. We have lots of things planned for the future that will make Relativity even more powerful, such as our upcoming JavaScript Business Rules Scripting based on Carlo’s RemObjects Script for .NET.

You can try out the tech preview of Relativity now, and the official release will be part of what we ship at the end of this month. I’m looking forward to your thoughts and feedback!

marc

# Visualizing Data with Core-Plot

Core Plot is an open source graphing framework for Cocoa developers that makes it really easy to add graphs to your applications for the Mac and iPhone. Over the past half year, i’ve had the chance to use Core Plot in several of the internal applications i have been working on and have been very impressed with it, so i wanted to talk a little bit about how to use the library, in general, and how to make generate charts based on data in a Data Abstract for OS X application, in particular.

The example shown here is from a small iPhone application i wrote to keep track of the status of our Continuous Integration servers. The application shows the success (or errors) of any builds done on the machine, as well as – and this is the part where Core Plot comes in – graphs the number of tests that run and fail, over time:

In the application in question, the chart is being displayed in a cell within a UITableView, so our example starts out by implementing a custom UITableViewCell class that will host the graph. But the same general principles apply to show a chart anywhere else (and, replacing UIView with NSView, for using charts in a desktop app).

Core Plot uses a Core Animation as underlying technology, and as such draws itself in a specialized view class CPLayerHostingView. So we start out by crating this view, and adding it as sub-view:

- (id)initWithStyle:(UITableViewCellStyle)style 
{
  self = [super initWithStyle:style reuseIdentifier:@"MyChartCell"];
  if (self)
  {
    CGRect frame = self.contentView.bounds;
    CPLayerHostingView *chartView = [[CPLayerHostingView alloc] initWithFrame: frame];
    [self addSubview:chartView];
    //...

Once the CPLayerHostingView is created, we can add a graphs to it, and configure them. There are several graph types supported, but we want a regular line graph with X/Y coordinates, so we’ll choose a CPXYGraph class. We’ll add the graph to the view, and set a padding, to give us some room between the edges of the table cell:

  // create an CPXYGraph and host it inside the view
  CPTheme *theme = [CPTheme themeNamed:kCPPlainWhiteTheme];
  graph = (CPXYGraph *)[theme newGraph];	
  chartView.hostedLayer = graph;
 
  graph.paddingLeft = 10.0;
  graph.paddingTop = 10.0;
  graph.paddingRight = 10.0;
  graph.paddingBottom = 10.0;

The next step is to set up a plot space. A plot space defines the coordinate system for one or more charts in a graph, essentially mapping logical values to the area in the graph. For example, your chart might show values ranging from, say, 0 to 1000. The plot space defines the scale of those values in relation to the graph. It also defines the visual range of values that can be seen on screen.

To match our X/Y graph, we’ll create a CPXYPlotSpace, and set it to a range of 0-100 for both axis:

  CPXYPlotSpace *plotSpace = (CPXYPlotSpace *)graph.defaultPlotSpace;
  plotSpace.xRange = [CPPlotRange plotRangeWithLocation:CPDecimalFromFloat(0)
                                                 length:CPDecimalFromFloat(100)];
  plotSpace.yRange = [CPPlotRange plotRangeWithLocation:CPDecimalFromFloat(0)
                                                 length:CPDecimalFromFloat(100)];

Finally, we’ll set up to axis, for the X and Y coordinates. Axis provide labels and tick-marks for the viewer, to give context to the values being shown.

The X/Y graph already provides a set of axis in for of a – you guessed it – CPXYAxisSet class. There are several properties on the axis’ that are worth looking at at tweaking to to suite out needs:

  • majorIntervalLength defines the number of units between “big” ticks on the axis. In this case it’s set to show one very 10 units. (if it were not for the exclusionRanges, discussed below, a numeric label would show for each major tick, as well).
  • minorTicksPerInterval specified how many small ticks will be shown between each major one. in this case, a value of 2 indicated that small would show for 5, 15, 25, etc. (a value of 9 would show ticks for every single unit).
  • Finally, exclusionRanges allows to define areas where no axis labels will be drawn. In my app, i don’t want to see any labels, so i set a range to exclude the entire visible graph
  CPXYAxisSet *axisSet = (CPXYAxisSet *)graph.axisSet;
 
  CPXYAxis *x = axisSet.xAxis;
  x.majorIntervalLength = length:CPDecimalFromFloat(10);
  x.constantCoordinateValue = length:CPDecimalFromFloat(2);
  x.minorTicksPerInterval = 2;
  x.borderWidth = 0;
  x.labelExclusionRanges = [NSArray arrayWithObjects:
                              [CPPlotRange plotRangeWithLocation:CPDecimalFromFloat(-100) 
                                                          length:CPDecimalFromFloat(300)], 
							  nil];;
 
  CPXYAxis *y = axisSet.yAxis;
  y.majorIntervalLength = length:CPDecimalFromFloat(10);
  y.minorTicksPerInterval = 1;
  y.constantCoordinateValue = length:CPDecimalFromFloat(2);
  y.labelExclusionRanges = [NSArray arrayWithObjects:
                              [CPPlotRange plotRangeWithLocation:CPDecimalFromFloat(-100) 
                                                          length:CPDecimalFromFloat(300)], 
							  nil];

Our graph is now fully set up – except for one important part: we still need to define one or more charts to display actual values. My application needs to show two graphs on top of each other – one showing the total number of tests that ran, and another showing the number of failures.

Core Plot supports a variety of chart (or, plot) types, such as bar and pie chart, but for this scenario, we want a simple line graph, which is done via a CPScatterPlot. (A scatter plot actually allows to draw a plot with points scattered all over the place, not just as string y = f(x) values, but we will use it in the more simplistic case).

We’ll create a new CPScatterPlot instance, configure some properties such as its lineWidth and lineColor, and most importantly, set the dataSource to the object that will provide the plot with its data – in this case, self. To round things off, we also tell the plot to fill the area underneath it with a gradient, going from almost-transparent green to nothing. This gives the plot some depth, without hiding the other plots.

  CPScatterPlot *dataSourceLinePlot = [[[CPScatterPlot alloc] init] autorelease];
  dataSourceLinePlot.identifier = @"AllTests";
  dataSourceLinePlot.dataLineStyle.lineWidth = 3.f;
  dataSourceLinePlot.dataLineStyle.lineColor = [CPColor greenColor];
  dataSourceLinePlot.dataSource = self;
  [graph addPlot:dataSourceLinePlot];
 
  // Put an area gradient under the plot above
  CPColor *areaColor = [CPColor colorWithComponentRed:0.3 
                                                green:1.0
                                                 blue:0.3
                                                alpha:0.3];
  CPGradient *areaGradient = [CPGradient gradientWithBeginningColor:areaColor 
                                                        endingColor:[CPColor clearColor]];
  areaGradient.angle = -90.0f;
  CPFill *areaGradientFill = [CPFill fillWithGradient:areaGradient];
  dataSourceLinePlot.areaFill = areaGradientFill;
  dataSourceLinePlot.areaBaseValue = CPDecimalFromString(@"1.75");

We repeat the above to add a second plot, except this time we use red for the line color and gradient, and set the identifier to @"FailedTests".

This was quite a bit setup, but our graph is now ready plot, and will ask its data source (our cell) for data by sending it messages from the CPPlotDataSource protocol. This protoco will be familiar to any Cocoa developer who worked with, for example, a UITableView or NSTableView, or any other control using a data source.

To the protocol, two methods need to be implemented. The first is numberOfRecordsForPlot: which will, simply enough, return the number of items (in case of a scatter plot, points, the chart will contain. The second can be one of three methods that return the actual data. We’ll implement numberForPlot:field:recordIndex:.

The data to display is retrieved via Data Abstract, using a simple DA SQL request on a table that contains all test runs:

	NSString *sql = [NSString stringWithFormat:@"SELECT TOP 50 ID, _TotalTestCount, \
                                     _FailedTestCount, FROM TestRuns ORDER BY ID DESC"];
	testRuns = [[rda getDataTableWithSQL:sql] retain];

The _TotalTestCount and _FailedTestCount fields are server-calculated based on a relation table and contain the total number of run and failed tests for each test run. We can Key Path operators to get the maximum value, and adjust our plot space accordingly, so that the entire graph fits into the chart:

  int count = [[testRuns rows] count];
  int maxTests = [[[testRuns rows] valueForKeyPath:@"@max._TotalTestCount"] intValue];
 
  CPXYPlotSpace *plotSpace = (CPXYPlotSpace *)graph.defaultPlotSpace;
  plotSpace.xRange = [CPPlotRange plotRangeWithLocation:CPDecimalFromFloat(0)
                                                 length:CPDecimalFromFloat(count)];
  plotSpace.yRange = [CPPlotRange plotRangeWithLocation:CPDecimalFromFloat(0)
                                                 length:CPDecimalFromFloat(maxTests)];

This is of course a bit redundant with the values we set earlier, but in real life application, the data access will usually happen in a different place than the initial setup (and data might get changed or refreshed during the course of the application), so i find it important to set up the graph properly to begin with, but then adjust the range to the real data.

With the data retrieved and the plot space adjusted, we can now implement the two data source methods.

numberOfRecordsForPlot: will simply return the number of rows in our table. numberForPlot:field:recordIndex will be called twice per data point, once for the X value (since we’re drawing a “non-scattered” plot, we’ll simply return the indexes) and one for the Y value, which we’ll retrieve from our data table. The different identifiers we assigned to the two plots will serve to distinguish which field to use for the Y value.

-(NSUInteger)numberOfRecordsForPlot:(CPPlot *)plot 
{
  return [rows count];
}
 
-(NSNumber *)numberForPlot:(CPPlot *)plot 
                     field:(NSUInteger)fieldEnum 
               recordIndex:(NSUInteger)index 
{
  switch (fieldEnum)
  {
    case CPScatterPlotFieldX: 
    {
        // inverse numbers, so first (latest) test run is on the right.
        int x = [rows count]-recordIndex; 
        return [NSDecimalNumber numberWithInt:x];
    }
    case CPScatterPlotFieldY:
    {
      if ([plot.identifier isEqual:@"AllTests"])
      {
        float v = [[[rows objectAtIndex:index] valueForKey:@"_TotalTestCount"] floatValue];
        return [NSNumber numberWithFloat:v];
      }
      else
      {
        float v = [[[rows objectAtIndex:index] valueForKey:@"_FailedTestCount"] floatValue];
        return [NSNumber numberWithFloat:v];
      }
    }
  }
  return nil;
}

And that’s it! You can find more info on Core Plot on their Google code page; i also recommend joining their mailing list for latest news and support.

# Using NSPredicateEditor with DADataTable

Today I want to talk about NSPredicateEditor and how to use it with a DADataTable.

What is NSPredicateEditor…

Cocoa’s NSPredicateEditor class provides an excellent way to compose predicates at runtime. The main benefit is that the user needs to know nothing about predicates and their format, but with help of NSPredicateEditor can easily build quite complex conditions they need. Using NSPredicateEditor also allows to avoid any kind of typing errors, because in most of cases all the user needs is to do is just to select appropriate fields and values from popup buttons.

Here is the screenshot of Bugs 7 showing NSPredicateEditor that represents following condition: (Live == 1 AND AreaID == 4 AND OwnerID == 5) OR ID == 37222

NSPredicateEditor uses a list of NSPredicateEditorRowTemplate objects, where each row template describes certain elementary comparison predicate object.  Combining different row templates allows the NSPredicateEditor to represent compound predicates with unlimited complexity.

Usually, a row template holds three elements: left and right expressions and a condition between them. In our case, the left expression represents the column from our table and the right expression represents the value pattern for that column. Obviously, it is good to have template with most suitable editor for each column data type. DataAbstract for OS X provides proper templates for each fields that can be used in the table. In addition to standard  NSPredicateEditorRowTemplate DA brings several custom row templates that simplify building predicates for the tables.

The most interesting here is the DALookupFieldPredicateEditorRowTemplate class. This class allows to create row template for a lookup field, where the user can chose certain lookup value from list of allowed values, proposed by a popup button. It allows user to deal with meaningful values instead of just IDs. This is well shown in the figure above.

I should also mention DAFallbackPredicateEditorRowTemplate. This class describes special template for any predicate that is not supported by a given set of templates. For example if your editor tries to render a NSPredicate that comes into your application from outside and cannot understand a certain part of the compound predicate, then it will render it as “unsupported predicate” with help of DAFallbackPredicateEditorRowTemplate. This assures that the sub-predicate, while not editable in the NSPredicateEditor, will not get lost.

…and How Can You Use it?

Let’s see how to configure NSPredicateEditor instance for composing predicates for a certain custom DADataTable.

When you need to configure the editor for exposing a sub-set of columns only, you need to create array of row templates obtained from each table field you want to see in editor:

NSMutableArray *rowTemplates = [NSMutableArray arrayWithCapacity:[[da bugs] fieldCount]];
for (NSString *f in allowedFields)
{
  DAFieldDefinition *fd = [[da bugs] fieldByName:f];
  [rowTemplates addObjectsFromArray:[fd predicateEditorRowTemplates]];
}

Then you should supplement that array with additional row template for building compound predicates

NSPredicateEditorRowTemplate *compoundTemplate =
  [[[NSPredicateEditorRowTemplate alloc] initWithCompoundTypes:
    [NSArray arrayWithObjects:
      [NSNumber numberWithInt: NSAndPredicateType],
      [NSNumber numberWithInt: NSOrPredicateType],
      [NSNumber numberWithInt: NSNotPredicateType],
      nil]] autorelease];
[rowTemplates addObject:compoundTemplate];

and (optionally) add an instance of DAFallbackPredicateEditorRowTemplate for rendering unsupported predicates.

[rowTemplates addObject:[DAFallbackPredicateEditorRowTemplate rowTemplate]];

Finally, you should assign that templates array to your NSPredicateEditor instance

[editor setRowTemplates:rowTemplates];

If you want to expose all visible fields in the editor, then you can just use the convenient defaultPredicateEditorRowTemplates] method on the table, and whole configuration will be reduced to single line of code, as followings:

[editor setRowTemplates:[[da bugs] defaultPredicateEditorRowTemplates]];

That’s all. Your predicate editor is properly configured and you can use it for building any kind of predicates for given table.

You can use that editor to let the user define NSPredicates both for local filtering data in table like:

NSPredicate *p = [editor predicate];
rows = [[[myTable rows] filteredArrayUsingPredicate:p] retain];
[tableView reloadData];

or for creating Dynamic Where for server-side filtering data.

NSPredicate *p = [editor predicate];
DADynamicWhereClause *clause = [DADynamicWhereClause dynamicWhereClauseWithPredicate:p];
request = [rda beginGetDataTable:@"MyTable"
                          select:[self fieldsToSelect]
                           where:clause
                           start:NO];
[request setDelegate:self];
[request start];

I recomend to review our DAFilters sample that ships with DA/OSX, to see NSPredicateEditor and the techniques shown here in action.

# Incremental Data Fetching in Data Abstract for OS X

Bugs 7, the new bug-tracking application i have been working on over the past month for internal use here at RemObjects, employs several different data access paradigms (all based on Data Abstract for OS X), to accommodate the different nature of the data in individual tables.

The most interesting one is the main Issues table that contains all the bugs and tasks that are logged in the system. That is due to the fact that (a) it is a huge table (with over 25,000 records as of now, sized at about 11 MB when transferred over the wire, compressed) and that (b) by the very nature of a bug tracking system, this table changes frequently, and needs to be updated on the clients, often.

Rather then downloading the entire table anew every time, we optioned for a solution that allowed us to incrementally fetch only those records that have changed, and integrate them with the local dataset. This way, only minimal traffic is occurred for the regular refresh (which our client, by default, does every two minutes).

In the following post, i want to give you a quick glimpse at how this was accomplished, and how you can leverage the same technology in your own Data Abstract applications.

The Server

A couple things happen on the middle tier server (written using Data Abstract for .NET), to support the incremental refresh. Like all tables in the Bugs database, our Issues table has an UpdatedDate field, which gets automatically adjusted by the business logic code on the server. Every time a new issue is created, or an existing issue is updated, the server puts the current UTC time into the UpdatedDate field, clearly marking the order in which issues have been touched.

This is handled by a simple BeforeProcessChange event handler on the server, which simply adjusts the received delta, as such:

method BugsDataService.bp_Issues_BeforeProcessChange(aSender: BusinessProcessor; ea: DeltaChangeEventArgs);
begin
  ea.DeltaChange.NewValues['UpdatedDate'] := DateTime.Now;
  ea.DeltaChange.NewValues['UpdatedByID'] := Session['UserID'];
end;

(Of course the actual code in our server performs a lot more checks and changes, to enforce business logic for our database – but that’s beyond the scope of this post.)

Also, our Issues table does not permit deleting of records (only closing of issues, which sets their status accordingly, but does not remove the rows from the database). This alleviates the problem of worrying about rows disappearing from the table altogether.

The Client

On the client, a bit more custom logic is necessary, to perform the incremental updating.

When the client application (“Bugs 7”) is first started, it checks whether a briefcase file with a local copy of the data is found from a previous run, or not.

If not, the client will start a request to download the complete set of data from the server. This is a one-time process, and will download the entire table with it’s (currently) 11MB across the wire. Once downloaded, it is stored in a briefcase file, so on next application start, the data can be loaded locally. After the download is finished, the application also takes note of the latest UpdatedDate value it can find in the table, for future reference. This is made easy by Cocoa’s KVC and Key Paths:

- (void)downloadIssueData 
{
	// Initial download can take long, server side, to gather data. 
	// allow for a longer timeout.
	[(ROHTTPClientChannel *)[[rda dataService] channel] setTimeout:360];
 
	// fetch full table from server	
	issues = [[rda getDataTable:@"Issues"] retain];
	maxDate = [[[issues rows] valueForKeyPath:@"@max.UpdatedDate"] retain];
 
	// save table to briefcase
	[self saveIssues];
}

If instead a briefcase was found, the Issues table simply gets loaded from that file, alongside the stored maxDate:

- (void)loadIssueDataFromBriefcase
{
	DABriefcase *briefcase = [DABriefcase briefcaseWithFolder:briefcasePath];
	issues = [[briefcase tableNamed:@"Issues"] retain];
	maxDate = [[briefcase.properties valueForKey:@"BugsMaxUpdateDate"] retain];
}

Whichever path was taken, the application now holds a local copy of the Issues table it can work with. The next step it to schedule the regular refreshes, and for that an NSTimer is configured, to fire at regular intervals, on a background thread.

This NSTimer will trigger our beginRefreshBugs method, which uses asynchronous requests to start checking for new issues. It uses the previously stored maxDate and a feature of Data Abstract called DA SQL, to fetch only those issues that have newly changed:

- (void)beginRefreshBugs
{
	// prevent two refreshs happening at the same time, if the NSTimer
	// triggers again before the previous refresh has finished.
	if (refreshingBugs) return; 
	refreshingBugs = YES;
 
	// build the DA SQL query
	NSString *sql = [NSString stringWithFormat:"SELECT * FROM Issues WHERE UpdatedDate >= '%d'",
						(int)[maxDate timeIntervalSince1970]];
 
	//and start an Async Request to the server
	DAAsyncRequest *ar = [rda beginGetDataTable:@"Issues" withSQL:sql start:NO];
	[ar setDelegate:self]; 
	[ar setContext:@"RefreshIssues"];
	[ar start];
}

The DAAsyncRequest, once started, will communicate with the server in a background thread, without blocking the caller. beginRefreshBugs will return right away, and not wait for the request to complete (or fail).

Once the request did complete, it will call back to a delegate method (in this case we assigned self as the delegate, above), called asyncRequest:didReceiveTable:. It is here that we handle integrating the received data back with our big issues table by sending it the mergeTable:withPrimaryKey: message. This will replace the data in any rows that have changed, as well as add any new rows to the table:

- (void)asyncRequest:(DAAsyncRequest *)request didReceiveTable:(DADataTable *)table
{
	// our class may handle access to more tables. check the context
	// to make sure we handle the right requests.
	if ([[request context] compare:@"RefreshBugs"] == NSOrderedSame)
	{
		// nothing received? nothing to do!
		if ([table rowCount] > 1) 
		{
			@try
			{
				[issues mergeTable:table withPrimaryKey:@"ID"];
				maxdate = [[[table rows] valueForKeyPath:@"@max.UpdatedDate"] retain];
				[self saveBugs];
			}
			@finally 
			{
				refreshingBugs = NO;
			}
		}
	}
}

Of course a refresh might also fail (for example due to a broken internet connection, for this case, we implement a second delegate method, called asyncRequest: didFailWithException:, as follows:

- (void)asyncRequest:(ROAsyncRequest *)request didFailWithException:(NSException *)exception
{
	if ([[request context] compare:@"RefreshBugs"] == NSOrderedSame)
	{
		refreshingBugs = NO;
	}

The View

The last step after receiving new issues is to update any affected views. This happens more or less automatically, as every view that shows one one more issues (whether it’s the regular grid view of issues, a chart visualizing issue data, or an individual issue’s detail view) will be have registered itself to observe DA_NOTIFICATION_TABLE_CHANGED notifications on issues. And like any other change to a data table, mergeTable:withPrimaryKey: will send such a notification if changes happened, allowing all views to update themselves.

In Bugs, all of this happens in the background, so over time the view(s) presented to the user just seamlessly adjust themselves, as changes happen – new issues come into views; issues resolved by other users disappear on their own, etc.

This topic just touches on a very small aspect on Bugs 7, which itself is part of a mch larger project, comprised of four different client applications (Mac and iPhone, based on DA/OS X, for Windows, based on DA/.NET and Gtk# and the Web) as well as a middle tier server. We will blog more about different aspects of this project over the next few months, ands we’re also working on a bigger case study, to appear at bugsapp.com, soon. Stay tuned to this space, for more.

# Obfuscar

A few releases ago we switched to Obfuscar for obfuscation of our libraries, after some critical issues with the commercial obfuscator that we used. I’ve been providing the original authors with new issues and patches for things I fixed in the process of using this project (ranging from bugs to features), however it looks like the original project is dead.

So now at code.remobjects.com/p/obfuscar you can find the latest version with my changes and binaries of that version. Among the new features are:

* Unreadable unicode characters for obfuscated identifiers
* Signing assemblies

Enjoy,

# Is that a Wiki in your Xcode?

We’re getting close to shipping our first release of Data Abstract for OS X as well as the 1.1 update for RemObjects SDK for OS X at the end of this month, and as part of crossing all the final Is and dotting all the remaining Ts, Carlo and i have been working on leveraging the tools in our new documentation tool chain to integrate the RemObjects Wiki (or, the relevant part of it) as Xcode documentation:

What you are seeing here is our wiki as standard Xcode doc set. You can browse and search it just as you would the standard Apple docs; you can navigate to it by Cmd+Option-Double-clicking identifiers in your code – in other words, you get the full documentation experience.

The version shown here works directly on the live wiki – meaning as soon as changes and updates are made to the pages online, you will see them reflected in the wiki as well. In the long term (not for this release, yet), we’ll also be providing an offline version, that contains a snapshot of the entire wiki, for offline use (i.e, when you’re working on the proverbial plane).

The Docset will also update via RSS feed, so content (in the offline version) and structure changes (such as the addition of new pages to the indexes, etc) will automatically update within Xcode, on a regular basis.

Stay tuned for more info on RO/OSX and DA/OSX, as we near release.

# An Update on Data Abstract for OS X

I realize it’s been a while since i talked about Data Abstract for OS X, so i figured today would be a good time to post a small teaser for what’s coming up in the next beta drop – most likely before the end of the week.

The previous beta, which we shipped in early June, provided read-only access to Data Abstract servers created using .NET and Delphi. Since then we’ve been busy both refining that support but also, more importantly, adding the ability to make changes, persist deltas and and apply updated back to the server.

Below, you’ll see a screenshot of a small sample application that downloads a table from our unified PCTrade sample database, allows to make changes, and post them back. in the background, you see the sample server (running in a Virtual Machine on Windows) accepting and handling those update requests.

The app uses the briefcase model, so it can work offline, with changes staying local until the user presses the “Update” button to send them back to the server; of course DA will also all you to sending changes to the server implicitly, to provide a more seamless “online” experience for the user.

As you might notice, the sample also leverages the new ROZeroConf support we shipped in May, to find the server on the local network without manual configuration (in this case, the server machine is called VEMMY, as can be seen in the popup button in the client’s toolbar – if more than one server were running on the network, the user could pick which one to use, defaulting to the one found first).

# Delphi Prism, M.D.

Over the past couple months, a lot of progress has been made with getting Delphi Prism integrated into MonoDevelop. As mentioned before (and in our ROadmap), one of the “big” focuses in the current product cycle leading up to the August release next month has been a major refactoring of our internal IDE code, both in preparation for Visual Studio 2010 (Beta 1 of which which we’ll be supporting in August) and MonoDevelop.

So without further ado, below you will find the first ever screenshot of Delphi Prism being fully integrated into Mono Develop on the Mac:

Notice how we have a working Class Browser, Syntax Highlighting and Code Completion, as well as integrated building using MSBuild/XBuild.

Of course MonoDevelop is taking shape on Windows as well and of course (always has) runs on Linux, and Delphi Prism will integrate with it on all three platforms. Windows, i suppose, is the least exciting of these three options (after all, we already have Visual Studio, there), but having a native IDE running on either Mac or Linux should be a great boon for everyone using Delphi Prism to develop for these platforms.

So what’s the time line here, you’re asking? Good question! I cannot quite confirm a specific ship date, for two reasons. For one, while basic integration is working, there’s still a lot of Ts to dot and Is to cross. Lockdown for the August releases is imminent, so MonoDevelop definitely will not make it into that (also due to reason #2) – but then, hey, one new IDE per release should be enough, right?

Secondly, there’s still the remaining issue of MonoDevelop being licensed under GPL. The MD team, our team and a few contributors from the Delphi Prism community have done a great job moving this forward, but we still have some ~20 or so modules left that need to be rewritten before MonoDevelop is non-GPL and we can ship integration for it. Your help would be appreciated with this, so if you have some spare time, please check out our remobjects.public.oxgene.md-project newsgroup to see how you can help (or drop me a direct mail).

As far as our coding efforts go, i would think we should be ready to ship a preview of MonoDevelop support sometime between the August and November releases. i would also hope that come November, MD will be GPL-clean, and we can officially ship this in the November release of Delphi Prism*.

Also, what about Sharp Develop? To be honest, we’re undecided. While SharpDevelop (a Windows-only IDE for .NET and Mono) is closely related to MonoDevelop, it is different enough that supporting it would mean extra work. With MonoDevelop starting to look good on Windows now, we’re not sure if there’s much sense going thru the effort to integrate with SharpDevelop as well. What’s your thought on that? Can you see any benefits to SD support (that MD does not cover)?

*So if you’re buying RAD Studio 2010, not Delphi Prism standalone, make sure to get Software Assurance to be eligible for new features in November!