For Data Abstract 5.0, we’ve taken a long look at performance, trying to find ways to make ‘Vinci’ even faster and more well-performing that the current releases.
One thing we noticed was that the two data table classes we have provided for client use in the Delphi edition of v4 – based on Borland’s ClientDataSet and ADO, respectively – have been a major bottle-neck in data retrieval performance, because both technologies provided bad (read: no) support for efficiently filling the tables. Essentially, when fetching data, especially large amounts of data from the server, the bulk of the processing time was spent actually getting the retrieved records into the CDS or ADO data tables – this took longer, in some cases, then even the network transfer of the data itself.
To solve this problem, we have implemented a completely new in-memory dataset component (TDAMemDataSet), and a DA data table implementation that leverages it (TDAMemDataTable). This data table has been built from the ground up to meet our needs for integration into DA and designed so that it works very well with how Data Abstract leverages the data tables. It has also been optimized to work particularly well with the new Bin2DataStreamer (more about that in a future post), where it will allow client-side filling of the table to be next to instantaneous.
Simply switching to this new data table should show a big performance boost for existing client applications (especially, but not only, if you are fetching large amounts of data). Data Abstract 5.0.21 will ship with a small converter utility that will enable you to update your applications to use the new MemDataTable (which of course is source interface compatible with the CDS and ADO ones), with a single click.
