Bug fixed: SelectedColumnsRowSet

In previous releases of v3.2, the rowType was incorrectly reported as the original rowType, not the rowType with just the selected columns. This would have caused an error 22202 when the wrong rowType was used.
Similar fixes to other RowSet classes. Fixed as of 12 June.

On importing data

I've started to provide better support for data import into Pyrrho. Version 3.x has been introducing ideas of provenance. Now I am enhancing the PyrrhoMgr application a bit. The changes will get published in the next few days. So far they amount to:
1. Supporting direct import from Access 2007 in addition to Access 2003 and SQL Server.
2. Supporting the "Percent" numeric format in Access. (Intriguingly in Access 2007 the default Percent format is for a long integer, and so validation changes every value to either 0.0% or 100.0% !)
3. Allowing the importer to specify the "From Culture" so that culture specific formats for numbers and dates can get converted into the culture used by the importing thread (the culture of the machine that PyrrhoMgr is running on).
As mentioned in the last posting, the server always uses an invariant culture.
As a result of some ongoing research, more changes in this area can be expected soon.

Why the version hasn’t changed

Changes like this are possible in Pyrrho because of the distinctive way that Pyrrho is designed. Nearly all DBMSs store data structures like indexes in permanent storage within the files making up the database. Pyrrho by contrast places in permanent storage only the data required for durability of each committed transaction. This approach minimises disk activity during normal operations, to about one-seventieth of the disk activity of rival systems.
The server’s data structures are private to the server, and are initialised when the server starts up, and the server brings its state up to date by re-reading the database file, which is just the transaction record. This is a time-consuming process for a large database, equivalent to a cold-start with resynchronisation step for other products.
The format of this data is as far as possible version and platform-independent – there is not even any assumption for what “double precision” means, or how many bits make up a long integer. This means that every aspect of the organisation of data structures can evolve without breaking backward compatibility. Database files created by version 0.1 can still be used in every version of Pyrrho, so all Pyrrho data bases can always be managed by the latest version of the server. New features, such as the recently introduced concept of provenance, or URI-based data types, introduce new or modified data formats which would not be understood by earlier versions, but they really should not be in use. Upgrading to the latest version is always recommended, and is free of charge.

Speeding up joins

The next version (still numbered 3.2) will have a new implementation of the matching code for joins. There will be situations where a significant performance improvement can be expected, for example where the join condition refers to a component of a primary or foreign key. In the source code this is called a TrivialJoin, but it is not necessarily quite so trivial where the join condition does not constrain all the components of the key...
I should be able to release this implementation this weekend following testing.
Update 4 June: so FDJoin is a better term than TrivialJoin, with consequential renaming of lots of things. I've introduced a new internal datatype called Wild for dealing with partial matches. And better handling of recursive traversal of multilevel indexes. It's all loking rather nice.

Version 3.2 is done

I will release the next version 3.2 tomorrow 28th May.
It contains the promised improvements to the infrastructure.

Towards stronger types internally

The next version of Pyrrho will use much stronger generics. The internal Tree structures will be Tree<Key,Value> instead of just Tree<Key> . But this is taking an age to sort out...
With any luck version 3.2 should be out next month (June 2009) though.
The change has flushed out a few obscure bugs, but unfortunately the changes to the code are extensive so no doubt there will be the occasional null refernce that slips through. So if you get a bad message from the server, please let me know about it...

The REST service

Pyrrho's new REST service is similar to ADO.NET data services, from which it differs by providing a default data model. For a Pyrrho database file myfile, the default data service is myfile.svc. Custom data models can be supplied, and then the data service is given the same name as the data model, e.g. mymodel.pdm.

The service is hosted by default on port 8080 on the host running the Pyrrho DBMS server, e.g.

http://example.com:8080/mydatabase.svc/

and the rest of the URI follows the rules for ADO.NET. As with ADO.NET, the detailed datamodel is obtained by adding $metadata to the end of the service URI.

The methods GET, PUT, POST, and DELETE are supported. If no resource path is specified, a GET on the service gives a synopsis of the service, and otherwise returns XML for the specified data. POST is used for inserting new entities, PUT for updating specified data in existing entities, and DELETE for deleting specified entities.

However, to make any changes to the database, you need to supply UserID and Role identifiers in the HTTP headers. This means that most browser access will support only GET,