Tuesday, 3 December 2019

Progress with Pyrrho v7 alpha, contd

Today's version (28 March 2020) on https://GitHub.com/MalcolmCrowe/ShareableDataStructures/tree/master/PyrrhoV7alpha
shows check constraints, cascades and triggers working.
V7 documentation is provided with the source code.

There is a new section in the SourceIntro.pdf document about multi-threading, uids, and dynamic memory layout. This has been substantially improved since the December 2019 version to prepare
for the next stage of implementation, which includes Persistent Stored Modules.

The main technical changes are to make more of Pyrrho's data structures immutable and shareable: now including RowSets and Cursors which are now TypedValues.

I hope to have a new full v7 version for the pyrrhodb.com website in the near future. The following is taken from the SourceIntro document in the alpha distribution, under the heading Stored Persistent Modules. 

This heading includes Trigger and sored Procedure definitions, which are parsed once only by the server[1]. Both use the SQL stored persistent modules language as described in the SQL standard, including the handling of conditions (exceptions). When such modules are invoked, they run in the definer’s role as specified by the SQL standard.

Following the design outlined in this document, although the transaction log contains only the source form of trigger and stored procedure code, while the in-memory database contains a compiled version. From version 7 parsing is done once only, and following parsing everything is referred to by uid, not by using string identifiers. As their name implies, uids are unique in the database, but they are private to the implementation, and are subject to change is later versions of the DBMS. The transaction log contains only the source code of these modules, and the format is forward and backward compatible.

There are differences in operation of the different versions, however. Up to version 6.3 of the DBMS (file format 5.1) the source code contained database object positions instead of the definer’s name for database objects. This approach is supported in version 7 of the DBMS for database files created with previous versions. Databases created with version 7 or later (file format >5.1) will contain the source code exactly as given by the definer. This is generally supported by previous versions of the DBMS, but objects will display differently in the Log$ system tables.

The in-memory data structures resulting from parsing include SqlValues, Queries, and Executables. Persistent modules exist in one of three forms, as follows:

  • Source code that has been deserialised from the database file on Load() is compiled into data structures that use uids based on lexical positions of source identifiers in the database file. If the server is restarted after committing the definition, the corresponding compiled code will be of this form.
  • For source code that has been defined by client interaction, and committed to the database, the compiled version will use uids allocated sequentially from within the range of file positions where the commit has occurred, until the server is restarted.
  • For source code that has not yet been committed, the compiled version will use temporary uids above 5×260 allocated according to the lexical position in the client input. This version is recursively converted to the above version on Commit.

Unless you are debugging the server, you should notice no difference between these versions. The uids that differ are not generally visible, and the in-memory compiled code structures are otherwise identical.

CheckConstraints are simpler as they consist merely of an SqlValueExpr that is tested to validate a given value.

[1] Up to version 6.3 of the DBMS source code was parsed on each use.