Chapter 3: Changes in the Firebird Engine

Although a key feature of Firebird 3, the Providers architecture is not new. Providers existed historically in Firebird's predecessors and, though well hidden, are present in all previous versions of Firebird. They were introduced originally to deal with a task that has been performed since then by “interface layers” such as ODBC, ADO, BDE and the like, to enable access to different database engines using a single external interface.

Subsequently, this Providers architecture (known then as Open Systems Relational Interface, OSRI) also showed itself as very efficient for supporting a mix of old and new database formats—different major on-disk structure versions—on a single server having mixed connections to local and remote databases.

The providers implemented in Firebird 3 make it possible to support all these modes (remote connections, databases with differing ODS, foreign engines) as well as chaining providers. Chaining is a term for a situation where a provider is using a callback to the standard API when performing an operation on a database.

The main element of the Providers architecture is the y-valve. On the initial attach or create database call y-valve scans the list of known providers and calls them one by one until one of them completes the requested operation successfully. For a connection that is already established, the appropriate provider is called at once with almost zero overhead.

Let's take a look at some samples of y-valve operation when it selects the appropriate provider at the attach stage. These use the default configuration, which contains three providers:

The typical client configuration works this way: when one attaches to a database called RemoteHost:dbname (TCP/IP syntax) or \\RemoteHost\dbname (NetBios) the Remote provider detects explicit network protocol syntax and, finding it first in the Provider list, redirects the call to RemoteHost.

When <database name> does not contain a network protocol but just the database name, the Remote provider rejects it and the Engine12 provider comes to the fore and tries to open the named database file. If it succeeds, we get an embedded connection to the database.

Use of providers is not limited to the three standard ones. Firebird 3 does not support pre-ODS 12 databases. Removing support for old formats from the engine helps to simplify its code and gain a little speed. Taking into account that this speed gain sometimes takes place in performance-critical places, like searching a key in an index block, avoiding old code and related branches really does make Firebird fly faster.

Nevertheless, the Providers architecture does make it possible to access old databases when changing to a higher version of Firebird. A suitable provider may be considered for inclusion in a later sub-release.

Q. Interfaces and providers are probably very good, but I have an old task written using plain API functions and for a lot of reasons I can't rewrite it in the near future. Does it mean I will have problems migrating to Firebird 3?

And what about performance when using the old API?

Firebird 3.0 introduces an additional, generalized, URL-style syntax for connection strings. The pattern is:

  [ <protocol> : // [ <host> [ : <port> ] ] ] / <database file path or alias>

  <protocol> ::= INET | WNET | XNET
            

INET resolves to TCP/IP, WNET to Named Pipes, while XNET surfaces the old “Windows local protocol” (shared memory).

Examples

Local connection is implied if <host> is omitted. Depending on settings, platform and Firebird version, it could be performed via either the embedded engine, XNET (shared memory) protocol or TCP/IP localhost loopback.

On the server side, the provider configuration is in the default order Remote, Engine12, Loopback. If the Remote provider fails to match the connection string because the protocol or host parts are missing, then Engine12, the embedded engine, handles it as a hostless connection. To connect locally using a specific transport protocol, it is necessary to specify that protocol:

  inet://<database file path or alias>
  or
  wnet://<database file path or alias>
  or
  xnet://<database file path or alias>
            

The term “plug-in” is used to name related but different things:

Firebird's plug-in architecture makes it possible to create plug-ins of predefined types. Each version of Firebird will have a fixed set of supported plug-in types. To add a further type, the first requirement is to modify the Firebird code. Our plug-in architecture facilitates both adding new types of plug-ins and simplifying the coding of the plug-in along generic lines.

To be able to implement a plug-in, say, for encrypting a database on the disk, the Firebird code has to be prepared for it: it must have a point from which the plug-in is called.

The set of plug-in types implemented in Firebird 3 comprises:

Plug-ins use a set of special Firebird interfaces. All plug-in-specific interfaces are reference counted, thus putting their lifetime under specific control. Interfaces are declared in the include file plug-in.h. A simple example for writing a plug-in module can be found in DbCrypt_example.

Plugin = PlugName ## this is the name to be referenced in firebird.conf
  {
    Module = LibName ## name of dynamic library
    RegisterName = RegName ## name given to plug-in by its developer
    Config = ConfName ## name of config record to be used
    ConfigFile = ConfFile ## name of a file that contains plug-in's configuration
  }
              

Config = ConfName
  {
    Key1 = Value1
    Key2 = Value2
    ...
  }
              

Plugin = WC1
  {
    RegisterName = BestCrypt
  }
Plugin = WC2
  {
    RegisterName = BestCrypt
  }
              

WireCryptPlugin = WC1, WC2
              

Q. There are plug-ins named Remote, Loopback, Arc4 in the default configuration, but no libraries with such names. How do they work?

Q. What do the names of Srp and Arc4 plug-ins mean?

Q. What do Win_Sspi and Legacy_Auth mean?

Q. How can I find out what the standard Authentication and User Manager plug-ins are?

An external name for the UDR engine is defined as

  '<module name>!<routine name>!<misc info>'
            

The <module name> is used to locate the library, <routine name> is used to locate the routine registered by the given module, and <misc info> is an optional user-defined string that can be passed to the routine to be read by the user.

Modules available to the UDR engine should be in a directory listed by way of the path attribute of the corresponding plugin_config tag. By default, a UDR module should be on <fbroot>/plugins/udr, in accordance with its path attribute in <fbroot>/plugins/udr_engine.conf.

The user library should include FirebirdUdr.h (or FirebirdUdrCpp.h) and link with the udr_engine library. Routines are easily defined and registered, using some macros, but nothing prevents you from doing things manually.

The state of a UDR routine (i.e., its member variables) is shared among multiple invocations of the same routine until it is unloaded from the metadata cache. However, it should be noted that the instances are isolated “per session”.

By default, UDR routines use the character set that was specified by the client.

Enabling an external routine in the database involves a DDL command to “create” it. Of course, it was already created externally and (we hope) well tested.

Syntax Pattern

  { CREATE [ OR ALTER ] | RECREATE | ALTER } PROCEDURE <name>
    [ ( <parameter list> ) ]
    [ RETURNS ( <parameter list> ) ]
    EXTERNAL NAME '<external name>' ENGINE <engine>

  { CREATE [ OR ALTER ] | RECREATE | ALTER } FUNCTION <name>
    [ <parameter list> ]
    RETURNS <data type>
    EXTERNAL NAME '<external name>' ENGINE <engine>

  { CREATE [ OR ALTER ] | RECREATE | ALTER } TRIGGER <name>
    ...
    EXTERNAL NAME '<external name>' ENGINE <engine>
              

Examples

create procedure gen_rows (
  start_n integer not null,
  end_n integer not null
) returns (
  n integer not null
) external name 'udrcpp_example!gen_rows'
  engine udr;

create function wait_event (
  event_name varchar(31) character set ascii
) returns integer
  external name 'udrcpp_example!wait_event'
  engine udr;

create trigger persons_replicate
  after insert on persons
  external name 'udrcpp_example!replicate!ds1'
  engine udr;
              

The external names are opaque strings to Firebird. They are recognized by specific external engines. External engines are declared in configuration files, possibly in the same file as a plug-in, as in the sample UDR library that is implemented in $(root)/plugins.

  external_engine = UDR {
  	plugin_module = UDR_engine
  }

  plugin_module = UDR_engine {
  	filename = $(this)/udr_engine
  	plugin_config = UDR_config
  }

plugin_config = UDR_config {
	path = $(this)/udr
  }
            

When Firebird wants to load an external routine (function, procedure or trigger) into its metadata cache, it gets the external engine through the plug-in external engine factory and asks it for the routine. The plug-in used is the one referenced by the attribute plugin_module of the external engine.