Firebird Documentation IndexFirebird 2.5 Release NotesAdministrative Features → Trace and Audit Services
Firebird Home Firebird Home Prev: Administrative FeaturesFirebird Documentation IndexUp: Administrative FeaturesNext: Monitoring Improvements

Trace and Audit Services

Vlad Khorsun

Overview of Features
The System Audit Session
User Trace Sessions
Trace Scope on Windows
Use Cases
Trace Plug-in Facilities

The new trace and audit facilities in v.2.5 were initially developed from the TraceAPI contributed to us by Nickolay Samofatov that he had developed for the Red Soft Database, a commercial product based on Firebird's code.

Overview of Features

The new trace and audit facilities enable various events performed inside the engine, such as statement execution, connections, disconnections, etc., to be logged and collated for real-time analysis of the corresponding performance characteristics.

A trace takes place in the context of a trace session. Each trace session has its own configuration, state and output.

The Firebird engine has a fixed list of events it can trace. It can perform two different sort of traces: a system audit and a user trace. How the engine forms the list of events for a session depends on which sort of trace is requested.

Tip

Every trace session is assigned a unique session ID. When any trace session begins, the Services Manager outputs this ID as the message

  Trace session ID nnnn started
          

where nnnn is the ID, of course.

The System Audit Session

A system audit session is started by the engine itself. To determine which events the session is “interested in”, it reads the contents of a trace configuration file as it goes to create the session.

A new parameter in firebird.conf, AuditTraceConfigFile points to the name and location of the file. There can be at most one system audit trace in progress. By default, the value of this parameter is empty, indicating that no system audit tracing is configured.

A configuration file contains list of traced events and points to the placement of the trace log(s) for each event. It is sufficiently flexible to allow different sets of events for different databases to be logged to separate log files. The template file file fbtrace.conf, found in Firebird's root directory, contains the full list of available events, with format, rules and syntax for composing an audit trace configuration file.

Tip About the fbtrace.conf File

The file contains a large amount of commented text explaining the purpose and syntax of each entry. As sub-releases progress, keep an eye on new events and facilities that will be added from time to time to improve the tracing capabilities.

For example, a late pre-release enhancement enables Services events to be traced by name and targeted using include and exclude filters.

As another example, the matching algorithm for path names was improved to interpret strings in the configuration file according to the platform character set and, basing the strings on UTF-8, to apply platform rules such as treating file names as case-insensitive on Windows and case-sensitive on POSIX. (Tracker reference CORE-2404, A. dos Santos Fernandes).

An improvement added in V.2.5.2 was the ability to configure a trace session to log details of both user and automatic sweeps. Search the template file for the “SWEEP_” options.

User Trace Sessions

A user trace session is managed by user, using some new calls to the Services API. There are five new service functions for this purpose:

  • start: isc_action_svc_trace_start

  • stop: isc_action_svc_trace_stop

  • suspend: isc_action_svc_trace_suspend

  • resume: isc_action_svc_trace_resume

  • list all known trace sessions: isc_action_svc_trace_list

The syntax for the Services API calls are discussed in the topic New Trace Functions for Applications in the chapter entitled Changes to the Firebird API and ODS.

Workings of a User Trace Session

When a user application starts a trace session, it sets a session name (optional) and the session configuration (mandatory). The session configuration is a text file conforming to the rules and syntax modelled in the fbtrace.conf template that is in Firebird's root directory, apart from the lines relating to placement of the output.

Note

Such files obviously do not live on the server. It will be the job of the application developer to design a suitable mechanism for storing and retrieving texts for passing in the user trace request.

For example, the command-line fbsvcmgr utility supports a saved-file parameter, trc_cfg.

The output of a user session is stored in set of temporary files, each of 1 MB. Once a file has been completely read by the application, it is automatically deleted. By default, the maximum total size of the output is limited to 10 MB. It can be changed to a smaller or larger value using the MaxUserTraceLogSize in firebird.conf.

Once the user trace session service has been started by the application, the application has to read its output, using calls to isc_service_query(). The service could be generating output faster than the application can read it. If the total size of the output reaches the MaxUserTraceLogSize limit, the engine automatically suspends the trace session. Once the application has finished reading a file (a 1 MB part of the output) that file is deleted, capacity is returned and the engine resumes the trace session automatically.

At the point where the application decides to stop its trace session, it simply requests a detach from the service. Alternatively, the application can use the isc_action_svc_trace_* functions to suspend, resume or stop the trace session at will.

Tip

The name of the character set of the attachment is included in any corresponding trace log records, placed between user:role and protocol:port, e.g.,


A.FDB (ATT_36, SYSDBA:NONE, WIN1251, TCPv4:127.0.0.1)
            

If no character set is specified in the DPB, NONE is written to the attachment character set slot in the trace log record.

(CORE-3008)

Who Can Manage Trace Sessions?

Any user can initiate and manage a trace session. An ordinary user can request a trace only on its own connections and cannot manage trace sessions started by any other users. Administrators can manage any trace session.

Abnormal Endings

If all Firebird processes are stopped, no user trace sessions are preserved. What this means is that if a Superserver or Superclassic process is shut down, any user trace sessions that were in progress, including any that were awaiting a resume condition, are fully stopped and a resume cannot restart them.

Note

This situation doesn't apply to the Classic server, of course, since each connection involves its own dedicated server instance. Thus, there is no such thing as “shutting down” a Classic server instance. No service instance can outlive the connection that instigated it.

User Trace Sample Configuration Texts

The following samples provide a reference for composing configuration texts for user trace sessions.

  1. Trace prepare, free and execution of all statements within connection 12345

    <database mydatabase.fdb>
    	enabled                true
    	connection_id          12345
    	log_statement_prepare  true
    	log_statement_free     true
    	log_statement_start    true
    	log_statement_finish   true
    	time_threshold         0
    </database>
                  
  2. Trace all connections of given user to database mydatabase.fdb, logging executed INSERT, UPDATE and DELETE statements and nested calls to procedures and triggers, and show corresponding PLANs and performance statistics.

    <database mydatabase.fdb>
    	enabled                true
    	include_filter         %(INSERT|UPDATE|DELETE)%
    	log_statement_finish   true
    	log_procedure_finish   true
    	log_trigger_finish     true
    	print_plan             true
    	print_perf             true
    	time_threshold         0
    </database>
                  

Command-line Requests for User Trace Services

A new command-line utility, named fbtracemgr, has been added for working interactively with trace services. It has its own syntax of switches and parameters, discussed in detail, with examples, in the Utilities chapter.

As well, the general Services utility, fbsvcmgr, can be used for submitting service requests from the command line, as exemplified in the following examples.

  1. Start a user trace named “My trace” using a configuration file named fbtrace.conf and read its output on the screen:

      fbsvcmgr service_mgr action_trace_start trc_name "My trace" trc_cfg fbtrace.conf
                  

    To stop this trace session, press Ctrl+C at the fbsvcmgr console prompt. (See also (e), below).

  2. List trace sessions:

      fbsvcmgr service_mgr action_trace_list
                  
  3. Suspend trace sesson with ID 1

      fbsvcmgr service_mgr action_trace_suspend trc_id 1
                  
  4. Resume trace sesson with ID 1

      fbsvcmgr service_mgr action_trace_resume trc_id 1
                  
  5. Stop trace sesson with ID 1

      fbsvcmgr service_mgr action_trace_stop trc_id 1
                  

    Tip

    List sessions (see b.) in another console, look for the ID of a session of interest and use it in the current console to stop the session.

Trace Scope on Windows

Tracker reference CORE-2588

On Windows, the trace tools exhibit shared memory conflicts if multiple engine instances in different Windows sessions are allowed to run trace in global namespace. Tracing scope has therefore been restricted to only those processes that are accessible from the current Windows session.

Use Cases

There are three general use cases:

  1. Constant audit of engine activity

    This is served by system audit trace. Administrator creates or edits the trace configuration file, sets its name via the AuditTraceConfigFile setting in firebird.conf and restarts Firebird. Later, the administrator could suspend, resume or stop this session without needing to restart Firebird.

    Important

    To make audit configuration changes known to the engine, Firebird must be restarted.

  2. On-demand interactive trace of some (or all) activity in some (or all) databases

    An application (which could be the fbtracemgr utility) starts a user trace session, reads its output and shows traced events to the user in real time on the screen. The user can suspend and resume the trace and, finally, stop it.

  3. Engine activity collection for a significant period of time (a few hours or perhaps even a whole day) for later analysis

    An application starts a user trace session, reading the trace output regularly and saving it to one or more files. The session must be stopped manually, by the same application or by another one. If multiple trace sessions are running, a listing can be requested in order to identify the session of interest.

Trace Plug-in Facilities

A new Trace API will provide a set of hooks which can be implemented as an external plug-in module to be called by the engine when any traced event happens. A plug-in will assume responsibility for logging such events in some custom way.

This Trace API exists in Firebird 2.5 and is in use. However, since it will be changed in forthcoming sub-releases, it is not officially published and must be regarded as unstable.

The implemented “standard” trace plug-in, fbtrace.dll (.so), resides in the \plugins folder of your Firebird 2.5 installation.

Prev: Administrative FeaturesFirebird Documentation IndexUp: Administrative FeaturesNext: Monitoring Improvements
Firebird Documentation IndexFirebird 2.5 Release NotesAdministrative Features → Trace and Audit Services