Command-line Utilities

Table of Contents

Backup Tools
ISQL Query Utility
gsec Authentication Manager
gfix Server Utility

Backup Tools

New On-line Incremental Backup
gbak Backup/Porting/Restore Utility

Firebird 2 brings plenty of enhancements to backing up databases: a new utility for running on-line incremental backups and some improvements to gbak to avoid some of the traps that sometimes befall end-users.

New On-line Incremental Backup

N. Samofatov

Fast, on-line, page-level incremental backup facilities have been implemented.

The backup engine comprises two parts:

NBak, the engine support module
NBackup, the tool that does the actual backups

Nbak

The functional responsibilities of NBAK are:

to redirect writes to difference files when asked (ALTER DATABASE BEGIN BACKUP statement)
to produce a GUID for the database snapshot and write it into the database header before the ALTER DATABASE BEGIN BACKUP statement returns
to merge differences into the database when asked (ALTER DATABASE END BACKUP statement)
to mark pages written by the engine with the current SCN [page scan] counter value for the database
to increment SCN on each change of backup state

The backup state cycle is:

nbak_state_normal -> nbak_state_stalled -> nbak_state_merge -> nbak_state_normal

In normal state writes go directly to the main database files.
In stalled state writes go to the difference file only and the main files are read-only.
In merge state new pages are not allocated from difference files. Writes go to the main database files. Reads of mapped pages compare both page versions and return the version which is fresher, because we don't know if it is merged or not.

Note

This merge state logic has one quirky part. Both Microsoft and Linux define the contents of file growth as "undefined" i.e., garbage, and both zero-initialize them.

This is why we don't read mapped pages beyond the original end of the main database file and keep them current in difference file until the end of a merge. This is almost half of NBak fetch and write logic, tested by using modified PIO on existing files containing garbage.

NBackup

The functional responsibilities of NBackup are

to provide a convenient way to issue ALTER DATABASE BEGIN/END BACKUP
to fix up the database after filesystem copy (physically change nbak_state_diff to nbak_state_normal in the database header)
to create and restore incremental backups.

Incremental backups are multi-level. That means if you do a Level 2 backup every day and a Level 3 backup every hour, each Level 3 backup contains all pages changed from the beginning of the day till the hour when the Level 3 backup is made.

Backing Up

Creating incremental backups has the following algorithm:

Issue ALTER DATABASE BEGIN BACKUP to redirect writes to the difference file
Look up the SCN and GUID of the most recent backup at the previous level
Stream database pages having SCN larger than was found at step 2 to the backup file.
Write the GUID of the previous-level backup to the header, to enable the consistency of the backup chain to be checked during restore.
Issue ALTER DATABASE END BACKUP
Add a record of this backup operation to RDB$BACKUP_HISTORY. Record current level, SCN, snapshot GUID and some miscellaneous stuff for user consumption.

Restoring

Restore is simple: we reconstruct the physical database image for the chain of backup files, checking that the backup_guid of each file matches prev_guid of the next one, then fix it up (change its state in header to nbak_state_normal).

Usage

    nbackup <options>

Valid Options

   -L <database>   Lock database for filesystem copy
   -N <database>       Unlock previously locked database
   -F <database>              Fixup database after filesystem copy
   -B <level> <database> [<filename>]  Create incremental backup
   -R <database> [<file0> [<file1>...]]   Restore incremental backup
   -U <user>             User name
   -P <password>         Password

Note

<database> may specify a database alias
incremental backups of multi-file databases are not supported yet
"stdout" may be used as a value of <filename> for the -B option

Improvements

(V.2.0.6) An improvement has been done for POSIX versions to address a problem whereby the full backup tool of the nBackup would hog I/O resources when backing up large databases, bringing production work to a standstill. Now, nBackup tries to read from the operating system cache before attempting to read from disk, thus reducing the I/O load substantially.

Note

The “cost” may be a 10 to 15 percent increase in the time taken to complete the full backup under high-load conditions.

Tracker reference CORE-2316.

User Manual

P. Vinkenoog

A user manual for NBak/NBackup has been prepared. It can be downloaded from the documentation area at the Firebird website: www.firebirdsql.org/pdfmanual/ - the file name is Firebird-nbackup.pdf.

gbak Backup/Porting/Restore Utility

Content

Changed Behaviours, New Switches

V. Horsun

The new gbak switch

  -RECREATE_DATABASE [OVERWRITE]

is a separate switch designed to make harder for the unsuspecting to overwrite a database accidentally, as could occur easily with the shortened form of the old switch:

  -R[EPLACE_DATABASE]

In summary:

gbak -R (or gbak -r) now applies to the new -R[ECREATE_DATABASE] switch and will never overwrite an existing database if the O[VERWRITE] argument is absent
The short form of the old gbak -R[EPLACE_DATABASE] is now -REP[LACE_DATABASE]. This switch does not accept the O[VERWRITE] argument.
The -REP[LACE_DATABASE] switch should be considered as deprecated, i.e. it will become unavailable in some future Firebird release.

This change means that, if you have any legacy batch or cron scripts that rely on “gbak -r” or “gbak -R” without modification, then the operation will except if the database exists.

If you want to retain the ability of your script to overwrite your database unconditionally, you will need to modify the command to use either the new switch with the OVERWRITE argument or the new short form for the old -REPLACE_DATABASE switch.

gbak -V and the “Counter” Parameter

During Firebird 1 development, an optional numeric <counter> argument was added to the -V[erbose] switch of gbak for both backup and restore. It was intended to allow you to specify a number and get a running count of rows processed as the row counter passed each interval of that number of rows. It caused undesirable side-effects and was removed before Firebird 1.0 was ever released. So, although it never happened, it was documented as “implemented” in the release notes and other places.