May to August 2022

May 2022
  • New QA. Problem with escaping special characters in the substitution patterns vs "r" prefix (i.e. make it be interpreted as raw data).

  • New QA. Problem with detection actual reason related to filesystem or I/O when running ISQL on localized OS (got  UnicodeDecodeError but error stack pointed to "wrong" line of code. One need to make argument "io_enc=locale.getpreferredencoding()" default when call act.isql().

  • New QA. Timestamp is shown without fractional part if it is ".000" (i.e. no milliseconds there).

  • OLTP-EMUL: crash of LI-V3.0.10.33585, SS, during heavy workload (IndexTableScan).

  • New QA: default transactions isolation level differ in firebird-driver vs FDB (snapshot vs RC).

  • New QA: regression in tests which worked fine before, related to user_factory() (got "AttributeError: 'list' object has no attribute 'items'").

  • New QA: regression with some tests which launch trace session ( core_3168; core_3008 ) - they hang.

  • New QA: one need to have ability to call act.trace() with specifying NON-privileged user who has necessary grant to trace attachments.

  • New QA: discussed ability to start two Tx for each of connections using DistributedTransactionManager.

  • QA fbtest. Linux, / 33593, Classic: several crashes while running FBT. No dump because client detached before his server process crashed (probably crashes related to tests for COREs: 4319, 4539 5685, 6040, 3064 and 4937).

  • False-negative outcome is issued when DB state is changed to shut_single or shut_multi, and this is done by NON-SYSDBA who has all necessary system privileges for that. Created ticket #7189.

  • New QA. Test on localized Windows can get weird "ExecutionError: ISQL execution failed / UnicodeDecodeError" without detalization.
    As is turned out, problem related to attempt to create database while its file already exists. Discussed ability to show SQLSTATE and/or gdscode.

  • Continued investigation of problem with FB 4.x  'gfix -online' and other errors related to shared memory when work in parallel with several
    databases via XNET protocol ("MappingIpc: Cannot initialize the shared memory / DB probably opened by another..." )

  • Regression when derived table has column evaluated as result of subquery with IN(), ANY() or ALL() predicate:
    "invalid BLR at offset ... / context already in use". Created ticket #7183.

  • Parallel restore can work slower than without "-par ..." switch if command contain "-v" rather than "-verbint " (FB 4.x  experimental build 2762).

  • WI-V3.0.10.33580, experimental build: non-readable data in mon$context_variables.mon$variable_value when it has non-ascii characters.

  • NEW QA: adapted ~45 tests for running under pytest + firebird-qa plugin

June 2022
  • NEW QA. Questions / proposals about feature to create temp DB using its alias // db_factory( filename = '#' ).

  • It is desirable to see page cache usage per attachment when connection established with specifying number of pages, e.g. "isql -cache 12345". Created ticket #7198.

  • QA fbtest., Classic, Linux: crash while running FBT. No dump because client detached before his server process crashed.

  • QA fbtest. Detected bug in build could not restore DB with results from .fbk for regular FBT. Bug related to index creation.

  • Added check of successful restore into qa_rundaily batch scripts (.bat and .sh).

  • Encryption files fbSampleDbCrypt.dll and fbSampleDbCrypt.conf exist only in FB 4.x (in examples/prebuilt/) but also required in FB-3.x for test purposes.

  • DROP database leads FB to hang if it is executing 'alter database decrypt' at that moment. Created ticket #7200.

  • NEW-QA. Adjusted added notes about encryption-related tests; adjusted pytest.ini (added declaration of encryption marker).

  • NEW-QA. Discussed ability to create temporary (test) database using reference to its ALIAS that was beforehand added into databases.conf; syntax "db_tmp = db_factory(filename='#tmp_5965_alias', ...)" was proposed.

  • NEW-QA. Discussed performance measuring tests with 'fragile' outcomes (psutil + median).

  • NEW-QA. Test for core-5418 ( #5691 ) requires write access to FB home folder because of need for direct log modification. Windows only: 'ISQL -ch utf8 ...' either silently quits to OS or issues non-expected 'malformed string' when non-ascii character occurs
    in the typed command. Created  #7202. NEW-QA. Questions about run act.isql() if we want completely avoid unicode-related errors.

  • Trace: it will be good to see performance statistics for DDL statements. Discussed with Vlad, created #7208.

  • Crash occurs if we try to create DB on then
    Problem relates to shmem-structures (fb13_user_mapping and  fb13_trace files from %programdata%\firebird\ )

  • NEW-QA. Questions about storage for settings that are common for several tests. Package 'configparser' will be used for that purpose (since plugin v. 0.16.0)

  • NEW-QA: db_factory() can NOT use init_script with specifying charset different than UTF8.

  • Check trace content when ISC_* variables missed and someone tries to make connection without specifying user / password (Alex request). Also checked #5746. Results were expected.

  • LINUX only: changing dialect of previously encrypted DB from 3 to 1 leads DB to be unrestorable ("Crypt plugin FBSAMPLEDBCRYPT failed to load").

  • QA fbtest. Detected several failed tests on LINUX and Windows, FB Server hangs when running test for CORE-6469.

  • QA fbtest. CS, Windows: hangs when running test "functional.tabloid.max-path-length" (which never did hang before).

  • Connect as NON-ascii user: number of octets in mon$user more than 2x in FB 4.x comparing to FB 3.x (e.g. 78 vs 31).

  • NEW QA: started first usage of configparser package and test_config.ini with shared settings for several tests (currently for only encryption-related).

  • NEW QA: one need to have somewhat like GLOBAL_mapping_factory().

  • Problems with test for CORE-4337 ("excessive" message about FAILED ATTACH_DATABASE appears in the trace).

  • Lot of changes in qa_rundaily.bat (mostly long-waited; done after change test-running machine)

  • QA fbtets. WI-V3.0.10.33601, special build: crash when run FBT suite (terminated abnormally (3221225477)), but that was not properly logged (log shows that all tests completed fine). Probably crash was on "bugs.core_3690_dialect_3" OR "bugs.core_3691".

  • QA fbtest. QA_rundaily.bat on new host: made benchmark about how performance depends on disks characteristics (including RAM drive).

  • QA_rundaily, on new machine: restored full functionality with uploading results to

  • Presense of %programdata%\fb13_user_mapping prevents from launching two FB instances (4.x and 5.x) at the same time.

  • Re-implemented test for CORE-4337.

  • NEW QA: problems with obtaining content of firebird.log when it has localized messages.

  • Benchmark of INSERTS which are performed differently (requested by Alexey Kovyazin):
      1) LOOP with prepared INSERT statement;
      2) LOOP with prepared EXECUTE BLOCK and INSERT in ES;
      3) INSERT 100000 rows from EXTERNAL table;
      4) BULK_INSERT.

  • NEW QA: adapted ~25 tests for running under pytest + firebird-qa plugin.

July 2022

  • QA fbtest. Implemented / refactored /adjusted  .fbt for 18 tests.

  • NEW QA. Adapted 42 tests for running under pytest + firebird-qa plugin.

  • QA fbtest. Problems with re-implementing test for CORE-4337 after recent commits.

  • New QA. Continued discussion solution of UnicodeDecode errors when obtaining content of firebird.log which contains localized messages or binary data (garbage).

  • Resumed work with batch for making FB snapshots. Problem with absent env variable VS150COMNTOOLS. Decided to add special variable into make.conf in order to create VS150COMNTOOLS explicitly in such cases. Additional MSVC10 required for build FB 3.x.

  • Finally disabled tests which evaluates EXP( ) for discovering minimal value that can be distinguish from zero.
    Reason: different results on hardware depending on 1) CPU manufacturer; 2) major FB-family (3.x vs 4.x).
    Example: exp(-744.0346068132731393) vs exp(-745.1332191019410399).

  •, Classic, Windows: crash (special build), related to setLockAttachment.

  • Re-implemented test for CORE-6097 (connecting to DB with non-ascii user name) because of problems after migrating to new host.

  • Test for CORE-4880 (performance-measurement and comparison) became fail for every run after commit that fixed #7222

  • QA fbtest. Re-implemented several tests which permanently failed: COREs: 4880, 4337, 5431; few functional/basic/ tests.

  • WI-5.0.0573, SS/CS: crash when run test for CORE-6353 (related to int128).

  • Windows, FB 5.x: several builds hanged on test "functional.tabloid.max-path-length".

  • ISQL for FB 4.x+: command 'help set' does not show 'SET KEEP_TRANS_params'.

  • QA-run scenario: it will be better to create several separate jobs and run them with requirement to finish for some (restrict) time scope, with forcible terminating otherwise and gathering debug info in that case (create dump and stack trace of firebird process(es)).

  • QA fbtest. Several tests failed on, with additional message: "undefined message number" (COREs: 0886, 2032, 6351; GH: 7034; tabloid.eqc.343715). Some tests became fail after years of normal work (core-3094, -3323, -5431).

  • LI-V3.0.10.33587, SS, experimental build: crash.

  • Connections that are created by 'gfix -sweep' and 'gstat -r' can not be handled by DB-level triggers.

  • Eventually found how to reproduce bug described in core-4337, test was again totally re-implemented.

  • Adjusted test bugs/core_5488_session_idle.fbt after discussion with Vlad.

  • Again fully re-implemented test CORE-3323 (it often failed on Windows). Reproduced bug on 2.5.0, checked on 2.5.1 and later.

  • Discussed with FB team implementation of test for #6802.

  • Discussed with FB team ways how to develope tests for both old and new qa (+ estimating time).

  • Test for gh-6947 leads Python.exe to crash (Windows only). Problem appeared since migration to new host (from Win-2008 Server R2 to Windows-10 workstation). Crash occurred only when FB worked in embedded mode.

  • QA fbtest. Windows, SS: fbt_run hanged on test for CORE-5948.

  • Resumed work on test for CORE-6071 (make it work also on Linux). Changes in plugin.conf required for this test work on Linux (need to create section with name of plugin in UPPERCASE form).

  • QA fbtest. Additional work with batch that makes standard and custom FB snapshots. Problems appeared after migrating to new host.

  • Attempt to connect to DB in shutdown state issues different message if we use XNET protocol: "SQLSTATE = 08006 connection lost to database" (instead of "SQLSTATE = HY000 database shutdown"). Created ticket #7239.

  • Additional investigation related to slow inserts when DB is in delta state (Alexey Kovyazin request). Initial work started in APR-2022.

  • Discussed with dimitr et al possible approaches of implementation of tests which relate to performance measure. It would be nice to have ability to get CPU cycles from mon$ table(s) (like this is shown in ProcessExplorer, "Performance" tab).

  • Weird messages in the trace:
    * "ERROR AT JStatement::prepare / 335544395 : table id NNNN is not defined"
    * "335544395 : table id NNNN is not defined" (ERROR AT JStatement::prepare).
    -- was encountered in one of trace logs.

  • Resumed discussion of ability to use config with parameters which are 'common' for several / all FB instances. Starting use [DEFAULT] section in firebird-driver.conf

  • firebird-driver source: question about DbInfoCode.ALLOCATION vs DbInfoCode.DB_SIZE_IN_PAGES

  • Refactored auxiliary script for parsing trace log: moved some settings into config, added [optional] parsing of FB sources in order to obtain full list of gdscodes, their mnemonas and descriptions.

  • Further fefactoring batch that builds FB snapshots (Windows-10).

  • OLTP-EMUL: crash of LI-V3.0.11.33608. Sent report FB team (Alex reply: crash caused by EDS::Transaction::jrdTransactionEnd)

  • It would be useful if Services API will return paths to 1) dir_plugins ; 2) dir_sample (both these macroses are mentioned in the firebird.conf).

  • NEW QA: Problem with run tests when FB in Classic mode: samle_qa = _vars_['sample_dir'] / 'qa' ==> INTERNALERROR> KeyError: 'sample_dir'.

  • FB-4.x snapshot build process failed on Windows ("NMAKE : fatal error U1073: don't know how to make '\extern\cloop\src\cloop\*.cpp'").

  • Implemented aux batch that will create dumps and stack traces for all running FB processes (actual for CS).

  • NEW QA. Discussed current activities and state of work. About 50 tests needed to adapt from old test suite.
    Resumed discussion ability to launch trace using NON-sysdba user/password/role (which has apropriate privilege).

  • NEW QA. Tests hang (pytest does not return control to OS) if we use wrong password when start it.

  • NEW QA. Discussed ability to work with firebird.log which contain lot of garbage (binary data). Related to

  • Nulls in rdb$procedures.rdb$debug_info lead to missed number of line and column when error raises in such PSQL object (and the same for rdb$triggers).

August 2022

  • NEW QA. Adapted 33 tests for running under pytest + firebird-qa plugin.

  • QA fbtest. Implemented / refactored /adjusted 9 tests from FBT suite.

  • NEW QA: started adaptation of tests which must rely on existing alias in the databases.conf (filename = '#').
  • NEW QA. Additional work and discussion about test for core-0733 (using per-connection parameters and custom driver-confir instance for that).

  • Discussion with FB team ability to see diff values in mon$database.mon$owner when we connect to existing DB using local protocol and then create new DB (result depends on whether we did specify user in ISQL command line or no).

  • NEW QA. Firebird-driver specific: connect to DB using Win_Sspi requires explicit specifying user as empty string.

  • NEW QA. Feature request: mapping_factory() will be useful if we want to create *global* mapping and can not use (for some reasons) self-security database.

  • NEW QA. Problem with reproducing issues described in the ticket CORE-6147.

  • NEW QA. Problem with creating test DB: one need to have ability to create self-sec DB using NON-local protocol in the 'CREATE DATABASE' command.

  • Re-implemented aux batch for monitoring disk activity using typeperf (needed for localized Windows with diff names of misc. counters)

  • Implemented scripts (bat and sh) for check duration of attempts to establish connection to node that for sure is unavaliable (, aka "bogon IP").
    Checked effect of ConnectionTimeout parameter. Windows will wait 21 second, Linux about 127...129 seconds and then issues "SQLSTATE = 08006".
    Important explanations (in russian):

  • NEW QA. Discussed with pcisar whether DefaultTimeZone must be honored also in the db_cfg_object.config.value (rather than only in db_cfg_object.session_time_zone.value).

  • Windows,, SS: crashes on new DB when mon$database is queried.

  • Linux: changed aux script that watches for crashes. Only first 20 frames of thread N1 must be show in the *body* of letter (t1 / bt 20).
    The whole stack trace must be compressed and is added as attachment. Discussed with Alex.

  • Windows,, SS & CS, intermediate snapshot from github: crash on CORE-4653.

  • Windows, error during build snapshot, "LINK : fatal error LNK1181: cannot open '...\re2.lib'".
    One need to define OS variable: "set FBBUILD_BUILDTYPE=release" before call make_icu.bat.

  • NEW QA. Could not run test for core-4998 on too old FB versions ( RC1, nov-2015).
    Got: "DatabaseError: invalid format for transaction parameter block / -wrong version of transaction parameter block"

  • New-QA, FB 3.x (only). Weird delay for ~5 s between iterations when making LOCAL connection (and NO running trace).

  • Total refactoring and lot of changes in the batch that makes FB snapshots on Windows.

  • Feature request. It will be usefult if ExtConnPool* parameters will be per-database. Discussed with Vlad.

  • NEW QA. Feature request. Parser of replication.conf can be improved: it will be useful (for QA-purposes) if it we can specify journal* directories using known MACROSES, e.g.: journal_directory = "$(dir_sampleDb)\fb-replication.journal" etc.

  • NEW QA. Problem with usage of "srv.database.set_replica_mode()": got "Internal error ... attempt to store data in dataless clumplet".

    Feature request. It will be useful to have ability to ask number of last segment that was successfully applied to replica DB.

  • New QA: replication-related tests. Strange warning "PytestAssertRewriteWarning", *ONLY* on Linux (seems that the only workaround is '--disable-warnings').

  • NEW QA. Result of crypt_hash() is always shown as HEX string in ISQL. One need to make some additional actions in Python tests to have proper result of comparing these results - see comments in functional\replication\

  • NEW QA. Almost all tests have been adapted for Windows, 11 remained marked as 'NOT IMPLEMENTED'. Further all tests will be checked on Linux.

  • Linux. Totally re-implemented .sh-scenario which makes FB snapshots (currently without usage of chroot; can be used only for test purposes).

  • WI-V3.0.10.33587, experimental build: crash on REPLICA ("Allocated 181475 page buffers of 250000 requested")

  • LI-V3.0.10.33587,experimental build: crash, related to allocation from null-pool (according to alex reply).

  • WI-V3.0.8.33452, special build: hangs, unable to establish connection (perhaps due to mapUser problems).

  • WI-V3.0.10.33587, CS, experimental build: 12 crashes plus weird messages in FB log:
     * "Replication plugin exists but cannot be loaded"
     * "SERVER: DuplicateHandle failed...";
     * "Operating system call WaitForSingleObject failed. Error code 0"

  • Implemented test for precise measure time need to for establishing attachment and further detach, with logging (requested by Alexey Kovyazin).
    Abnormally long 'pauses' can be easy detected in the log between N-th detach and (N+1)th attach, even when DB has FW = OFF and is in READ-ONLY mode.
    Weird delays can be observed either in CS or in SS - but only when LINGER = NULL or 0.
    Continued with additional logging using typeperf (disk activity counters, suggested by Vlad).
    The reason relates to forced flush of DB page cache when shmem files are created (fb_trace etc) in the FIREBIRD_LOCK dir.

  • LI-V3.0.10.33587, SS, experimental build: dozen crashes.

  • WI-V3.0.10.33616, experimental build: BUGCHECK  "too many savepoints (287), file: tra.cpp line: 1443".

  • LI-V3.0.10.33616, experimental build: lot of crashes.

  • LI-V3.0.10.33618, experimental build: crashes when executing special .sql which kills too old user attachments  (after worktime finished).

  • Refactored scenario (.sh-script ) which launches ISQL for kill too old attachments: needed to create dump of FB process before that action, with further comparison to dump that will occur in case of FB crash).

  • Windows builds: finished refactoring of batch scenario that makes FB snapshots, changed directory structure on FTP server.

  • WI-V3.0.11.33616, Classic: crash of one of FB processes at night, not related to sweep or any user activity.


Pavel Zotov
Moscow, Russian Federation