Firebird: Developer's Report: QA Stress Testing & Development

September to December 2021

December 2021

Created / modified QA tests for 6 tracker tickets
Trace block parser (aux Python script): refactored in order to increase performance
New QA: several iterations with opening multiple cursors on each step: issuing "delete from mon$attachments ..." (by separate SYSDBA attachment) leads to exception only in first iteration
Refactoring of aux script for trace block analyzing: extract and accumulate statistics about idle state of attachments
Bugcheck encountered when attempted to create test for gh-7070: "decompression overran buffer (179), file: sqz.cpp line: 293" (got in WI-V4.0.1.2691; WI-T5.0.0.347)
New QA: additional reports about tests runs outcome

November 2021

Created / modified QA tests for ~20 tracker tickets
Found / reported bugs:
Found way how to make time-related comparison much more accurate, fully re-implemented several tests appropriately (psutil package must be installed)
Analyzed IBE-setup_benchmark ( https://ibexpert.com/benchmark/setup_benchmark.exe )
Refactored script for building FB snapshots on Windows test server
Implemented test for check performance of string-comparison operators (">=", "<=", STARTING WITH, LIKE '
%', SIMILAR TO ...), related to #6915
New QA framework: upgraded firebird-driver to 1.3.3 and pytest 0.2.0, run all ISQL-based tests on official releases of FB 3.0.7, 4.0.0 and fresh snapshot of 4.0.1
Run multiple tests related to performance of search operations against columns with single-byte character sets
Scrollable cursors: made basic tests, created test for automated generation of actions with evaluating expected results and compare actual outcome with this result

October 2021

Created / modified QA tests for 14 tracker tickets
Found / reported bugs: #6989, #6998
OLTP-EMUL, workload by 20 sessions, applied to DB that involved in replication schema: crash when replica-DB is subject of loop "shutdown / online / shutdown/ ..."
OLTP-EMUL, work under replication: reproduced crash described in fb-devel vol 186 issue 3" ("read errno during replicaton")
firebird-driver (new QA): detected regression in 4.0.1.2602 vs 4.0.1.2592. Unable to connect using Python v.3.9.4 / and FB Driver v. 1.3.2, got: "SQLCODE: -904 / invalid database handle (no active connection)"
OLTP-EMUL: validated replication schema "one master - many replicas" (async)
Refactored aux script for detect missing FB tests runs
Refactored qa_rundaily.bat: added check for results of security DB initialization
firebird-driver / pytest (new QA): could not get result on firebird-driver v 1.3.4 and pytest-6.2.4: got "TypeError: isql_act() got an unexpected"

September 2021

Created / modified QA tests for ~30 tracker tickets
Found / reported bugs: #6946, #6947, #6966
OLTP-EMUL: add support of FB major family 5.0
firebird-driver (new QA), v1.2.1: cannot establish connection to FB 5.x (any protocol); problems with usage on FB 3.x and 4.x when selected protocol = XNET
firebird-driver (new QA): found minor inaccuracy in the documentation related to con.info
firebird.driver: discussed ability to obtain current DB encryption state WITHOUT querying mon$database.mon$crypt_state and mon$crypt_page columns; discussed ability to use customized DPB
firebird.driver (new QA): cannot obtain name of the key which was used to encrypt DB
firebird.driver (new QA): cannot obtain fb_info_tra_snapshot_number
firebird.driver (new QA): could not connect to FB 5.x, got (rarely) crash in one of Python libraries, strange output in console
OLTP-EMUL, FB 4.0.1.2585 and 5.0.0.192, Windows: tested replication between two different machines, workload = 20 sessions. Got bugcheck (wrong record length) followed by "Segment N replication failure at offset M + PK violation".
firebird.driver (new QA): weird behaviour with INT128 datatype in FB 4.x: can not use it in DML; can get Python AV (failed unit: DLLs\libffi-7.dll )
Misc improvements in fbt-cross-report

Pavel Zotov
Moscow, Russian Federation