May to August 2022

• New QA. Problem with escaping special characters in the substitution patterns vs "r" prefix (i.e. make it be interpreted as raw data).

• New QA. Problem with detection actual reason related to filesystem or I/O when running ISQL on localized OS (got  UnicodeDecodeError but error stack pointed to "wrong" line of code. One need to make argument "io_enc=locale.getpreferredencoding()" default when call act.isql().

• New QA. Timestamp is shown without fractional part if it is ".000" (i.e. no milliseconds there).

• OLTP-EMUL: crash of LI-V3.0.10.33585, SS, during heavy workload (IndexTableScan).

• New QA: default transactions isolation level differ in firebird-driver vs FDB (snapshot vs RC).

• New QA: regression in tests which worked fine before, related to user_factory() (got "AttributeError: 'list' object has no attribute 'items'").

• New QA: regression with some tests which launch trace session ( core_3168; core_3008 ) - they hang.

• New QA: one need to have ability to call act.trace() with specifying NON-privileged user who has necessary grant to trace attachments.

• New QA: discussed ability to start two Tx for each of connections using DistributedTransactionManager.

• QA fbtest. Linux, 3.0.10.33592 / 33593, Classic: several crashes while running FBT. No dump because client detached before his server process crashed (probably crashes related to tests for COREs: 4319, 4539 5685, 6040, 3064 and 4937).

• False-negative outcome is issued when DB state is changed to shut_single or shut_multi, and this is done by NON-SYSDBA who has all necessary system privileges for that. Created ticket #7189.

• New QA. Test on localized Windows can get weird "ExecutionError: ISQL execution failed / UnicodeDecodeError" without detalization.
  As is turned out, problem related to attempt to create database while its file already exists. Discussed ability to show SQLSTATE and/or gdscode.

• Continued investigation of problem with FB 4.x  'gfix -online' and other errors related to shared memory when work in parallel with several
  databases via XNET protocol ("MappingIpc: Cannot initialize the shared memory / DB probably opened by another..." )

• Regression when derived table has column evaluated as result of subquery with IN(), ANY() or ALL() predicate:
  "invalid BLR at offset ... / context already in use". Created ticket #7183.

• Parallel restore can work slower than without "-par ..." switch if command contain "-v" rather than "-verbint " (FB 4.x  experimental build 2762).

• WI-V3.0.10.33580, experimental build: non-readable data in mon$context_variables.mon$variable_value when it has non-ascii characters.

• NEW QA: adapted ~45 tests for running under pytest + firebird-qa plugin

• NEW QA. Questions / proposals about feature to create temp DB using its alias // db_factory( filename = '#' ).

• It is desirable to see page cache usage per attachment when connection established with specifying number of pages, e.g. "isql -cache 12345". Created ticket #7198.

• QA fbtest. 3.0.10.33596, Classic, Linux: crash while running FBT. No dump because client detached before his server process crashed.

• QA fbtest. Detected bug in build 5.0.0.502: could not restore DB with results from .fbk for regular FBT. Bug related to index creation.

• Added check of successful restore into qa_rundaily batch scripts (.bat and .sh).

• Encryption files fbSampleDbCrypt.dll and fbSampleDbCrypt.conf exist only in FB 4.x (in examples/prebuilt/) but also required in FB-3.x for test purposes.

• DROP database leads FB to hang if it is executing 'alter database decrypt' at that moment. Created ticket #7200.

• NEW-QA. Adjusted readme.md: added notes about encryption-related tests; adjusted pytest.ini (added declaration of encryption marker).

• NEW-QA. Discussed ability to create temporary (test) database using reference to its ALIAS that was beforehand added into databases.conf; syntax "db_tmp = db_factory(filename='#tmp_5965_alias', ...)" was proposed.

• NEW-QA. Discussed performance measuring tests with 'fragile' outcomes (psutil + median).

• NEW-QA. Test for core-5418 ( #5691 ) requires write access to FB home folder because of need for direct log modification. Windows only: 'ISQL -ch utf8 ...' either silently quits to OS or issues non-expected 'malformed string' when non-ascii character occurs
  in the typed command. Created  #7202. NEW-QA. Questions about run act.isql() if we want completely avoid unicode-related errors.

• Trace: it will be good to see performance statistics for DDL statements. Discussed with Vlad, created #7208.

• Crash occurs if we try to create DB on 4.0.0.927 then 4.0.0.2692.
  Problem relates to shmem-structures (fb13_user_mapping and  fb13_trace files from %programdata%\firebird\ )

• NEW-QA. Questions about storage for settings that are common for several tests. Package 'configparser' will be used for that purpose (since plugin v. 0.16.0)

• NEW-QA: db_factory() can NOT use init_script with specifying charset different than UTF8.

• Check trace content when ISC_* variables missed and someone tries to make connection without specifying user / password (Alex request). Also checked #5746. Results were expected.

• LINUX only: changing dialect of previously encrypted DB from 3 to 1 leads DB to be unrestorable ("Crypt plugin FBSAMPLEDBCRYPT failed to load").

• QA fbtest. Detected several failed tests on LINUX and Windows, FB 5.0.0.515. Server hangs when running test for CORE-6469.

• QA fbtest. 5.0.0.516 CS, Windows: hangs when running test "functional.tabloid.max-path-length" (which never did hang before).

• Connect as NON-ascii user: number of octets in mon$user more than 2x in FB 4.x comparing to FB 3.x (e.g. 78 vs 31).

• NEW QA: started first usage of configparser package and test_config.ini with shared settings for several tests (currently for only encryption-related).

• NEW QA: one need to have somewhat like GLOBAL_mapping_factory().

• Problems with test for CORE-4337 ("excessive" message about FAILED ATTACH_DATABASE appears in the trace).

• Lot of changes in qa_rundaily.bat (mostly long-waited; done after change test-running machine)

• QA fbtets. WI-V3.0.10.33601, special build: crash when run FBT suite (terminated abnormally (3221225477)), but that was not properly logged (log shows that all tests completed fine). Probably crash was on "bugs.core_3690_dialect_3" OR "bugs.core_3691".

• QA fbtest. QA_rundaily.bat on new host: made benchmark about how performance depends on disks characteristics (including RAM drive).

• QA_rundaily, on new machine: restored full functionality with uploading results to firebirdtest.com

• Presense of %programdata%\fb13_user_mapping prevents from launching two FB instances (4.x and 5.x) at the same time.

• Re-implemented test for CORE-4337.

• NEW QA: problems with obtaining content of firebird.log when it has localized messages.

• Benchmark of INSERTS which are performed differently (requested by Alexey Kovyazin):
    1) LOOP with prepared INSERT statement;
    2) LOOP with prepared EXECUTE BLOCK and INSERT in ES;
    3) INSERT 100000 rows from EXTERNAL table;
    4) BULK_INSERT.

• NEW QA: adapted ~25 tests for running under pytest + firebird-qa plugin.

July 2022

• QA fbtest. Implemented / refactored /adjusted  .fbt for 18 tests.

• NEW QA. Adapted 42 tests for running under pytest + firebird-qa plugin.

• QA fbtest. Problems with re-implementing test for CORE-4337 after recent commits.

• New QA. Continued discussion solution of UnicodeDecode errors when obtaining content of firebird.log which contains localized messages or binary data (garbage).

• Resumed work with batch for making FB snapshots. Problem with absent env variable VS150COMNTOOLS. Decided to add special variable into make.conf in order to create VS150COMNTOOLS explicitly in such cases. Additional MSVC10 required for build FB 3.x.

• Finally disabled tests which evaluates EXP( ) for discovering minimal value that can be distinguish from zero.
  Reason: different results on hardware depending on 1) CPU manufacturer; 2) major FB-family (3.x vs 4.x).
  Example: exp(-744.0346068132731393) vs exp(-745.1332191019410399).

• 3.0.8.33540, Classic, Windows: crash (special build), related to setLockAttachment.

• Re-implemented test for CORE-6097 (connecting to DB with non-ascii user name) because of problems after migrating to new host.

• Test for CORE-4880 (performance-measurement and comparison) became fail for every run after commit that fixed #7222

• QA fbtest. Re-implemented several tests which permanently failed: COREs: 4880, 4337, 5431; few functional/basic/ tests.

• WI-5.0.0573, SS/CS: crash when run test for CORE-6353 (related to int128).

• Windows, FB 5.x: several builds hanged on test "functional.tabloid.max-path-length".

• ISQL for FB 4.x+: command 'help set' does not show 'SET KEEP_TRANS_params'.

• QA-run scenario: it will be better to create several separate jobs and run them with requirement to finish for some (restrict) time scope, with forcible terminating otherwise and gathering debug info in that case (create dump and stack trace of firebird process(es)).

• QA fbtest. Several tests failed on 5.0.0.577, with additional message: "undefined message number" (COREs: 0886, 2032, 6351; GH: 7034; tabloid.eqc.343715). Some tests became fail after years of normal work (core-3094, -3323, -5431).

• LI-V3.0.10.33587, SS, experimental build: crash.

• Connections that are created by 'gfix -sweep' and 'gstat -r' can not be handled by DB-level triggers.

• Eventually found how to reproduce bug described in core-4337, test was again totally re-implemented.

• Adjusted test bugs/core_5488_session_idle.fbt after discussion with Vlad.

• Again fully re-implemented test CORE-3323 (it often failed on Windows). Reproduced bug on 2.5.0, checked on 2.5.1 and later.

• Discussed with FB team implementation of test for #6802.

• Discussed with FB team ways how to develope tests for both old and new qa (+ estimating time).

• Test for gh-6947 leads Python.exe to crash (Windows only). Problem appeared since migration to new host (from Win-2008 Server R2 to Windows-10 workstation). Crash occurred only when FB worked in embedded mode.

• QA fbtest. Windows, 3.0.11.33609 SS: fbt_run hanged on test for CORE-5948.

• Resumed work on test for CORE-6071 (make it work also on Linux). Changes in plugin.conf required for this test work on Linux (need to create section with name of plugin in UPPERCASE form).

• QA fbtest. Additional work with batch that makes standard and custom FB snapshots. Problems appeared after migrating to new host.

• Attempt to connect to DB in shutdown state issues different message if we use XNET protocol: "SQLSTATE = 08006 connection lost to database" (instead of "SQLSTATE = HY000 database shutdown"). Created ticket #7239.

• Additional investigation related to slow inserts when DB is in delta state (Alexey Kovyazin request). Initial work started in APR-2022.

• Discussed with dimitr et al possible approaches of implementation of tests which relate to performance measure. It would be nice to have ability to get CPU cycles from mon$ table(s) (like this is shown in ProcessExplorer, "Performance" tab).

• Weird messages in the trace:
  * "ERROR AT JStatement::prepare / 335544395 : table id NNNN is not defined"
  * "335544395 : table id NNNN is not defined" (ERROR AT JStatement::prepare).
  -- was encountered in one of trace logs.

• Resumed discussion of ability to use config with parameters which are 'common' for several / all FB instances. Starting use [DEFAULT] section in firebird-driver.conf

• firebird-driver source: question about DbInfoCode.ALLOCATION vs DbInfoCode.DB_SIZE_IN_PAGES

• Refactored auxiliary script for parsing trace log: moved some settings into config, added [optional] parsing of FB sources in order to obtain full list of gdscodes, their mnemonas and descriptions.

• Further fefactoring batch that builds FB snapshots (Windows-10).

• OLTP-EMUL: crash of LI-V3.0.11.33608. Sent report FB team (Alex reply: crash caused by EDS::Transaction::jrdTransactionEnd)

• It would be useful if Services API will return paths to 1) dir_plugins ; 2) dir_sample (both these macroses are mentioned in the firebird.conf).

• NEW QA: Problem with run tests when FB in Classic mode: samle_qa = _vars_['sample_dir'] / 'qa' ==> INTERNALERROR> KeyError: 'sample_dir'.

• FB-4.x snapshot build process failed on Windows ("NMAKE : fatal error U1073: don't know how to make '\extern\cloop\src\cloop\*.cpp'").

• Implemented aux batch that will create dumps and stack traces for all running FB processes (actual for CS).

• NEW QA. Discussed current activities and state of work. About 50 tests needed to adapt from old test suite.
  Resumed discussion ability to launch trace using NON-sysdba user/password/role (which has apropriate privilege).

• NEW QA. Tests hang (pytest does not return control to OS) if we use wrong password when start it.

• NEW QA. Discussed ability to work with firebird.log which contain lot of garbage (binary data). Related to gh_6785_test.py.

• Nulls in rdb$procedures.rdb$debug_info lead to missed number of line and column when error raises in such PSQL object (and the same for rdb$triggers).