Firebird Documentation Index → Firebird 1.5.6 Release Notes → SQL Language Enhancements → Data Manipulation Language (DML) |
Data manipulation language, or DML is the language of query statements, the commands we use to manipulate data, that is, to SELECT FROM, INSERT, UPDATE and DELETE from tables and to EXECUTE PROCEDUREs.
Calls to EXECUTE PROCEDURE ProcName(<Argument-list>) and SELECT <Output-list> FROM ProcName(<Argument-list>) can now accept local variables (in PSQL) and expressions (in DSQL and PSQL) as arguments.
Allow the result of a column to be determined by the outcome of a group of exclusive conditions.
Syntax Pattern
<case expression> ::= <case abbreviation> | <case specification> <case abbreviation> ::= NULLIF <left paren> <value expression> <comma> <value expression> <right paren> | COALESCE <left paren> <value expression> { <comma> <value expression> }... <right paren> <case specification> ::= <simple case> | <searched case> <simple case> ::= CASE <value expression> <simple when clause>... [ <else clause> ] END <searched case> ::= CASE <searched when clause>... [ <else clause> ] END <simple when clause> ::= WHEN <when operand> THEN <result> <searched when clause> ::= WHEN <search condition> THEN <result> <when operand> ::= <value expression> <else clause> ::= ELSE <result> <result> ::= <result expression> | NULL <result expression> ::= <value expression>
Examples
i) simple
SELECT o.ID, o.Description, CASE o.Status WHEN 1 THEN 'confirmed' WHEN 2 THEN 'in production' WHEN 3 THEN 'ready' WHEN 4 THEN 'shipped' ELSE 'unknown status ''' || o.Status || '''' END FROM Orders o;
ii) searched
SELECT o.ID, o.Description, CASE WHEN (o.Status IS NULL) THEN 'new' WHEN (o.Status = 1) THEN 'confirmed' WHEN (o.Status = 3) THEN 'in production' WHEN (o.Status = 4) THEN 'ready' WHEN (o.Status = 5) THEN 'shipped' ELSE 'unknown status ''' || o.Status || '''' END FROM Orders o;
Allows a column value to be calculated by a number of expressions, from which the first expression to return a non-NULL value is returned as the output value.
Syntax Pattern
<case abbreviation> ::= | COALESCE <left paren> <value expression> { <comma> <value expression> }... <right paren>
COALESCE (V1, V2) is equivalent to the following <case specification>:
CASE WHEN V1 IS NOT NULL THEN V1 ELSE V2 END
COALESCE (V1, V2,..., Vn), for n >= 3, is equivalent to the following <case specification>:
CASE WHEN V1 IS NOT NULL THEN V1 ELSE COALESCE (V2,...,Vn) END
Examples
SELECT PROJ_NAME AS Projectname, COALESCE(e.FULL_NAME,'[< not assigned >]') AS Employeename FROM PROJECT p LEFT JOIN EMPLOYEE e ON (e.EMP_NO = p.TEAM_LEADER); SELECT COALESCE(Phone,MobilePhone,'Unknown') AS "Phonenumber" FROM Relations;
Returns NULL for a sub-expression if it has a specific value, otherwise returns the value of the sub-expression.
Syntax Pattern
<case abbreviation> ::= NULLIF <left paren> <value expression> <comma> <value expression> <right paren>
Syntax Rules
NULLIF (V1, V2) is equivalent to the following <case specification>:
CASE WHEN V1 = V2 THEN NULL ELSE V1 END
Example
UPDATE PRODUCTS SET STOCK = NULLIF(STOCK,0)
User savepoints (alternative name nested transactions) provide a convenient method to handle business logic errors without needing to roll back the transaction. Available only in DSQL.
Use the SAVEPOINT statement to identify a point in a transaction to which you can later roll back.
Syntax Patterns
SAVEPOINT <identifier>;
<identifier> specifies the name of a savepoint to be created. After a savepoint has been created, you can either continue processing, commit your work, roll back the entire transaction, or roll back to the savepoint.
Savepoint names must be distinct within a given transaction. If you create a second savepoint with the same identifer as an earlier savepoint, the earlier savepoint is erased.
ROLLBACK [WORK] TO [SAVEPOINT] <identifier>;
This statement performs the following operations:
Rolls back changes performed in the transaction after the savepoint
Erases all savepoints created after that savepoint. The named savepoint is retained, so you can roll back to the same savepoint multiple times. Prior savepoints are also retained.
Releases all implicit and explicit record locks acquired since the savepoint. Other transactions that have requested access to rows locked after the savepoint must continue to wait until the transaction is committed or rolled back. Other transactions that have not already requested the rows can request and access the rows immediately.
This behaviour may change in future product versions.
The Savepoint undo log may consume significant amounts of server memory, especially if you update the same records in the same transaction multiple times. Use the RELEASE SAVEPOINT statement to release system resources consumed by savepoint maintenance.
RELEASE SAVEPOINT <identifier> [ONLY];
RELEASE SAVEPOINT statement erases the savepoint <identifer> from the transaction context. Unless you specify the ONLY keyword, all savepoints established since the savepoint <identifier> are erased too.
Example using Savepoints
create table test (id integer); commit; insert into test values (1); commit; insert into test values (2); savepoint y; delete from test; select * from test; -- returns no rows rollback to y; select * from test; -- returns two rows rollback; select * from test; -- returns one row
By default, the engine uses an automatic transaction-level system savepoint to perform transaction rollback. When you issue a ROLLBACK statement, all changes performed in this transaction are backed out via a transaction-level savepoint and the transaction is then committed. This logic reduces the amount of garbage collection caused by rolled back transactions.
When the volume of changes performed under a transaction-level savepoint is getting large (10^4-10^6 records affected) the engine releases the transaction-level savepoint and uses the TIP mechanism to roll back the transaction if needed.
If you expect the volume of changes in your transaction to be large, you can use the
TPB flag isc_tpb_no_auto_undo
to avoid the transaction-level savepoint being
created.
Implementing user savepoints in PSQL layer would break the atomicity rule for statements, including procedure call statements. Firebird provides exception handling in PSQL to undo changes performed in stored procedures and triggers. Each SQL/PSQL statement is executed under a system of automatic, internal savepoints, whereby either the entire statement will complete successfully or ALL its changes are rolled back and an exception is raised.
Each PSQL exception handling block is also bounded by automatic system savepoints.
The addition of the optional WITH LOCK clause provides a limited explicit pessimistic locking capability for cautious use in conditions where the affected row set is
extremely small (ideally, a singleton) AND
precisely controlled by the application code.
This is for experts only!
The need for a pessimistic lock in Firebird is very rare indeed and should be well understood before use of this extension is considered.
It is essential to understand the effects of transaction isolation and other transaction attributes before attempting to implement explicit locking in your application.
Syntax Pattern
SELECT ... FROM <sometable> [WHERE ...] [FOR UPDATE [OF ...]] [WITH LOCK] ...;
If the WITH LOCK clause succeeds, it will secure a lock on the selected rows and prevent any other transaction from obtaining write access to any of those rows, or their dependants, until your transaction ends.
If the FOR UPDATE clause is included, the lock will be applied to each row, one by one, as it is fetched into the server-side row cache. It becomes possible, then, that a lock which appeared to succeed when requested will nevertheless fail subsequently, when an attempt is made to fetch a row which becomes locked by another transaction.
The SELECT... WITH LOCK construct is available in DSQL and PSQL.
The SELECT... WITH LOCK construct can succeed only in a top-level, single-table SELECT statement. It is not available
in a subquery specification
for joined sets
with the DISTINCT operator, a GROUP BY clause or any other aggregating operation
with a view
with the output of a selectable stored procedure
with an external table
As the engine considers, in turn, each record falling under an explicit lock statement, it returns either the record version that is the most currently committed, regardless of database state when the statement was submitted, or an exception.
Wait behaviour and conflict reporting depend on the transaction parameters specified in the TPB block:
Table 4.1. How TPB settings affect explicit locking
TPB mode | Behaviour |
---|---|
isc_tpb_consistency |
Explicit locks are overridden by implicit or explicit table-level locks and are ignored |
isc_tpb_concurrency + isc_tpb_nowait |
If a record is modified by any transaction that was committed since the transaction attempting to get explicit lock started, or an active transaction has performed a modification of this record, an update conflict exception is raised immediately |
isc_tpb_concurrency + isc_tpb_wait |
If the record is modified by any transaction that has committed since the transaction attempting to get explicit lock started, an update conflict exception is raised immediately. If an active transaction is holding ownership on this record (via explicit locking or by a normal optimistic write-lock) the transaction attempting the explicit lock waits for the outcome of the blocking transaction and, when it finishes, attempts to get the lock on the record again. This means that, if the blocking transaction committed a modified version of this record, an update conflict exception will be raised. |
isc_tpb_read_committed + isc_tpb_nowait |
If there is an active transaction holding ownership on this record (via explicit locking or normal update), an update conflict exception is raised immediately. |
isc_tpb_read_committed + isc_tpb_wait |
If there is an active transaction holding ownership on this record (via explicit locking or by a normal optimistic write-lock), the transaction attempting the explicit lock waits for the outcome of blocking transation and when it finishes, attempts to get the lock on the record again. Update conflict exceptions can never be raised by an explicit lock statement in this TPB mode. |
When an UPDATE statement tries to access a record that is locked by another transaction, it either raises an update conflict exception or waits for the locking transaction to finish, depending on TPB mode. Engine behaviour here is the same as if this record had already been modified by the locking transaction.
No special gdscodes are returned from conflicts involving pessimistic locks.
The engine guarantees that all records returned by an explicit lock statement are actually locked and DO meet the search conditions specified in WHERE clause, as long as the search conditions do not depend on any other tables, via joins, subqueries, etc. It also guarantees that rows not meeting the search conditions will not be locked by the statement. It can NOT guarantee that there are no rows which, though meeting the search conditions, are not locked.
This situation can arise if other, parallel transactions commit their changes during the course of the locking statement's execution.
The engine locks rows at fetch time. This has important consequences if you lock several rows at once. Many access methods for Firebird databases default to fetching output in packets of a few hundred rows ("buffered fetches"). Most data access components cannot bring you the rows contained in the last-fetched packet, where an error occurred.
The FOR UPDATE clause provides a technique to prevent usage of buffered fetches, optionally with the
OF <column-names>
to enable positioned updates.
Alternatively, it may be possible in your access components to set the size of the fetch buffer to 1. This would enable you to process the currently-locked row before the next is fetched and locked, or to handle errors without rolling back your transaction.
Rolling back of an implicit or explicit savepoint releases record locks that were taken under that savepoint, but it doesn't notify waiting transactions. Applications should not depend on this behaviour as it may get changed in the future.
While explicit locks can be used to prevent and/or handle unusual update conflict errors, the volume of deadlock errors will grow unless you design your locking strategy carefully and control it rigorously.
Most applications do not need explicit locks at all. The main purposes of explicit locks are (1) to prevent expensive handling of update conflict errors in heavily loaded applications and (2) to maintain integrity of objects mapped to a relational database in a clustered environment. If your use of explicit locking doesn't fall in one of these two categories, then it's the wrong way to do the task in Firebird.
Explicit locking is an advanced feature, do not misuse it ! While solutions for these kinds of problems may be very important for web sites handling thousands of concurrent writers, or for ERP/CRM systems operating in large corporations, most application programs do not need to work in such conditions.
Originally, grouped sets could be grouped only on named columns. In Firebird 1.0, it became possible to group by a UDF expression. In 1.5, several further extensions to the handling of aggregate functions and the GROUP BY clause now allow groupings to be made by the degree of columns in the output specification (their 1-based "ordinal left-to-right position", as in the ORDER BY clause) or by a variety of expressions.
Not all expressions are currently allowed inside the GROUP BY list. For example, concatenation is not allowed.
Group By syntax
SELECT ... FROM .... [GROUP BY group_by_list] group_by_list : group_by_item [, group_by_list]; group_by_item : column_name | degree (ordinal) | udf | group_by_function; group_by_function : numeric_value_function | string_value_function | case_expression ; numeric_value_function : EXTRACT '(' timestamp_part FROM value ')'; string_value_function : SUBSTRING '(' value FROM pos_short_integer ')' | SUBSTRING '(' value FROM pos_short_integer FOR nonneg_short_integer ')' | KW_UPPER '(' value ')' ;
The group_by_item cannot be a reference to any aggregate-function (including any that are buried inside an expression) from the same context.
The having clause only allows aggregate functions or valid expressions that are part of the GROUP BY clause. Previously it was allowed to use columns that were not part of the GROUP BY clause and to use non-valid expressions.
When the context is an aggregate statement, the ORDER BY clause only allows valid expressions that are aggregate functions or expression parts of the GROUP BY clause.
Previously it was allowed to use non-valid expressions.
It is now possible to use an aggregate function or expression contained in the GROUP BY clause inside a subquery.
Examples
SELECT r.RDB$RELATION_NAME, MAX(r.RDB$FIELD_POSITION), (SELECT r2.RDB$FIELD_NAME FROM RDB$RELATION_FIELDS r2 WHERE r2.RDB$RELATION_NAME = r.RDB$RELATION_NAME and r2.RDB$FIELD_POSITION = MAX(r.RDB$FIELD_POSITION)) FROM RDB$RELATION_FIELDS r GROUP BY 1 /* ************ */ SELECT rf.RDB$RELATION_NAME AS "Relationname", (SELECT r.RDB$RELATION_ID FROM RDB$RELATIONS r WHERE r.RDB$RELATION_NAME = rf.RDB$RELATION_NAME) AS "ID", COUNT(*) AS "Fields" FROM RDB$RELATION_FIELDS rf GROUP BY rf.RDB$RELATION_NAME
Aggregate functions from different contexts can be used inside an expression.
Examples
SELECT r.RDB$RELATION_NAME, MAX(i.RDB$STATISTICS) AS "Max1", (SELECT COUNT(*) || ' - ' || MAX(i.RDB$STATISTICS) FROM RDB$RELATION_FIELDS rf WHERE rf.RDB$RELATION_NAME = r.RDB$RELATION_NAME) AS "Max2" FROM RDB$RELATIONS r JOIN RDB$INDICES i on (i.RDB$RELATION_NAME = r.RDB$RELATION_NAME) GROUP BY r.RDB$RELATION_NAME HAVING MIN(i.RDB$STATISTICS) <> MAX(i.RDB$STATISTICS)
Note! This query gives results in FB1.0, but they are WRONG!
Using a singleton SELECT expression inside an aggregate function is supported.
Example
SELECT r.RDB$RELATION_NAME, SUM((SELECT COUNT(*) FROM RDB$RELATION_FIELDS rf WHERE rf.RDB$RELATION_NAME = r.RDB$RELATION_NAME)) FROM RDB$RELATIONS r JOIN RDB$INDICES i on (i.RDB$RELATION_NAME = r.RDB$RELATION_NAME) GROUP BY r.RDB$RELATION_NAME
Using an aggregate function inside another aggregate function is possible if the inner aggregate function is from a lower context (see example above).
The ORDER BY clause lets you specify any valid expressions to sort query results. If the expression consists of a single number, it is interpreted as column (degree) number, as previously.
The ordering of nulls in the result set can be controlled using a nulls placement clause.
Results can be sorted so that nulls are placed either above (NULLS FIRST) or below (NULLS LAST) the sorted non-nulls.
Behaviour when nulls placement is unspecified is NULLS LAST.
Syntax Pattern
SELECT ... FROM .... [ORDER BY order_list]....; order_list : order_item [, order_list]; order_item : <expression> [order_direction] [nulls_placement] order_direction : ASC | DESC; nulls_placement : NULLS FIRST | NULLS LAST;
Restrictions
If NULLS FIRST is specified, no index will be used for sorting.
The results of a sort based on values returned from a UDF or a stored procedure will be unpredictable if the values returned cannot be used to determine a logical sorting sequence.
The number of procedure invocations from specifying a sort based on a UDF or stored procedure will be unpredictable, regardless of whether the ordering is specified by the expression itself or by an ordinal number representing an expression in the column-list specification.
An ordering clause for sorting the output of a union query may use only ordinal (degree) numbers to refer to the ordering columns.
i)
SELECT * FROM MSG ORDER BY PROCESS_TIME DESC NULLS FIRST
ii)
SELECT FIRST 10 * FROM DOCUMENT ORDER BY STRLEN(DESCRIPTION) DESC
iii)
SELECT DOC_NUMBER, DOC_DATE FROM PAYORDER UNION ALL SELECT DOC_NUMBER, DOC_DATA FROM BUDGORDER ORDER BY 2 DESC NULLS LAST, 1 ASC NULLS FIRST
In Firebird 1.5, zero can be accepted as an argument for FIRST in the SELECT FIRST m ..
SKIP n
construction. An empty result set will be returned.
(from the v.1.0 release notes)
Syntax Pattern
SELECT [FIRST (<integer expr m>)] [SKIP (<integer expr n>)]
Retrieves the first m rows of the selected output set. The optional SKIP clause will cause the first n rows to be discarded and return an output set of m rows starting at n + 1. In the simplest form, m and n are integers but any Firebird expression that evaluates to an integer is valid.
Available in SQL and DSQL except where otherwise indicated.
Parentheses are required for expression arguments and are optional otherwise.
The arguments can also bind variables, e.g.
SKIP ? * FROM ATABLE
returns the remaining dataset after discarding the n rows at the top, where n is passed in the "?" variable.
SELECT FIRST ? COLUMNA, COLUMNB FROM ATABLE
returns the first m rows and discards the rest.
An identifier that evaluates to an integer may also be used in GDML, although not in SQL or DSQL.
The FIRST clause is also optional, i.e. you can include SKIP in a statement without FIRST to get an output set that simply excludes the rows appointed to SKIP.
Example
SELECT SKIP (5+3*5) * FROM MYTABLE; SELECT FIRST (4-2) SKIP ? * FROM MYTABLE; SELECT FIRST 5 DISTINCT FIELD FROM MYTABLE;
1. This:
delete from TAB1 where PK1 in (select first 10 PK1 from TAB1);
will delete all of the rows in the table. Ouch! the sub-select is evaluating each 10 candidate rows for deletion, deleting them, slipping forward 10 more...ad infinitum, until there are no rows left. Beware!
2. Queries like:
... WHERE F1 IN ( SELECT FIRST 5 F2 FROM TABLE2 ORDER BY 1 DESC )
won't work as expected, because the optimization performed by the engine transforms the correlated
WHERE...IN (SELECT...)
predicate to a correlated EXISTS predicate. It's obvious
that in this case FIRST N doesn't make any sense:
... WHERE EXISTS ( SELECT FIRST 5 TABLE2.F2 FROM TABLE2 WHERE TABLE2.F2 = TABLE1.F1 ORDER BY 1 DESC )
The remaining items in this section are DML enhancements that were introduced in Firebird 1.0.x, described again for the reader's convenience.
It is now possible to aggregate a SELECT by grouping on the output of a UDF. e.g.
select strlen(rtrim(rdb$relation_name)), count(*) from rdb$relations group by strlen(rtrim(rdb$relation_name)) order by 2
A side-effect of the changes enabling grouping by UDFs is that, whereas previously you could not call built-in Firebird functions in GROUP BY, now, by creating a dummy UDF wrapper, you can do:
select count(*) from rdb$relations r group by bin_or((select count(rdb$field_name) from rdb$relation_fields f where f.rdb$relation_name = r.rdb$relation_name),1)
Internal function, available in SQL and DSQL, implementing the ANSI SQL SUBSTRING() function. It
will return a stream consisting of the byte at <pos> and all subsequent bytes up to the end of the
string. If the optional FOR <length>
is specified, it will return the lesser
of <length> bytes or the number of bytes up to the end of the input stream.
The first argument can be any expression, constant or identifier that evaluates to a string.
<pos> must evaluate to an integer. <pos> in the string starts at 1, like other SQL positional elements.
Neither <pos> nor <length> can be query parameters: they must evaluate to constants.
Because <pos> and <length> are byte positions, the identifier of the input string can be a binary blob, or a sub_type 1 text blob with an underlying one-byte-per-character charset. The function currently does not handle text blobs with Chinese (2 byte/char maximum) or Unicode (3 byte/char maximum) character sets.
For a string argument (as opposed to a blob), the function will tackle ANY charset.
Example
UPDATE ATABLE SET COLUMNB = SUBSTRING(COLUMNB FROM 4 FOR 99) WHERE ...
Please refer also to the later section on External Functions (UDFs) for details of changes and additions to external substring functions in the standard UDF library.
Firebird Documentation Index → Firebird 1.5.6 Release Notes → SQL Language Enhancements → Data Manipulation Language (DML) |