Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c9c8add
First variant of depends tracking - works only for procedures
AlexPeshkoff Aug 14, 2026
634f597
When object did change same message format does not guarantee correct…
AlexPeshkoff Aug 15, 2026
79a216c
Merge branch 'master' into depend_update
AlexPeshkoff Sep 19, 2026
9642f18
Added check for functions, triggers and computed fields
AlexPeshkoff Sep 21, 2026
5d80b88
increment build number
actions-user Sep 19, 2026
5625c39
Merge commit from fork
asfernandes Sep 20, 2026
f07747e
increment build number
actions-user Sep 20, 2026
88871cc
Add missing filter
asfernandes Sep 21, 2026
b661d2b
increment build number
actions-user Sep 21, 2026
887639d
Postfix: Decrement blr length after reading value from blr in `parse_…
TreeHunter9 Sep 22, 2026
762d0a0
increment build number
actions-user Sep 22, 2026
0eb5c83
Fix #9156: explicit PLAN rejected with 'index ... cannot be used in t…
dyemanov Sep 22, 2026
f2b45c2
increment build number
actions-user Sep 23, 2026
9490172
Update tzdata to version 2026d. (#9163)
github-actions[bot] Sep 24, 2026
4e3513c
Fix #9151: 'no current record for fetch operation' on a LEFT JOIN to …
dyemanov Sep 23, 2026
04f9698
increment build number
actions-user Sep 24, 2026
4a21978
Disable DDL under user savepoint
AlexPeshkoff Sep 25, 2026
44c9e4c
increment build number
actions-user Sep 25, 2026
3a6212f
Correct the behavior of `PoolAllocator` according to implementation o…
TreeHunter9 Sep 26, 2026
08b41ce
increment build number
actions-user Sep 26, 2026
4b0b477
No matter that deadlock does not happen it will be good idea to unloc…
AlexPeshkoff Sep 27, 2026
4f59e76
increment build number
actions-user Sep 27, 2026
3ae1cd8
Feature #9154 - gbak FAST_PATH mode (#9155)
asfernandes Sep 28, 2026
3ff4d8a
Fixed segfault in bugs/gh_8968_test.py
AlexPeshkoff Sep 28, 2026
9011c81
increment build number
actions-user Sep 29, 2026
5ab496e
New feature non-decimal integer literals (SQL:2023 T661) and undersco…
ChudaykinAlex Sep 29, 2026
3d07cc6
increment build number
actions-user Sep 29, 2026
5c474d2
WIP
dyemanov Sep 25, 2026
2a3e6bf
Fix #9173: Wrong result: partial index WHERE col IS NOT NULL is appli…
dyemanov Sep 30, 2026
28ff9db
Fixed regression caused by commit acf041e50264c0ab4e53aebbfd1616fc1bf…
AlexPeshkoff Sep 30, 2026
04b57d3
Work in progress
AlexPeshkoff Oct 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions builds/posix/make.shared.variables
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,10 @@ GFIX_Objects:= $(Svc_GFIX_Objects) $(call dirObjects,alice/main)
AllObjects += $(GFIX_Objects)

# gbak
Svc_GBAK_Objects:= $(call dirObjects,burp)
GBAK_Objects:= $(Svc_GBAK_Objects) $(call dirObjects,burp/main)
Svc_GBAK_Objects:= $(call dirObjects,burp) $(call dirObjects,burp/engine)
GBAK_Objects:= $(call dirObjects,burp) $(call dirObjects,burp/main)

AllObjects += $(GBAK_Objects)
AllObjects += $(GBAK_Objects) $(call dirObjects,burp/engine)

# gsec
Svc_GSEC_Objects:= $(call dirObjects,utilities/gsec)
Expand Down
5 changes: 5 additions & 0 deletions builds/win32/msvc15/burp.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@
<ClCompile Include="..\..\..\src\burp\burp.cpp" />
<ClCompile Include="..\..\..\src\burp\BurpTasks.cpp" />
<ClCompile Include="..\..\..\src\burp\canonical.cpp" />
<ClCompile Include="..\..\..\src\burp\RestoreMessageLayout.cpp" />
<ClCompile Include="..\..\..\src\burp\misc.cpp" />
<ClCompile Include="..\..\..\src\burp\mvol.cpp" />
<ClCompile Include="..\..\..\gen\burp\backup.cpp" />
Expand All @@ -211,10 +212,14 @@
<ClInclude Include="..\..\..\src\burp\burp_proto.h" />
<ClInclude Include="..\..\..\src\burp\burpswi.h" />
<ClInclude Include="..\..\..\src\burp\canon_proto.h" />
<ClInclude Include="..\..\..\src\burp\FastPathBurpProvider.h" />
<ClInclude Include="..\..\..\src\burp\FastPathRecordReader.h" />
<ClInclude Include="..\..\..\src\burp\FastPathRecordWriter.h" />
<ClInclude Include="..\..\..\src\burp\misc_proto.h" />
<ClInclude Include="..\..\..\src\burp\mvol_proto.h" />
<ClInclude Include="..\..\..\src\burp\OdsDetection.h" />
<ClInclude Include="..\..\..\src\burp\resto_proto.h" />
<ClInclude Include="..\..\..\src\burp\RestoreMessageLayout.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
Expand Down
17 changes: 16 additions & 1 deletion builds/win32/msvc15/burp.vcxproj.filters
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@
<ClCompile Include="..\..\..\src\burp\canonical.cpp">
<Filter>BURP files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\RestoreMessageLayout.cpp">
<Filter>BURP files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\misc.cpp">
<Filter>BURP files</Filter>
</ClCompile>
Expand Down Expand Up @@ -70,6 +73,15 @@
<ClInclude Include="..\..\..\src\burp\canon_proto.h">
<Filter>Header files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\burp\FastPathBurpProvider.h">
<Filter>Header files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\burp\FastPathRecordReader.h">
<Filter>Header files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\burp\FastPathRecordWriter.h">
<Filter>Header files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\burp\misc_proto.h">
<Filter>Header files</Filter>
</ClInclude>
Expand All @@ -82,8 +94,11 @@
<ClInclude Include="..\..\..\src\burp\resto_proto.h">
<Filter>Header files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\burp\RestoreMessageLayout.h">
<Filter>Header files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\burp\BurpTasks.h">
<Filter>Header files</Filter>
</ClInclude>
</ItemGroup>
</Project>
</Project>
3 changes: 3 additions & 0 deletions builds/win32/msvc15/engine_static.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,9 @@
<ClCompile Include="..\..\..\src\utilities\gsec\gsec.cpp" />
<ClCompile Include="..\..\..\src\utilities\gstat\ppg.cpp" />
<ClCompile Include="..\..\..\src\utilities\nbackup\nbackup.cpp" />
<ClCompile Include="..\..\..\src\burp\engine\FastPathBurpUtil.cpp" />
<ClCompile Include="..\..\..\src\burp\engine\FastPathRecordReader.cpp" />
<ClCompile Include="..\..\..\src\burp\engine\FastPathRecordWriter.cpp" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\src\dsql\AggNodes.h" />
Expand Down
9 changes: 9 additions & 0 deletions builds/win32/msvc15/engine_static.vcxproj.filters
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,15 @@
<ClCompile Include="..\..\..\src\utilities\nbackup\nbackup.cpp">
<Filter>Services</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\engine\FastPathBurpUtil.cpp">
<Filter>Services</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\engine\FastPathRecordReader.cpp">
<Filter>Services</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\engine\FastPathRecordWriter.cpp">
<Filter>Services</Filter>
</ClCompile>
<ClCompile Include="..\..\..\gen\jrd\scl.cpp">
<Filter>JRD files\GPRE cpp</Filter>
</ClCompile>
Expand Down
3 changes: 3 additions & 0 deletions builds/win32/msvc15/engine_test.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,9 @@
<ItemGroup>
<ClCompile Include="..\..\..\src\jrd\tests\CompressorTest.cpp" />
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\src\jrd\tests\EventManagerTest.cpp" />
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\src\jrd\tests\EngineTest.cpp" />
</ItemGroup>
Expand Down
3 changes: 3 additions & 0 deletions builds/win32/msvc15/engine_test.vcxproj.filters
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@
<ClCompile Include="..\..\..\src\jrd\tests\CompressorTest.cpp">
<Filter>source</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\jrd\tests\EventManagerTest.cpp">
<Filter>source</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\jrd\tests\EngineTest.cpp">
<Filter>source</Filter>
</ClCompile>
Expand Down
3 changes: 3 additions & 0 deletions builds/win32/msvc15/gbak.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,9 @@
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\..\src\burp\main\burpMain.cpp" />
<ClCompile Include="..\..\..\src\burp\main\FastPathBurpProviderStub.cpp" />
<ClCompile Include="..\..\..\src\burp\main\FastPathRecordWriterStub.cpp" />
<ClCompile Include="..\..\..\src\burp\main\FastPathRecordReaderStub.cpp" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
Expand Down
11 changes: 10 additions & 1 deletion builds/win32/msvc15/gbak.vcxproj.filters
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,14 @@
<ClCompile Include="..\..\..\src\burp\main\burpMain.cpp">
<Filter>BURP files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\main\FastPathBurpProviderStub.cpp">
<Filter>BURP files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\main\FastPathRecordWriterStub.cpp">
<Filter>BURP files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\burp\main\FastPathRecordReaderStub.cpp">
<Filter>BURP files</Filter>
</ClCompile>
</ItemGroup>
</Project>
</Project>
156 changes: 156 additions & 0 deletions doc/README.gbak_fast_path.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# gbak FAST_PATH mode

`FAST_PATH` is an optional, non-default backup and restore mode available through the Services Manager. It is intended to make
data-heavy `gbak` operations faster by moving table data directly between the `gbak` service and the database engine internal layers,
with less per-record processing.

The backup format is unchanged. A backup created with `FAST_PATH` can be restored normally, and an ordinary backup can be
restored with `FAST_PATH`. Database creation and metadata restore continue to use the normal restore process.

## When to use it

### Speeding up large operations

`FAST_PATH` is most useful for large backups or restores where moving and processing many rows is significant. It can
reduce CPU use and elapsed time, but the improvement depends on the database, data types, storage, and number of
parallel workers. Normal restore already has batch and bulk paths, so measure `FAST_PATH` against the current normal
restore with a representative backup or restore before making it the default for all jobs.

The mode is particularly suitable for data-heavy operations on persistent user tables. For restore, use a trusted backup
and plan appropriate post-restore data-quality checks, because the direct data path bypasses per-row validation performed
by some normal restore paths.

### Recovering data when normal restore fails

`FAST_PATH` restore can also be worth trying when a normal restore fails and the goal is to salvage the data. This works
only for a specific class of failures: normal restore rejects rows through its statement machinery — per-row `NOT NULL`
and domain validation expressions, and request/BLR compilation for the statement path. A backup whose rows no longer
satisfy the currently defined rules (for example, taken before a `NOT NULL` or domain constraint was added, or loaded
while enforcement was disabled) can therefore abort a normal restore while the stored values themselves are intact.

`FAST_PATH` skips that validation pass and the statement machinery: it converts each backed-up message directly into an
engine record with the same conversion routines normal storage uses, then buffers it through bulk insert. Rows rejected
only by validation can load this way.

Limits to keep in mind: the direct path still checks message lengths and field mappings and still converts data types,
and it is fail-fast — a conversion or engine error aborts the restore rather than skipping the row. It will not rescue
a corrupt or truncated backup stream, missing tables or columns, or unreadable blobs. Treat the recovered database as
containing potentially rule-violating rows: validation metadata itself is restored normally, so later DML is still
checked, but the loaded rows are not retrospectively validated. Inspect the data, fix or remove offending rows, and run
explicit post-restore checks before putting the database into production.

## Restore semantics

`FAST_PATH` changes the table-data transfer path, not the backup format or the normal metadata restore process. For eligible
persistent user tables, restore converts backed-up values directly into engine records (using the same conversion routines as
normal record storage) and buffers them through the engine's bulk-insert mechanism, bypassing normal restore statement
execution and its per-row validation processing.

This must not be confused with replaying application `INSERT` statements. Ordinary `gbak` restore also restores explicit
stored values rather than regenerating column defaults or identity values, and does not replay user DML trigger side effects.
Table `CHECK` constraints implemented through triggers are not a per-row validation guarantee of ordinary restore either.

The validation difference depends on the normal restore path being compared. Statement-based stores can evaluate field
validation expressions; the normal bulk-insert path has an explicit `NOT NULL` check but not the general store
validation-expression pass. `FAST_PATH` bypasses that `NOT NULL` check as well: the direct writer does not evaluate per-row
`NOT NULL` or domain validation expressions.

Unless affected by other restore options, validation metadata is restored normally. Retaining that metadata does not
retrospectively validate rows loaded through the direct writer.

Do not treat either restore mode as a complete audit of application data rules. If additional assurance is required, run
explicit post-restore checks for the relevant domain rules, table constraints, and application invariants. Ordinary restore
is not a way to replay trigger side effects.

The direct path still checks message lengths and field mappings and performs required data-type conversions; it is not a
complete backup-integrity validator.

`NO_VALIDITY` (`-no_validity`) is separate from `FAST_PATH`. It omits restored column and domain nullability flags and domain
validation BLR and source; it is not merely a temporary instruction to skip checks during data loading. Those omitted rules are
consequently unavailable for later enforcement unless recreated. `FAST_PATH` alone does not remove those definitions. Conversely,
`NO_VALIDITY` is not a blanket removal of all constraints: table `CHECK` trigger metadata and index-backed primary-key, unique,
and foreign-key constraints follow their normal restore paths.

## Usage

`FAST_PATH` requires the `-service` option (Services Manager). It works both via `gbak -service ... -fast` (or `-fast_path`) and via
`fbsvcmgr` / Services API options.

With `fbsvcmgr`, use `bkp_fast_path` for backup and `res_fast_path` for restore. Paths in these examples are server-side.
For example:

```text
fbsvcmgr localhost:service_mgr \
-user SYSDBA -password <password> \
action_backup dbname /path/source.fdb \
bkp_file /path/source.fbk bkp_fast_path
```

```text
fbsvcmgr localhost:service_mgr \
-user SYSDBA -password <password> \
action_restore bkp_file /path/source.fbk \
dbname /path/restored.fdb res_fast_path
```

The same mode is available through `gbak -service`, which forwards its switches via the service command line:

```text
gbak -service localhost:service_mgr -user SYSDBA -password <password> \
-backup /path/source.fdb /path/source.fbk -fast_path
```

```text
gbak -service localhost:service_mgr -user SYSDBA -password <password> \
-create_database /path/source.fbk /path/restored.fdb -fast_path
```

Services API clients set the corresponding bit in the service-start `isc_spb_options` value:

| Operation | `fbsvcmgr` option | Services API option bit |
|-----------|-------------------|-------------------------|
| Backup | `bkp_fast_path` | `isc_spb_bkp_fast_path` (`0x040000`) |
| Restore | `res_fast_path` | `isc_spb_res_fast_path` (`0x040000`; backup and restore option bits live in separate per-action namespaces) |

The option can be combined with the usual backup, restore, and parallel worker options. `FAST_PATH` does not change the normal
table-data selection rules. With `METADATA_ONLY`, backup omits table rows, and restore consumes but does not load any table rows
present in the backup; the direct row-transfer path is therefore not exercised.

## Scope and eligibility

The direct data path targets persistent user tables. Normal `gbak` selection happens before `FAST_PATH` eligibility is tested:
system relations with `RDB$SYSTEM_FLAG = 1` are excluded from the backed-up relation list, views do not have their rows backed up,
and external-table rows are omitted unless external-table conversion is requested. Temporary-table contents are not persistent
database data.

Restore uses the direct data path only for backups in format 8 or newer; tables from older backups are silently loaded through
the normal restore path.

Privileges are the same as for a normal backup: besides the `USE_GBAK_UTILITY` requirement for the attachment, the backup
checks schema `USAGE`, table `SELECT` and column `SELECT` for every table it reads directly.

Because `FAST_PATH` uses the engine provider directly instead of the provider list, database names must be local to the server
(a path or an alias). Names with a remote prefix such as `host:path` or `inet://host/path` are not routed and fail.

## Architecture

`FAST_PATH` selects the in-process engine provider. Direct VIO access needs an engine `JAttachment`, which the
Services Manager yvalve handle is not, so backup workers always use a private engine attachment - including with a
single worker. Those worker transactions join the main backup snapshot with `isc_tpb_at_snapshot_number` (the same
mechanism as `-parallel` workers).
The reader scans records with `VIO_next_record()` and fills the normal backup message layout; downstream backup serialization
remains unchanged.

Restore retains normal metadata processing, backup decoding, and blob and array handling. Eligible table-data workers convert each
message into an engine record and buffer it through the transaction's bulk-insert page images, the same mechanism the normal
restore batch path uses to reserve and fill data pages. The direct path is fail-fast: a conversion or engine error aborts the restore
rather than skipping a row. This is also true when `-incremental` is specified; normal-mode relations retain their existing
incremental error handling. Rows buffered in the bulk-insert images are flushed at table end and at commit. In-memory batches use an aligned
stride distinct from the serialized message length so packed records keep field alignment; the backup format itself is unpadded.

## Verification

`FAST_PATH` is a trade-off for trusted backups. Restore-test backups, check relevant data rules with application-specific queries, and
benchmark representative scalar-heavy, blob-heavy, and array-heavy workloads with one and multiple workers before enabling the option in
production jobs. CPU, elapsed time, and peak memory should all be compared, since the direct path changes allocation and batching behaviour
as well as per-row CPU.
Loading
Loading