Skip to main content
Version: 1.6

Release Notes 1.6

Your backups are data sources — and they are fast. FastBCP 1.6 reads tables straight from Oracle RMAN backup sets, full and incremental, encrypted SQL Server backups and SQL Server full and differential backups, always at their latest version, exports SQL Server backups up to 2.8× faster, and opens a backup it has already read in a few milliseconds. No restore, no database instance, no extra disk space.

0
Oracle instance needed

Point FastBCP at the directory of the RMAN backup pieces. No database to restore, no credentials.

2.8×
faster SQL Server backup exports

150 million rows from a compressed, striped backup to Parquet in 12 seconds.

~20 ms
to open a backup again

The next exports from the same SQL Server or Oracle backup start right away.

3×
faster in scheduled tasks

On Windows, a run without a window now gets the full speed of the CPU.

What's New: At a Glance​

Since
New source--connectiontype backup_rman: export tables from Oracle RMAN backup sets, multi-piece or not, compressed or not1.6.0
Incremental RMAN backupsThe latest version of a table, from a level 0 and its incremental backups, levels 1 to 4, the whole table. Read this before upgrading1.6.8
Differential backupsThe latest version of a table, from a full backup and its differential, the whole table. Read this before upgrading1.6.7
Encrypted backupsWITH ENCRYPTION and TDE databases, read with the private key of the certificate, no SQL Server needed1.6.6
SQL Server backupsExports up to 2.8× faster, large striped backups ready in seconds1.6.0 – 1.6.3
Backups opened againIn milliseconds, for SQL Server backups and Oracle RMAN backup sets; FASTBCP_INDEX_DIR for accounts without a home (1.6.6)1.6.3 – 1.6.6
Backup or live databaseThe same Parquet from backup_mssql and adbc_mssql, GeoParquet and float vectors included1.6.5
Columnstore backups7× less memory on a wide clustered columnstore table1.6.5
SQL Server datesThe whole datetime2 range, 9999-12-31 sentinels included, with backup_mssql and adbc_mssql1.6.2
ReliabilityMore SQL Server and Oracle backup tables read, with the right values; a damaged .bak no longer hangs an export; the end of some compressed backups no longer lost; an RMAN set without its last piece no longer read short1.6.0 – 1.6.8
Unsigned 64-bit integersExact values from HFSQL (OLE DB) and MySQL BIGINT UNSIGNED, in every format1.6.4
WindowsFull speed without a window: scheduled tasks, services and orchestrators no longer run on the efficiency cores1.6.0
Doc Wizardbackup_rman in the ADBC connectors; incremental chains in --database (1.6.8)1.6.0 – 1.6.8
LicenseThe ARPE.IO Software License Agreement ships with FastBCP1.6.0

Oracle RMAN Backup Sets — --connectiontype backup_rman​

Need the data of an Oracle database that is not reachable — on a backup server, in an archive, a database decommissioned long ago — or don't want to restore a large backup to read one table? FastBCP now reads RMAN backup sets directly through the embedded ArpeOraBackup ADBC driver.

  • --server is the directory holding the backup pieces.
  • --database names the pieces: names or patterns, separated by ; ("full_*", "full_*;ctl_*").
  • That's it: no user, no password, no Oracle.
.\FastBCP.exe `
--connectiontype backup_rman `
--server "D:\Backups\RMAN\ORCL" `
--database "full_*" `
--sourceschema "SALES" `
--sourcetable "ORDERS" `
--directory "D:\export\orders" `
--fileoutput "orders.parquet" `
--parallelmethod Physical `
--paralleldegree 8

Every RMAN Compression​

FastBCP to Parquet, a 1 M-row table, open of the backup included, Windows laptop (hybrid CPU), degree 8, first export of the backup.

BackupElapsed
Uncompressed0.73 s
AS COMPRESSED BACKUPSET, MEDIUM0.83 s
AS COMPRESSED BACKUPSET, HIGH0.82 s
AS COMPRESSED BACKUPSET, LOW1.04 s
AS COMPRESSED BACKUPSET, BASIC7.0 s
AS COMPRESSED BACKUPSET, HIGH, Oracle 19c6.0 s

The next exports from the same backup skip almost all of the open: see Backups opened again in milliseconds.

What It Reads​

Supported
Oracle versions19c, 21c, 23ai and 26ai, on Linux x86-64
Backup typesFull backup sets of a database, tablespaces, datafiles or PDBs, uncompressed or compressed with BASIC, LOW, MEDIUM or HIGH; incremental backups, levels 0 to 4 (1.6.8)
Multi-piece and multi-set backupsMAXPIECESIZE, several channels, FILESPERSET: every piece, by patterns or a ;-separated list, in any order
Several backupsThe latest version of each datafile is read
Database layoutContainer databases (root and PDBs) and non-CDB, bigfile and smallfile tablespaces
TablesHeap tables, with chained and migrated rows, more than 255 columns
Data typesNumbers, text in the database and national character sets, binary floats, RAW, LONG, LONG RAW, DATE and TIMESTAMP with or without time zone — see the datatypes mapping
Good to know
  • Parquet output only, on Windows x64 and Linux x64.
  • --parallelmethod accepts None and Physical (which resolves to the Pages method).
  • --query accepts SELECT * | col1, col2, … FROM [[container.]owner.]table.
  • Incremental level 1 to 4 backups are read with their level 0 since 1.6.8; from 1.6.1 to 1.6.7 they were refused with a clear message.
  • Refused with a clear message: encrypted backup sets (SET ENCRYPTION); partitioned, index-organized and clustered tables; LOB, INTERVAL, ROWID, JSON, VECTOR and object columns (the other columns of the table can be exported with --query); incomplete multi-piece sets.

See also: Oracle RMAN backup sets · Examples

Incremental Backups: the Latest Version of a Table​

Your RMAN strategy is a level 0 on Sunday and a level 1 every night, and the data you need is last night's, not Sunday's. Since 1.6.8, give FastBCP the level 0 and the incremental backups on it, and it reads the table in its latest version. No Oracle, no restore, no recovery, and always the whole table.

  • --database names the chain: a list in any order, a pattern, or the whole directory. Older backups can match too.
  • The pattern ** reaches the subdirectories of --server: "l0/*;l1/**" for a directory per level and per day ("l0\*;l1\**" on Windows, where / works too), "**" for everything below it.
  • FastBCP checks that each incremental backup belongs to the chain, whatever the files are called. A missing link fails the export, naming the backup needed: no silent hole in your data.
.\FastBCP.exe `
--connectiontype backup_rman `
--server "D:\Backups\RMAN\ORCL" `
--database "l0\*;l1\**" `
--sourceschema "SALES" `
--sourcetable "ORDERS" `
--directory "D:\export\orders" `
--fileoutput "orders.parquet" `
--parallelmethod Physical `
--paralleldegree 8
Supported
Incremental backupsLevels 0 to 4, differential and cumulative, FROM SCN, each with its own compression
LayoutOne directory, a directory per level or per day, anywhere below --server with **
TablesEvery table and data type a full backup reads, in the root, the PDBs or a non-CDB
VersionsBackups of Oracle 19c, 23ai and 26ai
A backup that cannot be usedThe export fails, naming it, and the logs say what can be done about it
SettingsFASTBCP_BACKUP_RESTORE_POINT and FASTBCP_BACKUP_RESTORE_MODE now apply to backup_rman too

Checked against the live tables, row for row, through chains of incremental backups on each of these versions.

Reading through a chain costs little more than reading the level 0 alone. A 20 M-row table (2.4 GB), each level 1 rewriting half its blocks, read with 8 threads, warm cache:

Backups readUncompressedMEDIUMBASIC
Level 0 alone0.27 s0.32 s5.2 s
Level 0 + 1 level 10.31 s——
Level 0 + 3 level 1—0.40 s8.0 s
Level 0 + 6 level 10.34 s——
Before upgrading

Some exports change: a level 0 and its incremental backups, refused up to 1.6.7, are now read; a backup set without its last piece, read short up to 1.6.7, now fails, naming it. FASTBCP_BACKUP_RESTORE_POINT=full reads the level 0 only. See Upgrading to 1.6.8, line by line.

Good to know
  • The redo is not applied: an incremental backup taken while the database is open is read as its blocks are.
  • With BASIC compression, a long chain of level 1 backups costs more to read. A cumulative backup, or a new level 0, keeps it short.
  • After a new incremental backup, only the new backup is opened in full: the level 0 already opened stays fast.
  • Image copies (BACKUP AS COPY, RECOVER COPY strategies) are not read.

SQL Server Backups: Faster, and Read Right​

--connectiontype backup_mssql arrived in 1.5. Across the 1.6 releases it got much faster on large backups, and reads more tables, with the right values.

Up to 2.8× Faster​

TPC-H SF100 database (150 M-row orders table, clustered columnstore), backed up with COMPRESSION and striped over 6 files (36 GB), exported to Parquet with --parallelmethod Physical --paralleldegree 24, Windows laptop.

Export of orders (150,000,000 rows)Elapsed
FastBCP 1.5.1error
FastBCP 1.6.033 s
FastBCP 1.6.3, first export of the backup19.4 s
FastBCP 1.6.3, next exports of the backup11.8 s

Large backups striped over several files also start much sooner since 1.6.0:

Before the first row1.5.11.6.0
Compressed backup, 6 files200 s25 s
Uncompressed backup, 6 files24 s7.4 s

Faster reading of compressed backups and of clustered columnstore tables: internal improvement.

The Whole SQL Server Date Range​

Real tables hold dates far from today: 9999-12-31 as "no end date", or a mistyped year such as 0201 or 9600. SQL Server accepts them all, but a single such datetime2(7) value failed the whole export with date … is out of range.

Since 1.6.2, datetime2(7) and datetimeoffset(7) columns are exported with a microsecond unit by default, with both --connectiontype backup_mssql and adbc_mssql: every date from 0001-01-01 to 9999-12-31 is exported, and a backup and a live export of the same table give the same Parquet schema.

The Same Parquet as adbc_mssql​

Reading a table from its backup should not change what your pipeline receives. Since 1.6.5, a --connectiontype backup_mssql export and an adbc_mssql export of the same table give the same Parquet, values and schema. Spatial columns come out ready to use in GeoParquet readers, and vectors as real lists of floats.

Column typeBefore 1.6.51.6.5
geography, geometrySQL Server binary serializationWKB with GeoParquet metadata and the CRS from the SRID
vector(n)SQL Server binary serializationList of n 32-bit floats
time(7)Nanosecond unitMicrosecond unit
sql_variantTextText, written as with adbc_mssql
warning

If your pipeline reads these columns from backup_mssql exports, check it before upgrading. To keep the SQL Server serialization of spatial columns, see adbc.arpemsbackup.geospatial.

Good to know
  • json and vector(n, float16) columns (SQL Server 2025) are refused with a clear message. Export the other columns by listing them in --query, or use adbc_mssql.
  • FASTBCP_UID_STRING_CASING now applies to backup_mssql too.

Far Less Memory on Columnstore Tables​

Exporting a wide clustered columnstore table from a backup could take more memory than many servers have. Since 1.6.5, memory stays bounded (internal improvement).

Wide clustered columnstore table, --parallelmethod Physical --paralleldegree 8Peak memory
Before 1.6.542 GB
FastBCP 1.6.55.9 GB

Differential Backups: the Latest Version of a Table​

Your backup plan is a full backup on Sunday and a differential every night, and the data you need is last night's, not Sunday's. Since 1.6.7, give FastBCP the full backup and its differential, and it reads the table in its latest version. No SQL Server, no restore, and always the whole table.

  • --database names the files: a list in any order, or a pattern. Older backups can match too.
  • FastBCP checks that the differential was taken from that full backup, whatever the files are called.
.\FastBCP.exe `
--connectiontype backup_mssql `
--server "D:\Backups\MSSQL" `
--database "sales_full.bak;sales_diff.bak" `
--sourceschema "dbo" `
--sourcetable "orders" `
--directory "D:\export\orders" `
--fileoutput "orders.parquet" `
--parallelmethod Physical `
--paralleldegree 8
Supported
BackupsOne file each, WITH NOINIT in one file, striped, compressed or not, encrypted
TablesEvery table and data type a full backup reads, rowstore and clustered columnstore
VersionsBackups of SQL Server 2017, 2019, 2022 and 2025
A backup that cannot be usedThe export fails, naming it, and the logs say what can be done about it

Checked against the live tables, through several forms of differential on each of these versions, and against a RESTORE of the same backups.

Before upgrading

A backup that could be the newest and cannot be used now fails the export, so some exports that worked with 1.6.6 fail, and a WITH NOINIT file with differentials reads fresher rows. FASTBCP_BACKUP_RESTORE_POINT=full goes back to the last full backup. See Upgrading to 1.6.7, line by line.

Good to know
  • Log backups are not applied. A differential taken during writes is read as its pages are, and can give other rows than a RESTORE. For an exact image, back up while the table is quiet.
  • With FASTBCP_BACKUP_RESTORE_MODE=best_effort, the backups skipped are reported on the standard error output only, not in the FastBCP logs. The logs do not name the files read either.
  • Against 1.6.6, the peak memory of a rowstore export is 20 to 25 % lower.
  • Not measured: a differential on a cold disk cache or on a network share.

Encrypted Backups, Read with Their Certificate​

Your backups are encrypted, as they should be, and until now that meant restoring them on a server that holds the certificate before reading a single table. Since 1.6.6, give FastBCP the private key of the certificate, and it reads backups WITH ENCRYPTION and backups of TDE databases directly. No SQL Server, no restore.

  • --sourceuser is the file of the private key.
  • --sourcepassword is its password, if it has one. It is hidden in all logs.
.\FastBCP.exe `
--connectiontype backup_mssql `
--server "D:\Backups\MSSQL" `
--database "sales_encrypted.bak" `
--sourceuser "D:\Secure\Keys\MyCert.pvk" `
--sourcepassword "KeyPassword" `
--sourceschema "dbo" `
--sourcetable "orders" `
--directory "D:\export\orders" `
--fileoutput "orders.parquet" `
--parallelmethod Physical `
--paralleldegree 8
Supported
EncryptionWITH ENCRYPTION, TDE, or both, also under two certificates; AES 128, 192 and 256, Triple DES
BackupsCompressed or not, striped, from SQL Server 2017, 2019, 2022 and 2025
Key files.pvk, .pfx / .p12, PEM, DER, or a .cer with its key next to it
Without the right keyThe export stops, and the message names the certificate by its thumbprint

Checked against the live tables: the Parquet of an encrypted backup equals the adbc_mssql export of the same table, on Windows and on Linux.

Good to know
  • SQL Server gives the files written by BACKUP CERTIFICATE to its service account alone. See Encrypted backups for how to export the key and read it.
  • Reading an uncompressed backup of a TDE database takes longer than a plain one, as every page is decrypted. The cost is far lower on a compressed backup.
  • Keys in an EKM provider are not supported. An uncompressed backup taken while the TDE key was being changed is refused when the export reaches a page still under the previous key.
warning

Whoever has the private key can read the backups it opens, with no server involved: keep the key file as carefully as the backups themselves.

Backups Opened Again in Milliseconds thanks to Auto-Cache​

Exporting several tables from one backup, one FastBCP run per table, used to read the structure of the whole backup again for every run: seconds, or minutes on a large compressed backup. Now FastBCP remembers each backup it has opened, and the next runs on the same backup start right away.

Opening the backupFirst exportNext exports
SQL Server, 3.8 GB compressed backup (1.6.3)2.1 s0.003 s
Oracle 19c RMAN backup, BASIC compression (1.6.4)6.9 s~0.02 s
Good to know
  • Nothing to set: it is on by default for --connectiontype backup_mssql and backup_rman.
  • FastBCP keeps a small file per backup (about 1 MB for a 4 GB SQL Server backup, a few MB for an Oracle one) in the local application data of the user running it: %LOCALAPPDATA%\ArpeMSBackup\index and %LOCALAPPDATA%\ArpeOraBackup\index on Windows, ~/.cache/arpemsbackup/index and ~/.cache/arpeorabackup/index on Linux.
  • A backup that is rewritten or moved is opened in full again. Files unused for 30 days, or beyond 2 GB in all, are removed automatically.
  • After a new incremental RMAN backup, only the new backup is opened in full (since 1.6.8).
  • A service account may have no usable home: since 1.6.6, FASTBCP_INDEX_DIR names another directory, and FastBCP warns when the index cannot be saved, instead of opening the backup in full at every run without saying so.

Unsigned 64-bit Integers​

PC SOFT HFSQL "8-byte unsigned integers", read through --connectiontype oledb, could not be exported exactly: Parquet readers saw them as floating-point numbers (18446744073709551615 read as 1.8446744073709552e+19), and CSV exports stopped on values above 9,223,372,036,854,775,807. Since 1.6.4 they keep their exact value in every format.

Format1.6.4
ParquetUnsigned 64-bit integer (UINT_64, UBIGINT in DuckDB), instead of a decimal read as DOUBLE
CSV / TSVPlain integer text, up to 18,446,744,073,709,551,615
JSONInteger
BSONInt64, or a lossless Decimal128 above the Int64 range
Good to know
  • MySQL BIGINT UNSIGNED columns are now written to Parquet as unsigned 64-bit integers too, instead of text.
  • CSV and TSV exports no longer stop on any decimal value beyond the 64-bit integer range, whatever the source.

Windows: Full Speed in Scheduled Tasks​

On Windows 11, a program started without a window — a scheduled task, a service, an orchestrator, a remote shell — runs on the efficiency cores of a hybrid CPU, at a reduced clock. FastBCP now asks Windows for full speed when it starts.

Same export, started without a windowElapsed
FastBCP 1.6.05.7 s
FastBCP 1.5.118.1 s

Set FASTBCP_ECOQOS=1 to keep FastBCP under the Windows policy, on a host where it must stay discreet.

Doc Wizard​

The Doc Wizard offers backup_rman in the ADBC (High Performance) connectors: the server field becomes the directory of the backup pieces, the database field the piece names or patterns, and only None and Physical are offered as parallel methods. Since 1.6.8, its help shows how to name a whole incremental chain (l0/*;l1/**, **).

License​

FastBCP now ships the ARPE.IO Software License Agreement (LICENSE.txt in the zip). The license grant is unchanged; the agreement adds that the binaries may not be reverse engineered, manually or with the help of an AI system, and that the software may not be used to train an AI model to reproduce it. Using an AI assistant to operate FastBCP stays permitted. See License.

Release History​

1.6.8​

backup_rmanIncremental backups, levels 0 to 4, differential and cumulative: the latest version of a table, from a level 0 and the incremental backups on it
backup_rman--database takes paths, absolute or relative to --server, and ** for any number of subdirectories ("l0/*;l1/**"). Each entry must match a file, or the export fails naming it
backup_rmanFASTBCP_BACKUP_RESTORE_POINT and FASTBCP_BACKUP_RESTORE_MODE now apply, and the logs say which settings apply and what can be done when a backup is refused
backup_rmanFixed: a backup set without its last piece was read short, without an error. It now fails, naming it
backup_rmanFixed: a column added with a DEFAULT read NULL on the rows written before; a table truncated, then its tablespace backed up alone, read empty (now refused, naming it)
backup_rmanArchived log pieces in the directory, more than 64 pieces and COPIES 2 are read instead of refused; the tables of a dropped PDB are no longer listed
Doc WizardIncremental chains in --database
warning

A level 0 and its incremental backups, refused up to 1.6.7, now read fresher rows: FASTBCP_BACKUP_RESTORE_POINT=full keeps the level 0. A backup set without its last piece now fails the export. See Upgrading to 1.6.8.

ComponentVersion
ArpeOraBackup ADBC driver0.3.1

1.6.7​

backup_mssqlDifferential backups: the latest version of a table, from a full backup and its differential
backup_mssqlFASTBCP_BACKUP_RESTORE_POINT and FASTBCP_BACKUP_RESTORE_MODE choose which version is read, and what happens when a backup cannot be used
backup_mssqlThe logs say which settings apply, and what can be done when a backup is refused
backup_mssqlFixed: the end of some compressed backups was lost without an error, on about 2 % of the backups tried. No page was lost on 168 test backups
backup_mssqlFixed: a file or partial backup after a full backup was taken for a full backup. It is now refused by name
Doc WizardFull and differential backups in --database
warning

Some exports that worked with 1.6.6 now fail, naming the backup: a WITH NOINIT file whose newest differential has its full backup elsewhere, or with a file, filegroup or partial backup after the full; a file that lost its end, even with best_effort when it holds no other version. And a WITH NOINIT file with differentials now reads fresher rows. See Upgrading to 1.6.7. The first export from each backup reads its structure once more.

ComponentVersion
ArpeMSBackup ADBC driver0.2.0

1.6.6​

backup_mssqlEncrypted backups: WITH ENCRYPTION and TDE databases, read with the private key of the certificate
backup_mssqlFixed: a damaged or crafted .bak could make an export hang. It now stops with an error
backup_mssql, backup_rmanFASTBCP_INDEX_DIR names the directory of the backup index, and a warning says when the index cannot be saved
Doc WizardThe key file and its password for backup_mssql
warning

A backup whose data ends past the end of the file now stops at the open with truncated or damaged backup: it failed before too, later in the export. On Windows, FASTBCP_INDEX_DIR must name its drive and start from its root: C:index and \index stop the run at startup.

ComponentVersion
ArpeMSBackup ADBC driver0.1.17
ArpeOraBackup ADBC driver0.1.3

1.6.5​

backup_mssqlThe same Parquet as adbc_mssql: geography, geometry, vector, time(7) and sql_variant are written differently
backup_mssqlFar less memory on columnstore tables
backup_mssqljson and vector(n, float16) columns (SQL Server 2025) are refused with a clear message
backup_mssqlFASTBCP_UID_STRING_CASING now applies
ComponentVersion
ArpeMSBackup ADBC driver0.1.15

1.6.4​

backup_rmanBackups opened again in milliseconds; faster reads on Windows (internal improvement)
backup_rman: LONG / LONG RAWValues longer than a database block came out wrong; they are now exported whole, up to 2 GB
backup_rman: TIMESTAMP WITH LOCAL TIME ZONEValues were shifted by the time zone offset of the database; they are now right. A database time zone given as a region name (Europe/Paris) is refused for these columns, with a clear message
OLE DB (HFSQL), MySQLUnsigned 64-bit integers exported exactly
warning

If you exported Oracle tables with LONG / LONG RAW or TIMESTAMP WITH LOCAL TIME ZONE columns from RMAN backups with FastBCP 1.6.0 to 1.6.3, export them again with 1.6.4.

ComponentVersion
ArpeOraBackup ADBC driver0.1.2

1.6.3​

backup_mssqlExports up to 2.8× faster, backups opened again in milliseconds
backup_mssqlClustered columnstore fixes: columns starting with NULL values, datetimeoffset(0) to (2), large decimal; errors name the column
ComponentVersion
ArpeMSBackup ADBC driver0.1.12

1.6.2​

backup_mssql, adbc_mssqlThe whole SQL Server date range: datetime2(7) and datetimeoffset(7) as microseconds by default
backup_mssqlFixes: backup directories with accented characters, clustered columnstore nchar / nvarchar strings and repeated values
ComponentVersion
ArpeMSBackup ADBC driver0.1.9

1.6.1​

backup_rmanIncremental level 1 backup sets are now refused with a clear message naming the backup set

FastBCP 1.6.0 read an incremental level 1 backup set as a full backup: the export held only the rows of the blocks changed since the previous backup, with no error. Full backups and incremental level 0 backups were read correctly.

warning

If you exported tables with FastBCP 1.6.0 from RMAN incremental level 1 backups, export them again from the full (or level 0) backup with 1.6.1 or later.

ComponentVersion
ArpeOraBackup ADBC driver0.1.1

1.6.0​

New source--connectiontype backup_rman
backup_mssqlLarge striped backups ready in seconds, fixes for older databases, PAGE-compressed and clustered columnstore tables
WindowsFull speed in scheduled tasks
Doc Wizardbackup_rman
LicenseARPE.IO Software License Agreement
ComponentVersion
ArpeOraBackup ADBC driver0.1.0 (new)
ArpeMSBackup ADBC driver0.1.8
AdbcFileExport0.5.6 (arpeorabackup driver profile)

The other components are unchanged since 1.5.1.

Copyright © 2026 Architecture & Performance.