Release Notes 1.4
One parallel method to rule them all. FastBCP 1.4 picks the fastest physical split for your engine on its own, brings Teradata to the high-performance ADBC family, and makes SQL Server parallel exports up to 2.6× faster.
Physical resolves to Ctid, Rowid, Physloc or NZDataSlice from the connection type.
At degree 8 on a 60M-row SQL Server heap, the export went from 126 s to 48 s.
The ADBC Teradata driver logs on over TLS by default, with no TTU to install.
What's New: At a Glance
| New parallel method | --parallelmethod Physical: FastBCP picks the physical split of the source engine | |
| New source | --connectiontype adbc_teradata: Teradata to Parquet through the embedded ArpeTeradata driver | |
| Faster exports | Physloc 1.6× to 2.6× faster, better load balancing for Timepartition | |
| Data correctness | Fixes for rows skipped or duplicated by Ntile, Timepartition and filters with OR — see the fixes | |
| Deprecations | Ctid, Rowid, Physloc and NZDataSlice will be removed in 2.0 — use Physical |
Physical: One Parallel Method for Every Engine
No more remembering which engine-specific method applies to which database. With --parallelmethod Physical, FastBCP splits the table on the physical row location of the source and picks the right strategy from the connection type.
| Source | Connection types | Resolved strategy |
|---|---|---|
| PostgreSQL | pgsql, pgcopy, adbc_pgsql | Ctid |
| Oracle | oraodp, adbc_oracle | Rowid |
| SQL Server | mssql, msoledbsql, adbc_mssql | Physloc |
| Netezza | nzsql, nzcopy, nzoledb, adbc_netezza | NZDataSlice |
- Windows
- Linux
.\FastBCP.exe `
--connectiontype pgsql `
--server "localhost:5432" `
--database "tpch" `
--user "FastUser" `
--password "FastPassword" `
--sourceschema "tpch_10" `
--sourcetable "orders" `
--directory "D:\export\orders" `
--fileoutput "orders.parquet" `
--parallelmethod Physical `
--paralleldegree 8
./FastBCP \
--connectiontype pgsql \
--server "localhost:5432" \
--database "tpch" \
--user "FastUser" \
--password "FastPassword" \
--sourceschema "tpch_10" \
--sourcetable "orders" \
--directory "/data/export/orders" \
--fileoutput "orders.parquet" \
--parallelmethod Physical \
--paralleldegree 8
The resolved strategy is logged, and OpenTelemetry spans carry it in fastbcp.method (with Physical in the new fastbcp.method.requested tag):
Distribute Method : Physical (Ctid)
--sourceschemaand--sourcetableare required.--querycan be added to filter the table (simple query with aWHEREclause).--distributekeycolumnis not used.- Any other source (
odbc,oledb,mysql,clickhouse,hana,teradata,adbc_teradata) stops with an explicit error. Withodbcandoledb, the error asks for a native connection type, because FastBCP cannot know which engine is behind the driver.
See also: Parallel Parameters
ADBC Teradata — --connectiontype adbc_teradata
Teradata joins the high-performance ADBC family. The embedded ArpeTeradata driver handles both the metadata and the data steps with the Teradata SQL dialect — and by default it needs no Teradata client software at all.
| Supported | ||
|---|---|---|
| Platforms | Windows x64 and Linux x64, Parquet output only (like every adbc_* connection type) | |
| Parallel methods | Every method of --connectiontype teradata: Random, Ntile, DataDriven, RangeId, Timepartition, and DBPartition on PPI tables | |
| Connection | --server host[:port] (legacy port, default 1025), --user, --password, optional --database — or a --sourceconnectstring in the Teradata provider spelling (Server, Port, Database, User Id, Password, SSL Mode, HTTPS Port, TMODE, TTU Path, …). Explicit adbc.arpeteradata.* keys win | |
| TLS logon (default) | sslmode=prefer uses TLS on the HTTPS port (443): no Teradata client software needed | |
| Legacy port | When the server has no TLS, or with SSL Mode=disable, the driver uses the legacy port through your Teradata Tools and Utilities (TTU), on Linux and Windows | |
| Logon mechanisms | LDAP, JWT, KRB5 and TDNEGO (ArpeTeradata 0.3.4). JWT and Kerberos work over TLS, without TTU |
- Trusted authentication (
--trusted) is not supported. - Finding TTU for the legacy port on Linux: FastBCP looks in the new
FASTBCP_TTU_PATHenvironment variable, then in attuorTeradataToolsAndUtilitiesBasedirectory next to the FastBCP binary; otherwise the driver searchesARPETERADATA_TTU_PATH,/opt/teradata/clientand the system library path. Teradata's TTU download can be unpacked next to the binary without installing it. FastBCP warns at startup when the TTU folder lackstdclientdiror the ICU links, or when the client directory is not named after its version. - On Windows, the driver finds
wincli32.dllitself (ARPETERADATA_TTU_PATH, its own directory, the registered TTU installation).
- Windows
- Linux
.\FastBCP.exe `
--connectiontype adbc_teradata `
--server "td-host" `
--database "TPCH" `
--user "FastUser" `
--password "FastPassword" `
--sourceschema "TPCH" `
--sourcetable "ORDERS" `
--directory "D:\export\orders" `
--fileoutput "orders.parquet" `
--parallelmethod Ntile `
--distributekeycolumn "o_orderkey" `
--paralleldegree 8
export FASTBCP_TTU_PATH=/opt/teradata/client/20.00
./FastBCP \
--connectiontype adbc_teradata \
--server "td-host" \
--database "TPCH" \
--user "FastUser" \
--password "FastPassword" \
--sourceschema "TPCH" \
--sourcetable "ORDERS" \
--directory "/data/export/orders" \
--fileoutput "orders.parquet" \
--parallelmethod Ntile \
--distributekeycolumn "o_orderkey" \
--paralleldegree 8
See also: Connection Parameters
Performance
| Before | 1.4 | |
|---|---|---|
Physloc, degree 8, 60M-row SQL Server heap | 126 s | 48 s |
Physlocis 1.6× to 2.6× faster at even degrees of parallelism, thanks to the newPhyslocdistribution.Timepartitionload balancing: internal improvement.Timepartitionwith quoted or case-sensitive date columns: columns such as"OrderDate"or[OrderDate]now work.
Deprecations
Ctid, Rowid, Physloc and NZDataSlice will be removed in FastBCP 2.0Use --parallelmethod Physical instead: it resolves to the same strategy from the connection type. The old values are still accepted — FastBCP runs the export as before and logs a warning:
Parallel method Ctid is deprecated and will be removed in 2.0 ==> use --parallelmethod Physical
Behaviour Changes
Physloc and Rowid Require --sourceschema and --sourcetable
Like Ctid, the Physloc and Rowid methods, used directly or through Physical, now require --sourceschema and --sourcetable. Without them, FastBCP stops at the parameter check:
ERROR in command line parameters : With Physical (Physloc) Method you must define source schema (--sourceschema) and source table (--sourcetable) ==> With Physical (Physloc) Method --sourceschema and --sourcetable are both mandatory
Before 1.4, Rowid crashed when they were missing. A Physloc export based on --query alone must now also give the schema and table.
Physloc Distribution
Physloc now splits both rowstore and clustered columnstore tables into even chunks, whatever the degree of parallelism (internal improvement). A clustered columnstore is logged as Physloc : <schema>.<table> is a clustered columnstore, splitting on the columnstore row id.
ArpeOracle 0.4.1 Security Defaults (--connectiontype adbc_oracle)
- TCPS now verifies the server certificate by default.
- Kerberos no longer forwards the TGT unless
krb5_forward=trueis set, and requires mutual authentication. - Token and OAuth2 logons require TCPS.
--help
-?, --helpis now listed in the usage.--help,/?and/helpnow behave like-?: they print the usage only, without the banner, and exit with code0. Before, they printed the usage, then the banner and fourERROR in command line parameterslines, and exited with code1.
Bug Fixes
Physloconly parallelised with an odd--paralleldegree: with a degree of 2, 4, 8 or 16, one chunk held every row and the other workers exported nothing; with 6 or 12, only three chunks got rows. Fixed by the newPhyslocdistribution: on rowstore tables, the chunks stay within 3% of even up to degree 192.RandomandPhyslocrow counts: the per-chunk log lines and the finalTotal data rowswere wrong. Fixed (internal improvement).Ntiledropped rows with sub-millisecond timestamp keys: this affected SQL Serverdatetime2(4..7), PostgreSQLtimestampand OracleTIMESTAMP(4..9). Rows could be silently skipped; every row is now exported exactly once whatever its precision (internal improvement). A bucket whose minimum isNULLno longer throws, and numeric bounds no longer depend on the regional settings (no more1,5withfr-FR).- Source filter with
OR: rows outside the source filter could be exported when the filter contained anOR, with every parallel method that rewrites theWHEREclause. Fixed (internal improvement). Timepartitionskipped rows with aNULLdate: they are now exported to a<column>=NULLdirectory.Timepartitionwith week granularity duplicated rows: fixed on every engine (SQL Server, PostgreSQL, Netezza, MySQL, Oracle, SAP HANA, Teradata, ClickHouse) (internal improvement).Performance Output Files SizeandDisk Write ThroughputshowedunknownwithDataDriven,TimepartitionandDBPartition. The size now covers every file written by the run, and still ignores unrelated files already in the directory.
Driver Improvements
- ArpeMSSQL 0.7.6 — reads
sql_variantand the SQL Server 2025JSONtype; an unreadable column no longer drops the connection. - ArpeTeradata 0.3.4 — new driver behind
--connectiontype adbc_teradata. - ArpeOracle 0.4.1 — stricter security defaults, see above.
Dependencies
| Component | Version |
|---|---|
| .NET packages | 10.0.13 |
| Microsoft.Data.SqlClient | 7.1.1 |
| SAP HANA .NET driver | 2.30.27 (Sap.Data.Hana.Net.v10.0, native client libraries from SAP HANA Client 2.30.27) |
| ArpeMSSQL ADBC driver | 0.7.6 |
| ArpePGSQL ADBC driver | 0.4.7 |
| ArpeOracle ADBC driver | 0.4.1 |
| ArpeNetezza ADBC driver | 0.3.7 |
| ArpeTeradata ADBC driver | 0.3.4 (new) |
| AdbcFileExport | 0.5.4 |
| AWSSDK.S3 | 4.0.104.4 |
| Azure.Storage.Files.DataLake | 12.28.0 |
Documentation
- Parallel Parameters:
Physicalmethod, deprecation ofCtid,Rowid,PhyslocandNZDataSlice. - Connection Parameters:
adbc_teradataconnection type. - FASTBCP_TTU_PATH: new environment variable.
- Examples: the parallel examples use
Physicalfor PostgreSQL, Oracle, SQL Server and Netezza; new Teradata examples in Other Databases.