Release Notes 1.4

Preview

FastBCP 1.4 is currently a preview version. Content is subject to change before general availability.

New Features

New Parallel Method — Physical

The new --parallelmethod Physical splits a table on the physical row location of the source engine. You no longer need to know which engine-specific method applies: FastBCP picks it from the connection type.

Source	Connection types	Resolved strategy
PostgreSQL	pgsql, pgcopy, adbc_pgsql	Ctid
Oracle	oraodp, adbc_oracle	Rowid
SQL Server	mssql, msoledbsql, adbc_mssql	Physloc
Netezza	nzsql, nzcopy, nzoledb, adbc_netezza	NZDataSlice

• --sourceschema and --sourcetable are required. --query can be added to filter the table (simple query with a WHERE clause). --distributekeycolumn is not used.

• Any other source (odbc, oledb, mysql, clickhouse, hana, teradata, adbc_teradata) stops with an explicit error. With odbc and oledb, the error asks for a native connection type, because FastBCP cannot know which engine is behind the driver.

• The resolved strategy is logged:

  Distribute Method : Physical (Ctid)

• OpenTelemetry spans carry the resolved method in fastbcp.method. When Physical was resolved, the new fastbcp.method.requested tag holds Physical.

Windows

.\FastBCP.exe `
  --connectiontype pgsql `
  --server "localhost:5432" `
  --database "tpch" `
  --user "FastUser" `
  --password "FastPassword" `
  --sourceschema "tpch_10" `
  --sourcetable "orders" `
  --directory "D:\export\orders" `
  --fileoutput "orders.parquet" `
  --parallelmethod Physical `
  --paralleldegree 8

Linux

./FastBCP \
  --connectiontype pgsql \
  --server "localhost:5432" \
  --database "tpch" \
  --user "FastUser" \
  --password "FastPassword" \
  --sourceschema "tpch_10" \
  --sourcetable "orders" \
  --directory "/data/export/orders" \
  --fileoutput "orders.parquet" \
  --parallelmethod Physical \
  --paralleldegree 8

See Parallel Parameters for more details.

ADBC Teradata — -C adbc_teradata

FastBCP now reads Teradata through the embedded ArpeTeradata ADBC driver, via the new -C adbc_teradata connection type. The driver is used for both the metadata step and the data step, with the Teradata SQL dialect (Ntile, Random, Timepartition and DBPartition behave as with -C teradata).

Key highlights:

• Parquet output only, like the other adbc_* connection types
• Windows x64 and Linux x64 only
• Connection with --server host[:port] (legacy port, default 1025), --user, --password and an optional --database, or a --sourceconnectstring in the Teradata provider spelling (Server, Port, Database, User Id, Password, SSL Mode, HTTPS Port, TMODE, TTU Path, …). Explicit adbc.arpeteradata.* keys win.
• Trusted authentication (-A / --trusted) is not supported
• The default logon (sslmode=prefer) uses TLS on the HTTPS port (443) and needs no Teradata client software. When the server has no TLS, or with SSL Mode=disable, the driver uses the legacy port through the customer's Teradata Tools and Utilities (TTU).

TTU for the legacy-port logon:

• Linux: FastBCP looks for TTU in the new FASTBCP_TTU_PATH environment variable, then in a ttu or TeradataToolsAndUtilitiesBase directory next to the FastBCP binary. When nothing is found, the driver searches itself (ARPETERADATA_TTU_PATH, /opt/teradata/client, the system library path). Teradata's TTU download can be unpacked next to the binary without installing it (see FASTBCP_TTU_PATH). FastBCP warns at startup when the TTU folder lacks tdclientdir or the ICU links, or when the client directory is not named after its version.
• Windows: the driver finds wincli32.dll itself (ARPETERADATA_TTU_PATH, its own directory, the registered TTU installation).

Windows

.\FastBCP.exe `
  --connectiontype adbc_teradata `
  --server "td-host" `
  --database "TPCH" `
  --user "FastUser" `
  --password "FastPassword" `
  --sourceschema "TPCH" `
  --sourcetable "ORDERS" `
  --directory "D:\export\orders" `
  --fileoutput "orders.parquet" `
  --parallelmethod Ntile `
  --distributekeycolumn "o_orderkey" `
  --paralleldegree 8

Linux

export FASTBCP_TTU_PATH=/opt/teradata/client/20.00

./FastBCP \
  --connectiontype adbc_teradata \
  --server "td-host" \
  --database "TPCH" \
  --user "FastUser" \
  --password "FastPassword" \
  --sourceschema "TPCH" \
  --sourcetable "ORDERS" \
  --directory "/data/export/orders" \
  --fileoutput "orders.parquet" \
  --parallelmethod Ntile \
  --distributekeycolumn "o_orderkey" \
  --paralleldegree 8

See Connection Parameters for more details.

Deprecations

Ctid, Rowid, Physloc and NZDataSlice

The engine-specific values Ctid, Rowid, Physloc and NZDataSlice are deprecated and will be removed in FastBCP 2.0. Use --parallelmethod Physical instead: it resolves to the same strategy from the connection type.

They are still accepted. FastBCP runs the export as before and logs a warning:

Parallel method Ctid is deprecated and will be removed in 2.0 ==> use --parallelmethod Physical

Behaviour Changes

Physloc and Rowid Require --sourceschema and --sourcetable

Like Ctid, the Physloc and Rowid methods, used directly or through Physical, now require --sourceschema and --sourcetable. Without them, FastBCP stops at the parameter check:

ERROR in command line parameters : With Physical (Physloc) Method you must define source schema (--sourceschema) and source table (--sourcetable) ==> With Physical (Physloc) Method --sourceschema and --sourcetable are both mandatory

Before 1.4, Rowid crashed when they were missing. A Physloc export based on --query alone must now also give the schema and table.

Physloc Distribution

• Even chunks on rowstore tables: the distribute key is now the two low bytes of the page id. The previous key was a multiple of 256 for most page ids, so with an even degree of parallelism (2, 4, 8, …) most chunks stayed empty and a single chunk did the work. Runs of 256 pages now go to each chunk.
• Clustered columnstore tables: %%physloc%% is a row id there, not a page address, so the rowstore key was the same for every row. FastBCP now detects a clustered columnstore index on --sourceschema.--sourcetable and splits on the whole row id. This is logged as Physloc : <schema>.<table> is a clustered columnstore, splitting on the columnstore row id. If the check fails, FastBCP logs a warning and uses the rowstore key.

Bug Fixes

• Random row counts: the rows and columns of each chunk were counted in a variable shared by the concurrent chunks, which mixed up the per-chunk row counts. Each chunk now counts its own rows. Physloc uses the same code path and is fixed too.

Dependencies

Component	Version
.NET packages	10.0.13
Microsoft.Data.SqlClient	7.1.1
SAP HANA .NET driver	2.30.27 (Sap.Data.Hana.Net.v10.0, native client libraries from SAP HANA Client 2.30.27)
ArpeMSSQL ADBC driver	0.7.6
ArpeOracle ADBC driver	0.4.0
ArpeTeradata ADBC driver	0.3.3 (new)
AWSSDK.S3	4.0.104.4
Azure.Storage.Files.DataLake	12.28.0

Documentation

• Parallel Parameters: Physical method, deprecation of Ctid, Rowid, Physloc and NZDataSlice.
• Connection Parameters: adbc_teradata connection type.
• FASTBCP_TTU_PATH: new environment variable.
• Examples: the parallel examples use Physical for PostgreSQL, Oracle, SQL Server and Netezza; new Teradata examples in Other Databases.