FASTBCP_CLOUD_MAXUPLOADTHREAD_PER_STREAM

Overview

The FASTBCP_CLOUD_MAXUPLOADTHREAD_PER_STREAM environment variable controls the maximum number of parallel threads used to upload each file to cloud storage during buffered uploads.

Syntax

Windows

$env:FASTBCP_CLOUD_MAXUPLOADTHREAD_PER_STREAM = "16"

Linux/macOS

export FASTBCP_CLOUD_MAXUPLOADTHREAD_PER_STREAM=16

Configuration

Property	Value
Type	Integer (> 0)
Default	8 when parallelmethod=None, 2 otherwise
Applies to	Cloud uploads when FASTBCP_CLOUD_ISBUFFERED=True
Since	FastBCP 1.0

Behavior

Default Values

FastBCP automatically adjusts the default based on the parallel extraction method:

Parallel Method	Default Threads	Reason
None (single stream)	8	FastBCP itself is not parallelized, so more bandwidth can be allocated to upload
Ntile, DataDriven, etc.	2	FastBCP is already extracting in parallel, so limit upload threads to avoid saturating network

How It Works

When FASTBCP_CLOUD_ISBUFFERED=True:

1. Data is written to a local temporary file
2. On stream close, the file is uploaded in parallel blocks
3. FASTBCP_CLOUD_MAXUPLOADTHREAD_PER_STREAM controls how many threads upload blocks concurrently
4. Maps to StorageTransferOptions.MaximumConcurrency in Azure SDK