OpenTelemetry

FastBCP supports OpenTelemetry for distributed tracing and log correlation. This allows you to correlate FastBCP operations with upstream callers (orchestrators, APIs, pipelines) and export structured logs to any OpenTelemetry-compatible backend such as Jaeger, Zipkin, or Grafana Tempo.

The --traceparent Parameter

The --traceparent parameter accepts a W3C Trace Context traceparent identifier. When provided, FastBCP uses this trace context to link its internal spans and logs to the calling system's trace, enabling end-to-end distributed tracing.

Syntax:

• Long form only: --traceparent "<traceparent_id>"

Format:

The traceparent value follows the W3C Trace Context specification:

{version}-{trace-id}-{parent-id}-{trace-flags}

Field	Format	Description
version	2 hex digits	Version of the trace context (currently 00)
trace-id	32 hex digits	Globally unique trace identifier
parent-id	16 hex digits	Identifier of the calling span
trace-flags	2 hex digits	Trace flags (e.g., 01 for sampled)

Example:

Windows

.\FastBCP.exe `
  ...
  --traceparent "00-abcdef1234567890abcdef1234567890-1234567890abcdef-01" `
  ...

Linux

./FastBCP \
  ...
  --traceparent "00-abcdef1234567890abcdef1234567890-1234567890abcdef-01" \
  ...

Tips

Pass the traceparent from your orchestrator (Airflow, Kestra, CI/CD pipeline, or any OpenTelemetry-instrumented caller) so that FastBCP operations appear as child spans in your distributed traces.

Configuring the OpenTelemetry Endpoint

FastBCP uses the Serilog OpenTelemetry sink to export logs. You can configure the endpoint using either environment variables or a FastBCP_Settings.json file.

Using Environment Variables

Set the OTEL_EXPORTER_OTLP_ENDPOINT environment variable to point to your OpenTelemetry collector:

Windows

$env:OTEL_EXPORTER_OTLP_ENDPOINT = "http://localhost:4317"

Linux

export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"

Using FastBCP_Settings.json

Add the OpenTelemetry sink to the WriteTo section of your FastBCP_Settings.json file. This gives you full control over the protocol, endpoint, and resource attributes.

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Microsoft": "Warning",
        "System": "Warning"
      }
    },

    "Properties": {
      "LogDirectory": "logs"
    },

    "Enrich": [
      "FromLogContext",
      "WithEnvironmentUserName",
      "WithProcessId",
      "WithThreadId",
      "WithMachineName",
      "fulltargetname"
    ],

    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "restrictedToMinimumLevel": "Information",
          "outputTemplate": "{Timestamp:yyyy-MM-ddTHH:mm:ss.fff zzz} -|- {Application} -|- {Level:u12} -|- {fulltargetname} -|- TraceId={TraceId} SpanId={SpanId} -|- {Message}{NewLine}{Exception}",
          "theme": "Serilog.Sinks.SystemConsole.Themes.SystemConsoleTheme::Literate, Serilog.Sinks.Console"
        }
      },
      {
        "Name": "OpenTelemetry",
        "Args": {
          "Endpoint": "http://localhost:4317",
          "Protocol": "Grpc",
          "ResourceAttributes": {
            "service.name": "FastBCP",
            "deployment.environment": "local-test"
          }
        }
      }
    ]
  }
}

Key settings in the OpenTelemetry sink:

Setting	Description
Endpoint	URL of your OpenTelemetry collector (default: http://localhost:4317)
Protocol	Transport protocol: Grpc (default) or HttpProtobuf
ResourceAttributes	Key-value pairs attached to every log entry (e.g., service name, environment)

Information

The TraceId and SpanId fields in the console output template allow you to see trace correlation directly in the console logs, even without an OpenTelemetry backend.

note

When both an environment variable and a FastBCP_Settings.json configuration are present, the settings file takes precedence for the OpenTelemetry sink configuration.