Configuration

Barca needs no configuration file. Settings come from command-line flags, environment variables and an optional barca.toml, in that order:

CLI flag > environment variable > barca.toml > built-in default

For example, three ways to select the staging environment:

CLI flag

barca get pipeline.py --env staging

Environment variable

export BARCA_ENV=staging
barca get pipeline.py

barca.toml

barca.toml

default_env = "staging"

Where things live

Barca keeps its cache and run history in .barca/, in the project root. Results are always written there first. A remote store, when configured, holds a copy of the result files and backups of the metadata database. The entries you may care about:

Path	What it is	Safe to delete
.barca/metadata.db	Run history and the record of every cached result (a local libSQL database; -wal and -shm belong to it).	Only to discard local history: every asset then recomputes on the next run (with a store, barca state restore puts a backup back instead). Delete metadata.db-wal and metadata.db-shm with it, with no barca command running.
.barca/artifacts/<node>/<run_hash>.<ext>	One file per cached result: .json, .pkl, .parquet or .csv. A partitioned asset has one directory per key (pipeline.py--sales_region_emea/).	Yes. A result whose file is gone is computed again when a run needs to read it, or downloaded again if it is in a remote store. Prefer --refresh <asset> to force a recompute.
.barca/envs/<name>/	The same metadata.db, artifacts/, backups/ and live-logs/ for a named environment (--env <name>).	As above, per environment.
.barca/backups/	With a store ([remote] uri): the five newest backups of metadata.db, each with its sidecar, and a marker on one the store has not confirmed yet.	Yes; then only the store’s copies are left to restore from.

Everything else barca writes under .barca/ (lock files, run-owners/, staging/, sql-cache/, range-cache/, live-logs/, branches/, inspection-cache/, auth/, gc-state.json, the databases state restore and state reset move aside, a batched call’s .batch-* directory, and the entries barca 0.25 and earlier left) is listed with its owner, when it is removed and whether barca gc evicts it in specs/interfaces/dot-barca-layout.md, the one owner of the layout. A .gitignore inside .barca/ keeps all of it out of git. | .barca/gc-state.json | From 0.25: what the last full measurement of .barca/ found, so the check after each command does not walk the artifact directories (Cache size). | Yes; the next check measures again. | | .barca/.gitignore | Written by barca, containing *, so git ignores everything under .barca/. | It is written again. | | .barca/live-logs/<run_id>.jsonl | A run’s output while it runs, read by barca logs -f and barca serve. The run deletes it once its lines are in metadata.db; a killed run’s file stays. | Yes, when no run is going. | | .barca/branches/<run>-<pid>/ | What a run’s parallel() branches returned. Removed when the calling step and the run end, and by the next run when its run was killed. | Yes, when no run is going. | | .barca/inspection-cache/<env>.json | Disposable evidence barca status reuses (with .lock and .epoch files), at most 8 MiB. | Yes, always. | | .barca/auth/ | barca serve token state (server mode), owner-only. | Not while barca serve runs; deleting it discards every token. |

Some rows appear only in the situations named. For backups see barca docs remote; for a missing artifact see barca docs cache, “A cached result whose artifact is missing”.

Do not edit files under .barca/, and do not open metadata.db with another program while barca is running. To look at results use barca sql and barca status.

Project root

The nearest barca.toml at or above the directory you run barca from marks the project root. Barca changes into the root before doing anything, so barca.toml, .barca/, node ids and the working directory of every step are the same wherever in the project you run it. File arguments are read relative to where you typed them. Without a barca.toml above you, the current directory is the root.

An empty barca.toml is enough to mark a root. When the root is not the current directory, barca prints barca: project root: <path> on stderr.

barca.toml

Every key is optional.

# The environment used when neither --env nor BARCA_ENV is given.
default_env = "dev"

[discovery]
# Which files a command with no file arguments reads. See the Discovery page.
exclude = ["scratch/**", "notebooks/**"]   # skipped in addition to the built-in list
include = ["pipelines/**/*.py"]            # only these files (replaces the walk)

[cache]
# How much disk .barca/ may use. "20GB", "500MB", "20GiB", a number of bytes, "10%" of the
# filesystem, or "off". Default: 10% of the filesystem. See "Cache size" below.
max_size = "20GB"

[server]
# Seconds `barca serve` waits for runs in flight on SIGTERM or SIGINT before putting them back
# in its run queue for the next start.
drain_timeout = 30
# How many runs `barca serve` executes at once; the others wait in its run queue (experimental).
max_runs = 4
# The most the weight= of the steps of every run of `barca serve` may add up to (experimental).
# Default: none (each run has only its own capacity).
capacity = 4
# Seconds a run under `barca serve` may take before it is stopped. Default: 0, no limit.
run_timeout_seconds = 3600

[execution]
# The most the weight= of the steps running at once may add up to (experimental).
# Default: the pool size. See "Heavy steps" on the Decorators page.
capacity = 4
# How many worker processes a run starts at most. Default: the cores, or the capacity when
# it is smaller. BARCA_POOL_SIZE wins over it.
pool_size = 4

[telemetry]
# The Datadog site traces go to: where the web UI's "View trace" links open. DD_SITE wins over
# it. Default: datadoghq.com when BARCA_TELEMETRY=datadog is set.
datadog_site = "datadoghq.com"

[remote]
# Root of the remote store. Setting it turns remote storage on:
# artifacts at {uri}/{env}/artifacts, backups of the history at {uri}/{env}/state/backups/.
# A plain directory (a shared mount) also works.
uri = "s3://my-bucket/barca/my-project"

# Optional overrides, used as written (the environment name is not added):
# artifacts_uri = "s3://my-bucket/custom-artifacts"

transfer_concurrency = 4    # artifact uploads and downloads in flight at once (at least 1)
transfer_timeout = 600      # seconds one transfer attempt may run (at least 1)
backup_interval = "1m"     # routine metadata backups; "off" retains final backups

# Passed to fsspec.filesystem(protocol, ...), one table per protocol: s3, gcs, abfs.
[remote.storage_options.s3]
endpoint_url = "https://<account-id>.r2.cloudflarestorage.com"

[server]
# The barca serve that is the team's ledger (experimental). get and run still run here,
# read cache hits from its ledger and post what they compute (needs [remote] with the
# server's store); status, history and stats answer from it; --server runs a command there.
url = "https://barca.internal.example.com"

Key	Default	Meaning
default_env	default	Environment name when --env and BARCA_ENV are absent.
require_env	false	true: a command that reads or writes an environment exits 2 unless --env or BARCA_ENV names it, even with default_env set. See Environments.
[discovery].include	none	Glob patterns; only matching files are discovered. See Discovery.
[discovery].exclude	none	Glob patterns skipped in addition to the built-in list.
[cache].max_size	10% of the filesystem	Size budget for .barca/ excluding recognized local metadata backups; past it barca evicts down to 80% of it. "off" for no limit. See Cache size.
[execution].capacity	the pool size	Experimental. The most the weight= of the steps running at once may add up to; a whole number of at least 1. A step heavier than it runs alone. See Heavy steps.
[server].drain_timeout	30	Seconds barca serve waits for runs in flight once stopped with SIGTERM or SIGINT, before it puts them back at the head of its run queue (they start again after the restart) and exits 0. 0 requeues at once; a second signal cancels them instead. --drain-timeout overrides it.
[execution].pool_size	the cores, or the capacity when smaller	How many worker processes a run starts at most; a whole number of at least 1. Without it a run starts no more workers than its capacity (every step weighs at least 1). BARCA_POOL_SIZE overrides it. barca plan (pool_source) and barca doctor say which applied.
[server].capacity	none	Experimental. The most the weight= of the steps of every run of barca serve may add up to, admitted first come, first served across runs (a heavy step is not overtaken by lighter ones). Bounds each run’s pool too. --capacity overrides it. See Scheduling.
[server].run_timeout_seconds	0 (no limit)	Seconds a run under barca serve may take, from when it leaves the queue; a run that takes longer is stopped and recorded as an error naming the setting. --run-timeout overrides it; Schedule(..., timeout_seconds=N) wins for its ticks. Before 0.31.0 the limit was a fixed 600 s.
[telemetry].datadog_site	datadoghq.com with BARCA_TELEMETRY=datadog, else none	Experimental. The Datadog site (datadoghq.com, datadoghq.eu, us3.datadoghq.com) whose web app the Runs page’s View trace links open. DD_SITE overrides it. A value that is not a host name stops barca serve with exit 2. See Telemetry.
[server].max_runs	one per core	Experimental. How many runs barca serve executes at once; the others wait in its run queue, in order. At least 1. --max-runs overrides it.
[remote].uri	none	Root of the remote store.
[remote].artifacts_uri	{uri}/{env}/artifacts	Artifact location, used as written.
[remote].transfer_concurrency	4	Transfers in flight at once.
[remote].transfer_timeout	600	Seconds one transfer attempt may run.
[remote].backup_interval	"1m"	Routine metadata backups: positive whole seconds or s/m/h/d suffix, or off. Final backups remain. Invalid values exit 2.
[remote.storage_options.<protocol>]	none	Options for the fsspec filesystem of that protocol.
[server].auth	"off"	Opt-in project bearer authentication: "off" or "token"; BARCA_SERVER_AUTH overrides it, serve --auth overrides both. Local tokens bootstrap grants initial access; see Server Mode.
[server].url	none	The barca serve that is the team’s ledger (BARCA_SERVER overrides it): http:// or https://, with any proxy path prefix. get and run run here and read and post through its ledger (requires [remote] pointing at the server’s store); status, history and stats answer from it; sql resolves its views from it; a bare --server runs the command there; --local ignores it. See Server mode and Server Mode.

An unknown key or a malformed file is a usage error (exit 2) that names the key and lists the valid ones:

invalid ./barca.toml: TOML parse error at line 1, column 1
unknown field `defualt_env`, expected one of `default_env`, `require_env`, `remote`, `discovery`, `cache`, `server`, `execution`

Keep credentials out of barca.toml; they come from the environment.

Cache size

From 0.25, .barca/ has a size limit: [cache].max_size, or BARCA_CACHE_MAX_SIZE, and with neither, 10% of the filesystem .barca/ is on. "off" removes the limit (earlier versions had none: every new run hash added a file and kept the old one).

• KB, MB, GB, TB are decimal (20GB is 20 × 10⁹ bytes); KiB, MiB, GiB, TiB are binary; a bare number is bytes. A percentage is of the filesystem’s size, read at each check.
• Budgeted size covers .barca/ across every environment, including the metadata databases (counted, and each trimmed to a tenth of the limit, at most 1 GiB; barca docs cache), except recognized local metadata backup files. Those restore copies, their owned sidecars, markers and temporary copies keep the five-copy rotation and are reported outside the budget. Physical totals and free-space usage include them. Unrelated files in backup directories and other directories named backups remain budgeted.

After every barca get and barca run, after every run under barca serve, and once a minute under barca serve, barca checks the size. Over the limit, it deletes files in this order until budgeted bytes are at or below 80% of the limit:

1. Leftovers: staging directories of processes that are no longer running, *.moved-aside* directories, artifact files no metadata row names (what barca state reset and barca state restore leave behind), sql-cache/ copies and range-cache/ directories (least recently used first).
2. Superseded results: a result whose node (or partition key) has a newer successful result. Oldest first.
3. Local copies the artifact store holds: downloaded again when needed. Least recently read first.
4. Current results: computed again when needed. Least recently read first.

It never deletes what the command produced, read or returned; results a run in flight holds; anything but leftovers and superseded results while another run is in progress on this .barca/; the current result of a freshness=Manual node the store does not hold; an unnamed artifact file while a run is in progress or in the minute after it was written; the metadata databases, their backups and the copies reset and restore moved aside; or anything outside .barca/. The remote store is never touched: bound it with your bucket’s lifecycle rules.

When something was evicted, stderr says so in one line, and --json results gain cache: {size_bytes, limit_bytes, evicted_files, evicted_bytes}. An evicted result that a later run needs is downloaded or computed again under the same run hash, with reason evicted. barca status shows it as stale with reason evicted. barca gc runs the same eviction on demand (--dry-run to see it first, --max-size for a one-off limit; when it cannot reach its target, held says what holds the rest), and barca doctor reports the size against the limit and the free space. The full description is in barca docs disk (Disk Use).

[remote].state, [remote].state_uri and [remote].push_retries were removed in 0.26.0 with the shared history (RFC-0009). A file that still sets one is a usage error (exit 2) naming it, even under BARCA_REMOTE=off: remove the key. Backups need no setting.

Environment variables

Variables that override barca.toml:

Variable	Overrides	Notes
BARCA_ENV	default_env	Overridden by --env.
BARCA_REQUIRE_ENV	require_env	1 (or true, yes) requires --env or BARCA_ENV; 0 (or false, no) does not; anything else exits 2.
BARCA_CAPACITY	[execution].capacity	Experimental. A whole number of at least 1; anything else exits 2.
BARCA_SERVER_AUTH	[server].auth	Experimental. off (default) or token; overridden by serve --auth; anything else exits 2. See Server Mode.
BARCA_SERVER_RECONNECT	none	Experimental. Seconds a --server client following a run (and logs -f --server) waits for a restarting server to come back (default 60; 0 gives up at once). Not a number of seconds exits 2.
BARCA_CACHE_MAX_SIZE	[cache].max_size	Same values; an invalid one exits 2. For a container that sets its limit when it is deployed.
BARCA_REMOTE	process override	off uses local artifacts/history, takes no backups and overrides all remote settings. Unset/empty keeps normal resolution; other values exit 2.
BARCA_REMOTE_URI	[remote].uri	Enough on its own to turn remote storage and history backups on.
BARCA_TRANSFER_CONCURRENCY	[remote].transfer_concurrency	A positive integer. In 0.18.0 an invalid value exits 3, not 2.
BARCA_TRANSFER_TIMEOUT	[remote].transfer_timeout	Positive integer seconds, per attempt. In 0.18.0 an invalid value exits 3, not 2.
BARCA_TRANSFER_RETRIES	none	Experimental. How many times a failed artifact upload or download is retried after its first attempt (default 3), with waits of 0.5 s, 1 s, 2 s, …; a missing object or an authentication error is not retried. Read by the Python transfer helper as written, not checked by barca.
BARCA_BACKUP_INTERVAL	[remote].backup_interval	Overrides routine backup cadence, including served-run completion; off retains final CLI/server shutdown backups.
BARCA_STORAGE_OPTIONS	[remote.storage_options.*]	JSON keyed by protocol, merged over the toml tables key by key.
FSSPEC_<PROTOCOL>_<OPTION>	one storage option	fsspec’s own convention (FSSPEC_S3_ENDPOINT_URL, FSSPEC_GCS_PROJECT). Overridden by both of the above.
BARCA_SERVER	[server].url	The server a bare --server uses. Overridden by --server <url>.
DD_SITE	[telemetry].datadog_site	The Datadog site the web UI’s trace links open; one that is not a host name stops barca serve with exit 2. See Telemetry.
BARCA_ARTIFACT_URI	the artifact location	Used as written, with no environment name added, and history stays local. Kept from 0.4.0. Warns when combined with a non-default environment. No backups are taken. Prefer BARCA_REMOTE_URI.

An empty value counts as unset. BARCA_REMOTE=off overrides the URI, transfer and storage-options settings above while preserving the selected local environment. BARCA_STATE, BARCA_STATE_URI and BARCA_PUSH_RETRIES were removed in 0.26.0: setting one is a usage error (exit 2) naming it.

Variables with no barca.toml key:

Variable	Effect
BARCA_OUTPUT	json or pretty: the output format when no flag is given. Any other value exits 2.
BARCA_PROGRESS_SECS	Seconds between still running lines on stderr (default 15; 0 turns them off).
BARCA_POOL_SIZE	Number of Python worker processes (default: [execution].pool_size, else the available cores, or the capacity when it is smaller).
BARCA_TRACE_TIMING	When set, prints a timing breakdown on stderr.
BARCA_TELEMETRY	datadog sends finished runs to Datadog. See Telemetry for it and the DD_* variables.
BARCA_COMM_COST_SECONDS	The scheduler’s estimate of the cost of dispatching one step.
BARCA_TRACE_LIBRARIES	1 or true: with BARCA_TELEMETRY=datadog and ddtrace installed, workers patch every library ddtrace supports. See Telemetry.
BARCA_SLACK_WEBHOOK_URL	A Slack incoming webhook (https://): barca posts when a job starts failing, recovers, or is still failing a day later. Never printed. See Telemetry.
BARCA_SLACK_NOTIFY	changes (default), failures or all: which finished runs post to Slack; any other value warns and changes applies.
BARCA_UI_URL	Base URL of the web UI; Slack posts link the run id to it.
BARCA_SERVER_TOKEN	Experimental. The bearer credential the CLI and the deprecated Python Client send to a server with token auth; not combinable with URL userinfo; an invalid one exits 2. See Server Mode.
BARCA_LOG	json: barca serve writes every line as one JSON object on stdout instead of text on stderr; text or unset is the default. Under serve any other value exits 2; other commands ignore it with a warning. See Telemetry.

The CLI contract marks each variable stable or experimental. BARCA_SOCKET, BARCA_WORKER, BARCA_WORKER_ID and BARCA_WORKER_AGENT are set by barca for its own worker processes, and BARCA_LIFELINE, BARCA_JOB_PGID, BARCA_UPLOAD_SNAPSHOT_PREFIX and BARCA_DOCTOR_REQUEST for its Python helpers; do not set them. BARCA_TEST_* variables are hooks for barca’s own tests.

Credentials are read by the storage libraries from their usual variables (AWS_*, GOOGLE_APPLICATION_CREDENTIALS, AZURE_STORAGE_*). BARCA_REMOTE_URI plus those is a complete remote setup with no barca.toml. See Remote Storage.

Server mode

A configured server is the team’s ledger (experimental; barca docs server-mode, Deploying, RFC-0007). Which server:

--server <url> > BARCA_SERVER > [server].url in barca.toml

What a command does with it:

Command	Without a flag	--server [<url>]	--local
get, run	runs here; cache hits from the server’s ledger, results posted to it	runs on the server, printed here	runs here against this machine’s ledger; nothing sent
status, history, stats	answered by the server (status adds local per node)	answered by the server	this machine’s records
list	this machine’s files	the deployed nodes	this machine’s files
sql	views and barca.* tables from the server’s ledger; the query runs here	the same, from that server	this machine’s results

barca get total                                                # here, against the ledger
barca get total --server https://barca.internal.example.com   # that server, for this command
export BARCA_SERVER=https://barca.internal.example.com
barca get total --server                                       # the configured server
barca get total --local                                        # this machine only

A bare --server with no server configured is a usage error (exit 2). When the URL came from BARCA_SERVER or barca.toml, barca prints [barca] server <url> (<source>) once on stderr. --local is a usage error with --server. A local get/run against the ledger needs [remote] pointing at the server’s store (a usage error otherwise, naming both stores); a server that cannot be reached is exit 3 naming it, with --local as the remediation. --env with --server must name the server’s environment.

Environments (--env)

--env <name> keeps a separate cache, history and remote location per name, so dev and prod never share results. get, run, status, sql, history, stats and serve accept it. plan and list read no state and do not. The name comes from --env, then BARCA_ENV, then default_env, then default, and may contain letters, digits, ., _ and -; anything else exits 2.

get and run print the environment and artifact store they use once at startup, on stderr, when the environment comes from default_env or an artifact store is configured, so a one-off command on a machine set up for another environment says which one it uses:

[barca] env prod (default_env in barca.toml); artifact store local (.barca/envs/prod/artifacts)
[barca] env dev (--env); artifact store s3://bucket/proj/dev/artifacts

The source in brackets is --env, BARCA_ENV, default_env in barca.toml or built-in default; credentials and signed URI queries are redacted. The line is never on stdout, so --json output is unchanged. barca serve always prints it, with the metadata DB and its format added: ...; metadata .barca/envs/prod/metadata.db (db format 1) (see Deploying).

On a machine with more than one environment (.barca/metadata.db for default, and each directory of .barca/envs/), a command that names none with --env or BARCA_ENV prints one note on stderr naming the environment it uses and the ones there are, in every output mode (stdout and --json are unchanged):

[barca] note: no --env given, so this uses env staging (default_env in barca.toml); this project has 2 environments (prod, staging). Pass --env NAME or set BARCA_ENV; require_env = true in barca.toml (or BARCA_REQUIRE_ENV=1) makes it required

require_env = true at the top level of barca.toml (or BARCA_REQUIRE_ENV=1, which beats it; BARCA_REQUIRE_ENV=0 turns it off) makes the environment required: every command that reads or writes one (get, run, status, history, logs, stats, sql, gc, state and serve) exits 2 without --env or BARCA_ENV, naming the environments there are, even when default_env is set, and barca doctor reports it as a failed environment check. plan, list and docs read no environment and are not affected.

	env = default	named env <e>
local history	.barca/metadata.db	.barca/envs/<e>/metadata.db
local artifacts	.barca/artifacts/	.barca/envs/<e>/artifacts/
remote artifacts	{uri}/default/artifacts/	{uri}/<e>/artifacts/
remote history	{uri}/default/state/metadata.db	{uri}/<e>/state/metadata.db

barca get total --env dev
barca status total --env dev     # what is cached in dev
barca sql "select * from total" --env dev

An environment that has never been used has no results: barca status --env <e> reports every node as never_run, and barca sql "show tables" --env <e> lists nothing. Neither is an error, so a misspelled name is not reported.