and syntax of configuration, see [bees configuration format
reference](config-format.md).
+## Writing a configuration file
+
+A configuration file only needs to contain the keys you want to change.
+Every key you leave out takes its built-in default, so the typical file is
+short. Start from an empty file, not from a copy of the built-in
+configuration: `--show-builtin-config` output is a reference to read, and
+pasting it into a file gives you hundreds of keys to keep in sync with
+future defaults for no benefit. It also makes a common mistake easy: a
+key that appears twice in one file is a **duplicate key** error, and a copied
+file with your own `[filter.videos]` added at the end has every `[filter.*]`
+key twice.
+
+A minimal global configuration file (`/etc/bees/bees.conf`) looks like this:
+
+```ini
+[config]
+ version = CURRENT
+
+[state.hash]
+ size = 1G
+```
+
+The `[config]` section is optional. Leave it out to float to each new
+release's defaults, or set `version` to pin them: `CURRENT` for this
+build's schema, `STABLE` for the last released one, or a release alias such
+as `v0.11` to freeze the defaults that release shipped. An empty `[config]`
+section is legal and does nothing. See [`version`](#options) below.
+
+Sections are named by dotted key prefix, so these two spellings are the same
+key and you can use whichever reads better:
+
+```ini
+[state.hash]
+ size = 1G
+
+state.hash.size = 1G
+```
+
+The same syntax works on the command line. `--option` (`-o`) takes any
+text the configuration file would accept, including section headers and
+multiple lines, so a whole rule can go in one shell-quoted argument:
+
+```sh
+bees -o '[filter.videos]
+ name-pattern = [.](mp4|mkv|webm)$
+ match-action = REJECT' /mnt/fs
+```
+
+Keys given by `--option` and `--config` files are all treated as one file,
+so the same duplicate-key rule applies across them.
+
+Check what bees will actually use with `--show-config ROOT_PATH`, where
+`ROOT_PATH` is the filesystem you will run bees on: it merges the built-in
+defaults, the global and local files, and the command line, and prints the
+result. It needs the filesystem because some values depend on it: the
+`${ROOT}`, `${UUID}`, `${LABEL}` and `${FS_BYTES}` interpolations, the
+choice of local configuration file, and limits checked against the
+filesystem block size.
+
+### Example: exclude files by extension
+
+Define a rule, then reference it from the extent scanner's filter chain.
+**A rule that is not referenced by a chain is never evaluated**; defining
+`[filter.videos]` on its own does nothing.
+
+```ini
+[filter.videos]
+ name-pattern = [.](mp4|mkv|webm)$
+ match-action = REJECT
+
+[scan.extent.*]
+ method = 1
+ filter = common.* videos ACCEPT
+```
+
+The chain lists rule names without the `filter.` prefix: `videos` here is
+`[filter.videos]`, and `common.*` is every `[filter.common.NAME]` rule.
+The `ACCEPT` at the end accepts whatever no rule rejected. The pattern is
+a regular expression by default (`pattern-type = regexp`); `[.]` is used
+instead of `\.` because configuration values process backslash escapes
+before the pattern sees them, so `\.` would need to be written `\\.`.
+See [Value escape sequences](config-format.md#value-escape-sequences).
+
+> **Filters need `method = 1`.** Filter chains are only evaluated by the
+> `scan_next` extent scanner. With the default `method = 0` the chain is
+> parsed and validated but never run, so every extent is accepted and
+> nothing in the `filter_*` [event counters](event-counters.md#filter)
+> moves. The legacy `--scan-mode 4` option selects extent scanning but does
+> not set `method`, so set it in the configuration file. The default is
+> expected to change to `1` in a future release.
+
+### Example: exclude a subvolume
+
+`name-scope = subvol` matches the path of the subvolume containing each
+reference, from the filesystem root:
+
+```ini
+[filter.no-backups]
+ name-scope = subvol
+ name-pattern = ^/backups(/|$)
+ match-action = REJECT
+
+[scan.extent.*]
+ method = 1
+ filter = common.* no-backups ACCEPT
+```
+
+### Example: use a different hash function
+
+Add a hash domain with a higher `priority` than the built-in `[hash.crc64]`
+(priority 100). The name after `hash.` is a label of your choosing:
+
+```ini
+[hash.fast]
+ function = xxhash3
+ priority = 200
+```
+
+Changing the function makes an existing `beeshash.dat` useless until it is
+repopulated; see [hash domain notes](#hash-domain-notes).
+
+### Checking that a filter does what you meant
+
+The `filter_*` [event counters](event-counters.md#filter) in the stats
+output count every verdict by role and by the rule that decided it:
+`filter_dst_reject_videos` going up means the `videos` rule above is
+rejecting extents. If only `filter_*_accept_ACCEPT` moves, no rule is
+matching — check the pattern, the chain spec, and `method`. For
+per-extent detail, set `[log.filter] level = 7` (one line per
+evaluation) or `8` (condition-by-condition trace); see
+[log categories](#categories).
+
## [config] section
The `[config]` section defines where bees looks for configuration files and how defaults are versioned. Configuration is layered in order of increasing precedence:
implementation and rejected with a clear message.
* **`priority`**
- Lookup priority. The highest-priority insert-enabled domain becomes the
- global hash function; ties are broken by config-file order. `0` disables
- lookup for the domain.
+ Lookup priority, a non-negative integer. Domains are consulted in
+ descending priority order. The highest-priority insert-enabled domain
+ becomes the global hash function; ties are broken by config-file order.
+ `0` disables lookup for the domain.
Default: `100`.
+ The number is an ordering, not a weight or a capability check: bees does
+ not probe the host for hardware support, and `200` is not "twice as
+ preferred" as `100`. Any value above the built-in `[hash.crc64]` domain's
+ `100` selects your domain instead.
+
+ On this release only the top-priority domain is used, so `priority`
+ amounts to a selector. It is defined as an ordering because of where the
+ design is going: a lookup will try the highest-priority domain first and,
+ on a miss, fall back to the next. That allows two things a single
+ function cannot. One is reading hashes from the btrfs csum tree, which
+ needs a different function for compressed data than for uncompressed
+ data. The other is migrating from one function to another without
+ discarding the hash table: a new domain at higher priority takes over
+ inserts while the old one, at lower priority, keeps serving lookups for
+ entries it wrote until they are evicted.
+
* **`step`**
Distance in bytes between the start of one hash window and the next.
Must equal the btrfs checksum block size, `4096`.
Maximum physical extent size for this tier. Extents larger than this are skipped by the scanner. Accepts [size values](config-format.md#size-values); `max` means no upper bound. Default: `max`.
* **`filter`**
- Space-separated filter chain spec applied to each extent in this tier before deduplication. The chain is evaluated left to right; tokens are filter section names or the terminal keywords `ACCEPT` and `REJECT`. The default wildcard value `common.* ACCEPT` runs the common rules and accepts anything that passes through them. See [Filter chain ordering](#filter-chain-ordering).
+ Space-separated filter chain spec applied to each extent in this tier before deduplication. The chain is evaluated left to right; tokens are filter section names or the terminal keywords `ACCEPT` and `REJECT`. Section names are given without the `filter.` prefix, so `common.*` names every `[filter.common.NAME]` section and `videos` names `[filter.videos]`. The default wildcard value `common.* ACCEPT` runs the common rules and accepts anything that passes through them. See [Filter chain ordering](#filter-chain-ordering).
Default (from wildcard): `common.* ACCEPT`.
+ > **Only evaluated when `method = 1`.** With `method = 0` the chain is
+ > parsed and validated but never run. See `method` below.
+
* **`method`**
- Temporary selector for extent-scan implementation. Default: `0`.
+ Selector for the extent-scan implementation. Default: `0`.
Inherited from `[scan.extent.*]` if not set per tier.
- `0` selects the production `scan_one_extent` path.
- `1` selects the temporary `scan_next_extent` hook for integration tests.
+ `0` selects the `scan_one_extent` path, which does not evaluate filter
+ chains.
+ `1` selects the `scan_next_extent` path, which is the only one that
+ evaluates the tier's `filter` chain. **Set `method = 1` whenever you
+ configure filters**; without it, filter rules are silently inert. The
+ legacy `--scan-mode 4` option does not set `method`. The default is
+ expected to change to `1` in a future release, at which point this key
+ becomes a compatibility fallback.
* **`first-transid`**
Where this scanner starts **the first time the tier is created** — that
Each named filter rule (`[filter.NAME]`) corresponds to one evaluation step in a filter chain. Rules are combined into chains by the scanner configuration. Within a chain, rules are evaluated in order; each rule has a **match action** (applied when all conditions match) and a **no-match action** (applied otherwise).
+> **Two things a rule needs before it does anything.** First, a chain has
+> to reference it: a `[filter.NAME]` section that no
+> [`scan.extent.*.filter`](#wildcard-defaults-scanextent) chain names is
+> parsed and validated but never evaluated. Second, the extent scanner has
+> to be running with `method = 1`; the default `method = 0` does not
+> evaluate filter chains at all. See
+> [Writing a configuration file](#writing-a-configuration-file) for a
+> complete example, and the `filter_*`
+> [event counters](event-counters.md#filter) to confirm a rule is deciding
+> verdicts.
+
### Wildcard defaults: [filter.*]
The special section `[filter.*]` sets default values inherited by all named filter sections. A key defined in `[filter.*]` applies to every `[filter.NAME]` section that does not define its own value for that key.
projects / bees / commitdiff