removable media), because the local config can still affect settings outside
the `[paths]` section.
-If the `local-config-filename` is an absolute path starting with `/`, the local configuration is _trusted_, and may set options in the `[paths]` and `[reports.*]` sections.
+If the `local-config-filename` is an absolute path starting with `/`, the local configuration is _trusted_, and may set options in the `[path]` and `[report.*]` sections.
If the `local-config-filename` is a relative path not starting with `/`, the
local configuration is _untrusted_, and may _not_ set options in the
-`[paths]` or `[reports.*]` sections. It may still set other options that are allowed in local
+`[path]` or `[report.*]` sections. It may still set other options that are allowed in local
configuration files.
If you know what you're doing--or want to throw caution to the wind--use `${ROOT}` to set an absolute path through the mount point of the target filesystem. e.g.
startup if `openat2` is unavailable rather than fall back to `openat(2)`,
which cannot enforce `RESOLVE_BENEATH` or `RESOLVE_NO_SYMLINKS`.
* Because `O_TMPFILE` inodes are anonymous, `dir` does not need to be listed
- in the trusted `[paths]` section — an attacker cannot redirect bees's
+ in the trusted `[path]` section — an attacker cannot redirect bees's
persistent writes by setting this option.
* **`compression`**
level = 7 # raise planner detail when investigating dedup
```
-## [paths] section
+## [path] section
-The `[paths]` section defines writable directories used by bees for persistent state and reporting. This includes the on-disk hash table, crawl checkpoints, status reports, and other data required to resume deduplication after a restart.
+The `[path]` section defines writable directories used by bees for persistent state and reporting. This includes the on-disk hash table, crawl checkpoints, status reports, and other data required to resume deduplication after a restart.
* **`home`** *(legacy `$BEESHOME`)*
Path to the directory where bees stores its persistent state for the target filesystem.
* If the path is **absolute**, it refers to the exact directory on the host system.
* **`reports`**
- Base directory for status and statistics report files configured by `[reports.*]`.
+ Base directory for status and statistics report files configured by `[report.*]`.
* Default: `/run/bees`
* Must be empty or an absolute path. If non-empty it must be absolute, and
is caught rather than silently ignored.
* May be set empty to decline a base directory entirely. This is only valid
if every active report uses an absolute filename; an active report with a
- relative filename and an empty `paths.reports` is a startup error.
+ relative filename and an empty `path.reports` is a startup error.
* Substitution variables `${UUID}`, `${LABEL}`, and `${BASENAME}` are expanded.
* This directory is created if an active report with a relative filename
needs it. Only the final path component is created (a plain `mkdir()`,
* Using an **absolute path** may be appropriate if state should be kept outside the target filesystem.
* Make sure the `home` directory is writable by bees; otherwise persistence will fail and bees may restart from scratch.
-## [reports.*] sections
+## [report.*] sections
-The `[reports.*]` sections define periodic status files. Each named section
-under `reports` creates one possible output file; for example,
-`[reports.stats]` configures the `stats` report.
+The `[report.*]` sections define periodic status files. Each named section
+under `report` creates one possible output file; for example,
+`[report.stats]` configures the `stats` report.
-The special wildcard section `[reports.*]` supplies defaults inherited by all
+The special wildcard section `[report.*]` supplies defaults inherited by all
named reports. A report is written only when `active = yes` and `filename` is
not empty.
* **`active`**
Enables this report. Default: `no`.
The legacy `$BEESSTATUS` environment variable overrides
- `reports.legacy.filename` and activates `reports.legacy`.
+ `report.legacy.filename` and activates `report.legacy`.
* **`interval`**
Target interval between rewrites. Accepts [duration values](config-format.md#duration-values).
If the filename begins with `/`, the file is written relative to the
mount namespace root (`/`) when bees starts up, before any mount
namespace change inside the bees process; otherwise, the file is
- written relative to `paths.reports`.
+ written relative to `path.reports`.
The filename may contain `/` and multiple path components; each intermediate
directory must already exist (bees does not create them). A symlink in an
### Built-in reports
The built-in config defines these named report sections. They are inactive
-unless explicitly enabled, except that `$BEESSTATUS` activates
-`reports.legacy`.
+unless explicitly enabled, except that `$BEESSTATUS` activates `report.legacy`.
| Section | Filename | Interval | Mode | Content |
|---------|----------|----------|------|---------|
-| `[reports.legacy]` | `${UUID}.status` | `1s` | `0400` | `totals rates threads progress` |
-| `[reports.stats]` | `${UUID}.stats` | `10s` | `0444` | `totals rates` |
-| `[reports.activity]` | `${UUID}.activity` | `1s` | `0440` | `threads` |
-| `[reports.progress]` | `${UUID}.progress` | `1s` | `0444` | `progress` |
-| `[reports.tasks]` | `${UUID}.tasks` | `1s` | `0400` | `workers queue tasks` |
+| `[report.legacy]` | `${UUID}.status` | `1s` | `0400` | `totals rates threads progress` |
+| `[report.stats]` | `${UUID}.stats` | `10s` | `0444` | `totals rates` |
+| `[report.activity]` | `${UUID}.activity` | `1s` | `0440` | `threads` |
+| `[report.progress]` | `${UUID}.progress` | `1s` | `0444` | `progress` |
+| `[report.tasks]` | `${UUID}.tasks` | `1s` | `0400` | `workers queue idle tasks` |
## [state] section
The `[state]` section is the master switch for on-disk persistence. When
persistence is enabled, bees stores its hash table in `beeshash.dat` and its
-crawl checkpoint in `beespoint.ini` under the directory named by `paths.home`.
+crawl checkpoint in `beespoint.ini` under the directory named by `path.home`.
The `[state.hash]` and `[state.point]` subsections configure the two state
files in detail.
* `yes` (default) — bees creates the file at startup, sized to
`state.hash.size`.
* `no` — startup fails if the file is missing. Useful to detect a
- misconfigured `paths.home` before bees writes a fresh empty hash table
+ misconfigured `path.home` before bees writes a fresh empty hash table
to the wrong location.
* Has no effect when `state.persistent = no`.
These restrictions protect against abuse if the filesystem is untrusted
(e.g. removable media):
-* Writable path options (`[paths.*]`) are ignored in an untrusted local configuration file.
-* Report output options (`[reports.*]`) are ignored in an untrusted local configuration file.
+* Writable path options (`[path.*]`) are ignored in an untrusted local configuration file.
+* Report output options (`[report.*]`) are ignored in an untrusted local configuration file.
* Configuration file paths (`[config.*-config-filename]`) are ignored in all local configuration files.
* Symlinks are not permitted in the path to an untrusted local configuration file.
* An untrusted local configuration file must exist within the filesystem (i.e. escape from the filesystem with "../" is not allowed).
* beescrawl.dat | state of SEARCH_V2 crawlers. ASCII text. bees will create this.
* beesstats.txt | statistics and performance counters. ASCII text. bees will create this.
* BEESSTATUS: Compatibility shortcut for
- `reports.legacy.filename=$BEESSTATUS` and `reports.legacy.active=yes`.
+ `report.legacy.filename=$BEESSTATUS` and `report.legacy.active=yes`.
The legacy report contains a snapshot of current bees state: performance
counters and current status of each thread. The file is meant to be human
readable, but understanding it probably requires reading the source. You can
watch -n1 cat $BEESSTATUS
-Status reports can also be configured with `paths.reports` in the `[paths]`
-section and with `[reports.*]` sections in a
+Status reports can also be configured with `path.reports` in the `[path]`
+section and with `[report.*]` sections in a
[configuration file](config-file.md). Other options can be set there,
passed directly with `-o key=value` on the command line, or in some
cases changed by editing constants in `src/bees.h`. See
If bees or the filesystem seems to be stuck, check the configured status
reports. The legacy `$BEESSTATUS` environment variable activates
- `reports.legacy`; `[reports.activity]` and `[reports.tasks]` can provide
+ `report.legacy`; `[report.activity]` and `[report.tasks]` can provide
more detailed thread and task state. bees describes what it is doing (and
how long it has been trying to do it) through these files.
# optional per-filesystem config. Can use variables (BASENAME,
# UUID, LABEL, ROOT). Relative paths are treated as untrusted:
- # they cannot override [paths] or [reports.*], but they may
+ # they cannot override [path] or [report.*], but they may
# still affect other settings (for example [tempfile]).
local-config-filename = /etc/bees/uuid.d/${UUID}.conf
# >= this value are suppressed.
level = 5
-# The [paths] section defines paths that bees can write to.
+# The [path] section defines paths that bees can write to.
# This section is ignored in untrusted local configuration files
# (see `config.local-config-filename` for details).
-[paths]
+[path]
# Directory for hash table and crawl checkpoints (legacy `$BEESHOME`
# environment variable).
# report with a relative filename needs it.
reports = /run/bees
-# The [reports.*] sections define status output files written periodically
+# The [report.*] sections define status output files written periodically
# by bees. Each named section produces one output file.
#
-# All keys in [reports.*] are read from the trusted config only (absolute-path
+# All keys in [report.*] are read from the trusted config only (absolute-path
# local config, global config, or command line). Untrusted sources (relative-
# path local config) are ignored to prevent arbitrary file writes.
-[reports.*]
+[report.*]
# Whether this report output is active.
# Both active = yes and a non-empty filename are required to write.
- # The $BEESSTATUS environment variable overrides reports.legacy.filename
- # and activates reports.legacy regardless of this key.
+ # The $BEESSTATUS environment variable overrides report.legacy.filename
+ # and activates report.legacy regardless of this key.
active = no
# Target interval between rewrites of this report file.
# If the filename begins with '/', all leading slashes are stripped and
# it is resolved via openat(2) against the filesystem root ('/') opened
# at startup before any mount namespace change.
- # Otherwise it is resolved via openat(2) against the paths.reports directory.
+ # Otherwise it is resolved via openat(2) against the path.reports directory.
# Empty after interpolation = this output is disabled even if active = yes.
filename =
# tasks - task dependency tree
content =
-# [reports.legacy] reproduces the $BEESSTATUS status file. The default
+# [report.legacy] reproduces the $BEESSTATUS status file. The default
# filename /run/bees/${UUID}.status matches the beesd and OpenRC conventions.
-[reports.legacy]
+[report.legacy]
filename = ${UUID}.status
content = totals rates threads progress
mode = 0400
# active = yes
-# [reports.stats] reports only aggregate I/O statistics.
+# [report.stats] reports only aggregate I/O statistics.
-[reports.stats]
+[report.stats]
filename = ${UUID}.stats
interval = 10s
mode = 0444
# active = yes
-# [reports.activity] reports live thread state. May contain file paths from
+# [report.activity] reports live thread state. May contain file paths from
# the filesystem being deduplicated.
-[reports.activity]
+[report.activity]
filename = ${UUID}.activity
interval = 1s
# group = wheel
# active = yes
-# [reports.progress] reports scan progress (sizes and ETAs only).
+# [report.progress] reports scan progress (sizes and ETAs only).
-[reports.progress]
+[report.progress]
filename = ${UUID}.progress
interval = 1s
mode = 0444
# active = yes
-# [reports.tasks] reports the work queue, worker threads, and task dependency
+# [report.tasks] reports the work queue, worker threads, and task dependency
# tree. May contain task titles which include file paths.
-[reports.tasks]
+[report.tasks]
filename = ${UUID}.tasks
interval = 1s
content = workers queue idle tasks
mode = 0400
- # active = yes
# The [tempfile] section defines parameters for the anonymous temporary files
# that bees creates with O_TMPFILE during deduplication.
BEESTRACE("Opening home directory");
- const auto home_dir = m_config->get_trusted("paths.home", [](const string &s) { return s; } );
+ const auto home_dir = m_config->get_trusted("path.home", [](const string &s) { return s; } );
BEESTRACE("Opening home directory '" << home_dir << "'");
if (home_dir.at(0) == '/') {
// Absolute paths use plain openat (dirfd is ignored
graph_blob << m_ctx->get_progress();
BEESLOGINFO(graph_blob.str());
- // Skip the on-disk stats file when paths.home is unavailable
+ // Skip the on-disk stats file when path.home is unavailable
// (state.persistent=no with no home directory). The log copy
// above still fires so operators see the same content; only
// the file write is suppressed to avoid throwing from the
using namespace std;
using namespace crucible;
-/// Configuration for a single report output section (e.g. [reports.legacy]).
+/// Configuration for a single report output section (e.g. [report.legacy]).
struct BeesReportConfig {
string name; ///< Section name (e.g. "legacy").
bool active = false; ///< Whether this output is written.
Optional environment variables:
BEESHOME Path to hash table and checkpoint files
(default is .beeshome/ in the root of the filesystem).
- [paths.beeshome]
+ [path.home]
- BEESSTATUS Compatibility shortcut for reports.legacy.filename and
- reports.legacy.active=yes.
- [reports.legacy.filename] [reports.legacy.active]
+ BEESSTATUS Compatibility shortcut for report.legacy.filename and
+ report.legacy.active=yes.
+ [report.legacy.filename] [report.legacy.active]
const char *base_dir = getenv("BEESHOME");
if (base_dir) {
- bconfig.set("paths.home", base_dir);
+ bconfig.set("path.home", base_dir);
}
// $BEESSTATUS=/run/foo is equivalent to:
- // -o 'reports.legacy.filename = /run/foo'
- // -o 'reports.legacy.active = yes'
+ // -o 'report.legacy.filename = /run/foo'
+ // -o 'report.legacy.active = yes'
const char *beesstatus = getenv("BEESSTATUS");
if (beesstatus) {
- bconfig.set("reports.legacy.filename", beesstatus);
- bconfig.set("reports.legacy.active", "yes");
+ bconfig.set("report.legacy.filename", beesstatus);
+ bconfig.set("report.legacy.active", "yes");
}
// Have to block signals before we create a bunch of threads
const auto poll_max = bc->get_config().get("loop.poll-max", bees_parse_duration);
bc->roots()->set_poll_bounds(poll_min, poll_max);
- // Build the BeesReporter from [paths] and [reports.*] config
+ // Build the BeesReporter from [path] and [report.*] config
{
const auto reports_dir = bc->get_config().subst(
- bc->get_config().get_trusted("paths.reports", [](const string &s) { return s; }));
- BEESLOGINFO("paths.reports = '" << reports_dir << "' [paths.reports]");
+ bc->get_config().get_trusted("path.reports", [](const string &s) { return s; }));
+ BEESLOGINFO("paths.reports = '" << reports_dir << "' [path.reports]");
// Parse all report sections into a vector first
const auto parse_report_interval = [](const string &s) {
return interval;
};
vector<BeesReportConfig> configs;
- for (const auto &sec_name : bc->get_config().sections("reports")) {
- const auto prefix = string("reports.") + sec_name + ".";
+ for (const auto &sec_name : bc->get_config().sections("report")) {
+ const auto prefix = string("report.") + sec_name + ".";
BeesReportConfig cfg;
cfg.name = sec_name;
cfg.active = bc->get_config().get_trusted(prefix + "active", bees_parse_bool);
- BEESLOGINFO("reports." << sec_name << ".active = " << (cfg.active ? "yes" : "no")
- << " [reports." << sec_name << ".active]");
+ BEESLOGINFO("report." << sec_name << ".active = " << (cfg.active ? "yes" : "no")
+ << " [report." << sec_name << ".active]");
cfg.interval_s = bc->get_config().get_trusted(prefix + "interval", parse_report_interval);
cfg.filename = bc->get_config().subst(
bc->get_config().get_trusted(prefix + "filename", [](const string &s) { return s; }));
return mode_t(stoul(s, nullptr, 8));
});
if (cfg.active) {
- BEESLOGINFO("reports." << sec_name << ".interval = " << cfg.interval_s << "s"
- << " [reports." << sec_name << ".interval]");
- BEESLOGINFO("reports." << sec_name << ".filename = '" << cfg.filename << "'"
- << " [reports." << sec_name << ".filename]");
+ BEESLOGINFO("report." << sec_name << ".interval = " << cfg.interval_s << "s"
+ << " [report." << sec_name << ".interval]");
+ BEESLOGINFO("report." << sec_name << ".filename = '" << cfg.filename << "'"
+ << " [report." << sec_name << ".filename]");
ostringstream mode_oss;
mode_oss << '0' << oct << setw(3) << setfill('0') << static_cast<unsigned>(cfg.mode);
- BEESLOGINFO("reports." << sec_name << ".mode = " << mode_oss.str()
- << " [reports." << sec_name << ".mode]");
+ BEESLOGINFO("report." << sec_name << ".mode = " << mode_oss.str()
+ << " [report." << sec_name << ".mode]");
}
// user -> uid (name or numeric)
cfg.uid = uid_t(stoul(user_str));
} else {
const string user_name = user_str[0] == ':' ? user_str.substr(1) : user_str;
- BEESTRACE("looking up user '" << user_name << "' [reports." << sec_name << ".user]");
+ BEESTRACE("looking up user '" << user_name << "' [report." << sec_name << ".user]");
cfg.uid = GetPw(user_name).pw_uid;
}
if (cfg.active) {
- BEESLOGINFO("reports." << sec_name << ".user = '" << user_str << "'"
- << " [reports." << sec_name << ".user]");
+ BEESLOGINFO("report." << sec_name << ".user = '" << user_str << "'"
+ << " [report." << sec_name << ".user]");
}
}
cfg.gid = gid_t(stoul(group_str));
} else {
const string group_name = group_str[0] == ':' ? group_str.substr(1) : group_str;
- BEESTRACE("looking up group '" << group_name << "' [reports." << sec_name << ".group]");
+ BEESTRACE("looking up group '" << group_name << "' [report." << sec_name << ".group]");
cfg.gid = GetGr(group_name).gr_gid;
}
if (cfg.active) {
- BEESLOGINFO("reports." << sec_name << ".group = '" << group_str << "'"
- << " [reports." << sec_name << ".group]");
+ BEESLOGINFO("report." << sec_name << ".group = '" << group_str << "'"
+ << " [report." << sec_name << ".group]");
}
}
// content: space- or comma-separated token list
const auto content_str = bc->get_config().get_trusted(prefix + "content", [](const string &s) { return s; });
if (cfg.active) {
- BEESLOGINFO("reports." << sec_name << ".content = " << content_str
- << " [reports." << sec_name << ".content]");
+ BEESLOGINFO("report." << sec_name << ".content = " << content_str
+ << " [report." << sec_name << ".content]");
}
string tok;
for (const char c : content_str) {
}
}
- // paths.reports may be empty to decline a base directory entirely;
+ // path.reports may be empty to decline a base directory entirely;
// that is only valid if every active report uses an absolute
// filename. If it is set, it must be absolute even when no relative
// report filename actually uses it, so a misconfigured path is caught
// at startup rather than silently ignored.
if (!reports_dir.empty() && reports_dir.at(0) != '/') {
- THROW_ERROR(runtime_error, "paths.reports must be empty or an absolute path, got '" << reports_dir << "' [paths.reports]");
+ THROW_ERROR(runtime_error, "path.reports must be empty or an absolute path, got '" << reports_dir << "' [path.reports]");
}
// Open only the needed fds now, before any mount namespace changes.
Fd reports_path_fd;
if (needs_reports_path) {
if (reports_dir.empty()) {
- THROW_ERROR(runtime_error, "paths.reports is empty but a relative report filename requires it [paths.reports]");
+ THROW_ERROR(runtime_error, "path.reports is empty but a relative report filename requires it [path.reports]");
}
// Deliberately a single mkdir(), not mkdir -p: this only lets a
// system that has /run gain /run/bees without an external helper.
// Deeper hierarchies are left for the administrator to manage.
if (mkdir(reports_dir.c_str(), 0755) < 0 && errno != EEXIST) {
- THROW_ERRNO("mkdir paths.reports '" << reports_dir << "'");
+ THROW_ERRNO("mkdir path.reports '" << reports_dir << "'");
}
reports_path_fd = open_or_die(reports_dir, FLAGS_OPEN_DIR);
}