File
Local filesystem sink for walkerOS server flows. Appends events to a file as JSON Lines (JSONL), tab-separated values (TSV), or comma-separated values (CSV). Useful for debug logging, audit trails, replay sources, and lightweight local persistence without standing up a database or external service.
File is a server destination in the walkerOS flow:
Receives events server-side from the collector, serialises each event, and appends a line to a local file. No network calls, no SDKs, just `fs.createWriteStream` with `flag: 'a'`.
Installation
- Integrated
- Bundled
Configuration
This destination uses the standard destination config wrapper (consent, data, env, id, ...). For the shared fields see destination configuration. Package-specific fields live under config.settings and are listed below.
Settings
| Property | Type | Description | More |
|---|---|---|---|
filename | string | object | Output filename. Static string or Mapping.Value (e.g. { fn: "$code:..." } for daily rotation, { key: "data.tenant" } for sharding). | |
format | 'jsonl' | 'tsv' | 'csv' | Serialisation format. Defaults to jsonl. | |
fields | Array<string> | Event paths used as columns for tsv/csv formats. Object values are JSON-stringified. Required when format is tsv or csv. |
Mapping
This package does not define custom rule-level settings. For the standard rule fields (consent, condition, data, batch, name, policy) see mapping.
Examples
CSV with object cell
An event is written as a CSV row with the data object JSON-stringified and properly quoted for embedded commas and quotes.
Daily rotation
A mapping function derives a date-stamped filename from the event timestamp to rotate JSONL files daily.
JSONL append
An event is appended as a single JSON line to a static filename using default JSONL formatting.
Tenant sharding
The filename is resolved from an event field so events are partitioned into per-tenant JSONL files.
TSV log
An event is written as a tab-separated line selecting specific fields for a compact access-log style file.
The destination opens one long-lived WriteStream per resolved filename during init() (for static filenames) or on first matching event (for dynamic filenames). On flow hot-swap or server shutdown, destroy() closes all cached streams.
Filename templating
filename accepts either a static string or a standard walkerOS Mapping.Value. Common patterns:
Tenant sharding
Each event lands in a file named after the tenant value (e.g. acme, venti). Pair with a static suffix via fn: if you need a .jsonl extension.
Daily rotation
Inside the $code: function, value is the event being processed. One file per UTC day, created automatically. No logrotate, no cron.
Formats
JSONL (default)
One JSON object per line. The entire event is serialised with JSON.stringify. Easy to ingest with jq, DuckDB (read_json_auto), ClickHouse JSONEachRow, BigQuery external tables, and Athena.
TSV / CSV
Specify fields: string[] listing the event paths to extract as columns. Object values are JSON-stringified into a single cell. CSV output follows RFC 4180 quoting.
Limits
- One file handle is opened per resolved filename and kept open until
destroy(). Sharding by high-cardinality keys (e.g.user.session) can exhaust the OS file descriptor table. Be deliberate about cardinality. - External rotation (e.g.
logrotate) leaves the cached handle pointing at the rotated inode. Use the date-token pattern above instead. - No batching. Each event is written individually; Node's stream layer buffers under the hood.
- Write errors log a warning and drop the event. They never fail the flow.