Cache

Cache is a built-in config property available on every step type (source, transformer, destination). It prevents redundant processing by caching results and serving them on subsequent matching requests.

Configuration

Add cache to any step in your flow config:

"sources": {
  "express": {
    "package": "@walkeros/server-source-express",
    "cache": {
      "rules": [
        {
          "match": { "key": "ingest.method", "operator": "eq", "value": "GET" },
          "key": ["ingest.method", "ingest.path"],
          "ttl": 300
        }
      ]
    }
  }
}

Cache Rule Properties

Property	Type	Description
match	MatchExpression	When to apply this cache rule
key	string[]	Dot-paths to build the cache key from
ttl	number	Time-to-live in seconds
update	Record<string, Value>	Optional modifications applied on read

match is optional, omit it for an always-match rule.

Cache Properties

Property	Type	Default	Description
stop	boolean	false (true for sources)	Stop flow on cache HIT and skip remaining steps, returning the cached value
store	string	Built-in memory	Reference to a store in flow.stores
namespace	string	(none)	Optional key prefix. Omit to write keys directly to the store. Same store + same key + same namespace = same cache entry.
rules	CacheRule[]	Required	At least one cache rule

Behavior by Step Type

Source Cache (stop by default)

Source cache intercepts requests before the pipeline runs. Sources default to stop: true. On cache HIT, the cached respond value is returned immediately and the pipeline is skipped. Set stop: false to skip only the source step and let the pipeline continue.

"sources": {
  "express": {
    "cache": {
      "rules": [
        { "match": { "key": "ingest.method", "operator": "eq", "value": "GET" }, "key": ["ingest.method", "ingest.path"], "ttl": 300 }
      ]
    }
  }
}

Transformer Cache (step-level)

Transformer cache skips the transformer's execution on HIT but continues the chain with the cached event. Useful for expensive computations like API enrichment. With stop: true, the remaining chain stops on HIT and downstream transformers and destinations are skipped.

Halt scopes

Pre-collector transformer caches with stop: true halt the entire pipeline on HIT, destinations do not see the event. Destination caches with stop: true halt only the per-destination delivery, sibling destinations still receive the event.

"transformers": {
  "enricher": {
    "code": {
      "type": "enricher",
      "push": "$code:async (event) => ({ event: { ...event, data: { ...event.data, enriched: true } } })"
    },
    "cache": {
      "rules": [
        { "key": ["event.name"], "ttl": 60 }
      ]
    }
  }
}

Destination Cache (deduplication)

Destination cache skips the push on HIT, preventing duplicate events from reaching the destination. Other destinations still receive the event. With stop: true, the before transformer chain is also skipped on HIT.

"destinations": {
  "gtag": {
    "package": "@walkeros/web-destination-gtag",
    "cache": {
      "rules": [
        { "key": ["event.name", "event.data.id"], "ttl": 300 }
      ]
    }
  }
}

Key Resolution

Cache keys use dot-path resolution from a structured context:

• ingest.method: request method from source ingest
• ingest.path: request path from source ingest
• event.name: event name
• event.data.id: nested event data field
• event.user.loggedIn: user properties

Update Rules

Use update to modify the result on both HIT and MISS:

"cache": {
  "rules": [{
    "key": ["ingest.method", "ingest.path"],
    "ttl": 300,
    "update": {
      "headers.X-Cache": { "key": "cache.status" },
      "headers.Cache-Control": { "value": "public, max-age=300" }
    }
  }]
}

cache.status resolves to "HIT" or "MISS". Static values use { "value": "..." }.

Custom Store

By default, cache uses a shared in-memory store. Reference a custom store for persistence:

{
  "stores": {
    "files": {
      "package": "@walkeros/server-store-fs",
      "config": { "settings": { "basePath": "./cache" } }
    }
  },
  "sources": {
    "express": {
      "cache": {
        "store": "files",
        "rules": [...]
      }
    }
  }
}

The filesystem store keeps entries on the local disk of one instance. To share cache entries across instances, use @walkeros/server-store-s3 or @walkeros/server-store-gcs instead. The Sheets store cannot hold cache entries.

Custom stores can be either synchronous (in-memory) or asynchronous (file system, S3, GCS, the cache wrapper). Cache lookups await the underlying store.get internally, so HIT/MISS semantics behave identically regardless of which backing you wire.

Migration from prior v4 cache

If you're upgrading from an earlier v4 build, three breaking changes apply:

• cache.full is now cache.stop. Search-and-replace "full" for "stop" in every cache block. Default behavior unchanged (sources default to stop: true, transformers and destinations default to stop: false).
• cache.match: "*" is now omitted. Drop the "match": "*" entry from rules. A rule with no match matches every event, the behavior is identical to the previous wildcard.
• Implicit per-step namespacing is gone. Previous builds prefixed cache keys with s:, t:, or d: based on step type, so a source and a destination writing the same key never collided. That prefix is no longer applied. If you share a store across step types and need isolation, set cache.namespace explicitly (for example "namespace": "source").