Cache

Cache is a built-in config property available on every step type (source, transformer, destination). It prevents redundant processing by caching results and serving them on subsequent matching requests.

Configuration

Add cache to any step in your flow config:

"sources": {
  "express": {
    "package": "@walkeros/server-source-express",
    "cache": {
      "rules": [
        {
          "match": { "key": "ingest.method", "operator": "eq", "value": "GET" },
          "key": ["ingest.method", "ingest.path"],
          "ttl": 300
        }
      ]
    }
  }
}

Cache Rule Properties

Property	Type	Description
match	MatchExpression | '*'	When to apply this cache rule
key	string[]	Dot-paths to build the cache key from
ttl	number	Time-to-live in seconds
update	Record<string, Value>	Optional modifications applied on read

Cache Properties

Property	Type	Default	Description
full	boolean	false (true for sources)	Stop flow on cache HIT and skip remaining steps, returning the cached value
store	string	Built-in memory	Reference to a store in flow.stores
rules	CacheRule[]	Required	At least one cache rule

Behavior by Step Type

Source Cache (full by default)

Source cache intercepts requests before the pipeline runs. Sources default to full: true. On cache HIT, the cached respond value is returned immediately and the pipeline is skipped. Set full: false to skip only the source step and let the pipeline continue.

"sources": {
  "express": {
    "cache": {
      "rules": [
        { "match": { "key": "ingest.method", "operator": "eq", "value": "GET" }, "key": ["ingest.method", "ingest.path"], "ttl": 300 }
      ]
    }
  }
}

Transformer Cache (step-level)

Transformer cache skips the transformer's execution on HIT but continues the chain with the cached event. Useful for expensive computations like API enrichment. With full: true, the remaining chain stops on HIT and downstream transformers and destinations are skipped.

"transformers": {
  "enricher": {
    "package": "@walkeros/server-transformer-enricher",
    "cache": {
      "rules": [
        { "match": "*", "key": ["event.name"], "ttl": 60 }
      ]
    }
  }
}

Destination Cache (deduplication)

Destination cache skips the push on HIT, preventing duplicate events from reaching the destination. Other destinations still receive the event. With full: true, the before transformer chain is also skipped on HIT.

"destinations": {
  "ga4": {
    "package": "@walkeros/web-destination-ga4",
    "cache": {
      "rules": [
        { "match": "*", "key": ["event.name", "event.data.id"], "ttl": 300 }
      ]
    }
  }
}

Key Resolution

Cache keys use dot-path resolution from a structured context:

• ingest.method: request method from source ingest
• ingest.path: request path from source ingest
• event.name: event name
• event.data.id: nested event data field
• event.user.loggedIn: user properties

Update Rules

Use update to modify the result on both HIT and MISS:

"cache": {
  "rules": [{
    "match": "*",
    "key": ["ingest.method", "ingest.path"],
    "ttl": 300,
    "update": {
      "headers.X-Cache": { "key": "cache.status" },
      "headers.Cache-Control": { "value": "public, max-age=300" }
    }
  }]
}

cache.status resolves to "HIT" or "MISS". Static values use { "value": "..." }.

Custom Store

By default, cache uses a shared in-memory store. Reference a custom store for persistence:

{
  "stores": {
    "redis": { "package": "@walkeros/server-store-redis" }
  },
  "sources": {
    "express": {
      "cache": {
        "store": "redis",
        "rules": [...]
      }
    }
  }
}