Cache

Cache is a built-in config property available on every step type (source, transformer, destination). It prevents redundant processing by caching results and serving them on subsequent matching requests.

Configuration

Add cache to any step in your flow config:

"sources": {
  "express": {
    "package": "@walkeros/server-source-express",
    "cache": {
      "rules": [
        {
          "match": { "key": "ingest.method", "operator": "eq", "value": "GET" },
          "key": ["ingest.method", "ingest.path"],
          "ttl": 300
        }
      ]
    }
  }
}

Cache Rule Properties

Property	Type	Description
match	MatchExpression	When to apply this cache rule
key	string[]	Dot-paths to build the cache key from
ttl	number	Time-to-live in seconds
update	Record<string, Value>	Optional modifications applied on read

match is optional, omit it for an always-match rule.

Cache Properties

Property	Type	Default	Description
stop	boolean	false (true for sources)	Stop flow on cache HIT and skip remaining steps, returning the cached value
store	string	Built-in memory	Reference to a store in flow.stores
namespace	string	(none)	Optional key prefix. Omit to write keys directly to the store. Same store + same key + same namespace = same cache entry.
rules	CacheRule[]	Required	At least one cache rule

Behavior by Step Type

Source Cache (stop by default)

Source cache intercepts requests before the pipeline runs. Sources default to stop: true. On cache HIT, the cached respond value is returned immediately and the pipeline is skipped. Set stop: false to skip only the source step and let the pipeline continue.

"sources": {
  "express": {
    "cache": {
      "rules": [
        { "match": { "key": "ingest.method", "operator": "eq", "value": "GET" }, "key": ["ingest.method", "ingest.path"], "ttl": 300 }
      ]
    }
  }
}

Transformer Cache (step-level)

Transformer cache skips the transformer's execution on HIT but continues the chain with the cached event. Useful for expensive computations like API enrichment. With stop: true, the remaining chain stops on HIT and downstream transformers and destinations are skipped.

Halt scopes

Transformer caches with stop: true in a source chain (source.before, source.next) or in the collector chain (collector.next) halt the event on HIT: no destination sees it. In a destination's before chain, a HIT skips the rest of that chain and the destination receives the cached event. Destination caches with stop: true halt only the per-destination delivery; sibling destinations still receive the event. To drop an event by rule rather than by cache, use a route stop (see Dropping an event with stop).

The dedup step of flow-complete.json has no code of its own, only a cache. It runs in the server source's next chain, so a hit with stop: true drops a resent event before the collector:

/flows/server/transformers/dedup/cache

{
  "stop": true,
  "namespace": "dedup",
  "rules": [
    {
      "key": [
        "event.id"
      ],
      "ttl": 3600
    }
  ]
}

A resend with the same event id is stopped for an hour. Guide chapter quality

/flows/server/transformers/dedup/cache/stop: true. stop: true turns a cache hit into a drop. Guide chapter quality

/flows/server/transformers/dedup/cache/namespace: "dedup". namespace keeps cache keys of different steps apart. Guide chapter quality

/flows/server/transformers/dedup/cache/rules

[
  {
    "key": [
      "event.id"
    ],
    "ttl": 3600
  }
]

rules say which events are cached, by which key, for how long. Guide chapter quality

/flows/server/transformers/dedup/cache/rules/0/key

[
  "event.id"
]

key is a list of paths; together they identify a duplicate. Guide chapter quality

/flows/server/transformers/dedup/cache/rules/0/ttl: 3600. ttl in seconds. Guide chapter quality

The first event with this id passes and its id is remembered for an hour. A resend with the same id is stopped (second request, proven in the flow test).

Event

{
  "name": "order complete",
  "entity": "order",
  "action": "complete",
  "data": {
    "id": "ORD-100",
    "total": 149.8,
    "currency": "EUR",
    "coupon": "WELCOME10"
  },
  "id": "ev-order-complete",
  "trigger": "load",
  "timestamp": 1700000000000,
  "consent": {
    "functional": true,
    "marketing": true
  },
  "globals": {
    "language": "en",
    "pageGroup": "checkout"
  },
  "user": {
    "device": "d3v1c3",
    "session": "s3ss10n",
    "id": "cust-42",
    "email": "jane@example.com"
  },
  "source": {
    "type": "express",
    "platform": "server",
    "url": "https://www.example.com/checkout/thanks"
  },
  "nested": [
    {
      "entity": "product",
      "data": {
        "id": "SKU-1",
        "name": "Trail Runner",
        "price": 129.9,
        "quantity": 1
      }
    },
    {
      "entity": "product",
      "data": {
        "id": "SKU-2",
        "name": "Running Socks",
        "price": 9.95,
        "quantity": 2
      }
    }
  ]
}

Out

return {
  "event": {
    "name": "order complete",
    "entity": "order",
    "action": "complete",
    "data": {
      "id": "ORD-100",
      "total": 149.8,
      "currency": "EUR",
      "coupon": "WELCOME10"
    },
    "id": "ev-order-complete",
    "trigger": "load",
    "timestamp": 1700000000000,
    "consent": {
      "functional": true,
      "marketing": true
    },
    "globals": {
      "language": "en",
      "pageGroup": "checkout"
    },
    "user": {
      "device": "d3v1c3",
      "session": "s3ss10n",
      "id": "cust-42",
      "email": "jane@example.com"
    },
    "source": {
      "type": "express",
      "platform": "server",
      "url": "https://www.example.com/checkout/thanks"
    },
    "nested": [
      {
        "entity": "product",
        "data": {
          "id": "SKU-1",
          "name": "Trail Runner",
          "price": 129.9,
          "quantity": 1
        }
      },
      {
        "entity": "product",
        "data": {
          "id": "SKU-2",
          "name": "Running Socks",
          "price": 9.95,
          "quantity": 2
        }
      }
    ]
  }
}

First delivery passes: example firstDelivery of step dedup. Guide chapter quality

Destination Cache (deduplication)

Destination cache skips the push on HIT, preventing duplicate events from reaching the destination. Other destinations still receive the event. With stop: true, the before transformer chain is also skipped on HIT.

A thank-you page reload sends the same order again with a new event id, so the dedup step above lets it through. Meta dedups on the order id instead, and the rule's match limits the cache to order complete:

/flows/server/destinations/meta/cache

{
  "stop": true,
  "rules": [
    {
      "match": {
        "key": "event.name",
        "operator": "eq",
        "value": "order complete"
      },
      "key": [
        "event.data.id"
      ],
      "ttl": 86400
    }
  ]
}

A thank-you page reload gets a new event id for the same order; Meta gets each order once. Guide chapter quality

/flows/server/destinations/meta/cache/rules/0/match

{
  "key": "event.name",
  "operator": "eq",
  "value": "order complete"
}

A rule match limits the cache to order complete. Guide chapter quality

Key Resolution

Cache keys use dot-path resolution from a structured context:

• ingest.method: request method from source ingest
• ingest.path: request path from source ingest
• event.name: event name
• event.data.id: nested event data field
• event.user.loggedIn: user properties

Update Rules

Use update to modify the result on both HIT and MISS:

"cache": {
  "rules": [{
    "key": ["ingest.method", "ingest.path"],
    "ttl": 300,
    "update": {
      "headers.X-Cache": { "key": "cache.status" },
      "headers.Cache-Control": { "value": "public, max-age=300" }
    }
  }]
}

cache.status resolves to "HIT" or "MISS". Static values use { "value": "..." }.

Custom Store

By default, cache uses a shared in-memory store. Reference a custom store for persistence:

{
  "stores": {
    "files": {
      "package": "@walkeros/server-store-fs",
      "config": { "settings": { "basePath": "./cache" } }
    }
  },
  "sources": {
    "express": {
      "cache": {
        "store": "files",
        "rules": [...]
      }
    }
  }
}

The filesystem store keeps entries on the local disk of one instance. To share cache entries across instances, use @walkeros/server-store-s3 or @walkeros/server-store-gcs instead. The Sheets store cannot be a cache tier (a cache.store target); a cache can still wrap it.

The filesystem store rejects a key that begins with /, so start key with a field such as ingest.method, or set namespace.

A store can carry a cache of its own. In the complete example, reads from the customers store are memoized for five minutes:

/flows/server/stores/customers/cache

{
  "rules": [
    {
      "ttl": 300
    }
  ]
}

Store cache memoizes customer reads for five minutes. Guide chapter state-stores

Custom stores can be either synchronous (in-memory) or asynchronous (file system, S3, GCS, the cache wrapper). Cache lookups await the underlying store.get internally, so HIT/MISS semantics behave identically regardless of which backing you wire.

Migration from prior v4 cache

If you're upgrading from an earlier v4 build, three breaking changes apply:

• cache.full is now cache.stop. Search-and-replace "full" for "stop" in every cache block. Default behavior unchanged (sources default to stop: true, transformers and destinations default to stop: false).
• cache.match: "*" is now omitted. Drop the "match": "*" entry from rules. A rule with no match matches every event, the behavior is identical to the previous wildcard.
• Implicit per-step namespacing is gone. Previous builds prefixed cache keys with s:, t:, or d: based on step type, so a source and a destination writing the same key never collided. That prefix is no longer applied. If you share a store across step types and need isolation, set cache.namespace explicitly (for example "namespace": "source").