createMemorySpecCache

import { createMemorySpecCache } from '@rudra-js/core';

Call createMemorySpecCache to keep the model's answers in your server's memory, with your own expiry and size limit.

createMemorySpecCache(options?: MemorySpecCacheOptions) => SpecCache

Reference

createMemorySpecCache(options?)

Parameters

• optionsoptional: How long entries last and how many the cache holds. Leave it out for the defaults.
  • ttlMsoptional: How long an entry lasts, in milliseconds. Defaults to 60000, which is one minute.
  • maxEntriesoptional: How many entries the cache holds. Defaults to 10000. Once it is full, the entry used longest ago goes first.
  • nowoptional: A function that returns the current time in milliseconds. Defaults to Date.now. Pass your own in a test to move time forward without waiting.

Returns

A SpecCache to pass to createComponentGenerator as cache.

Caveats

• Each server process keeps its own copy. Run several instances and each one calls the model for answers the others already have. To share one cache between them, pass your own SpecCache.
• Expiry is checked when an entry is read. An expired entry stays in memory until then, or until a newer one pushes it out.
• ttlMs and maxEntries have to be finite numbers of at least 0, and maxEntries a whole one. Anything else throws a RangeError. Watch for Number() on an environment variable that isn't set, which gives NaN.

Usage

import { createAnthropicProvider } from '@rudra-js/anthropic';
import { createComponentGenerator, createMemorySpecCache } from '@rudra-js/core';

export const generator = createComponentGenerator({
  provider: createAnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY! }),
  cache: createMemorySpecCache({ ttlMs: 10 * 60_000, maxEntries: 2_000 }),
});

Source on GitHub@rudra-js/core on npm