Core - v1.0.0
    Preparing search index...

    Interface InferenceEngine<TRuntime>

    Text generation and embedding.

    Which model answers is a configuration decision, not a call-site one, so the model argument is optional everywhere and callers normally omit it. That is what lets a deployment change models without touching a business service.

    Inference is expensive and frequently optional — an implementation whose Capability.isAvailable is false should be checked before calling rather than relied on to degrade, since there is no sensible empty answer to a prompt.

    Bayu Dwiyan Satria

    1.0.0

    1.0.0

    interface InferenceEngine<TRuntime = unknown> {
        ask(runtime: TRuntime, prompt: string, model?: string): Promise<string>;
        embed(runtime: TRuntime, text: string, model?: string): Promise<number[]>;
        isAvailable(runtime: TRuntime): boolean;
    }

    Type Parameters

    • TRuntime = unknown

      The runtime handle the engine is resolved from.

    Hierarchy (View Summary)

    Index
    ask embed isAvailable

    • Runs a prompt and returns the generated text.

      Parameters

      • runtime: TRuntime

        The runtime handle.

      • prompt: string

        The prompt.

      • Optionalmodel: string

        Model to run. Falls back to the configured default.

      Returns Promise<string>

      The generated text.

    • Embeds text as a vector, for indexing or similarity search.

      Parameters

      • runtime: TRuntime

        The runtime handle.

      • text: string

        The text to embed.

      • Optionalmodel: string

        Embedding model to run. Falls back to the configured default.

      Returns Promise<number[]>

      The embedding.

    • Reports whether the capability is actually usable on this runtime.

      Parameters

      Returns boolean

      true when calls will reach a real backing resource.

      A capability backed by an optional resource answers false when that resource was never provisioned. Implementations must document what each method does in that state — degrade or throw — and must not vary it, since a caller choosing between two implementations is relying on the answer meaning the same thing in both.

    Settings
    Member Visibility

    On This Page
    Methods