modelProviders

Adds an AI model that Switchboard can ask wherever it asks one, answered by the extension's code.

Details #

A model provider lets your extension be the model. Switchboard hands it a request (instructions, a prompt and sometimes a JSON Schema) and your code answers with text. Use it to reach a model on your own network, such as Ollama, or any service Switchboard has no built-in support for.

Declare it in switchboard.json under adds.modelProviders. Every entry has an id (lowercase, digits and . _ -, up to 64 characters, starting with a letter), a title (1 to 80 characters) and an optional description. Ids must be unique within the list.

{
  "adds": {
    "modelProviders": [
      { "id": "local", "title": "My local model" }
    ]
  }
}

import { defineExtension, modelProvider } from '@switchboard/sdk';

export default defineExtension({
  modelProviders: [
    modelProvider('local', {
      generate: async ({ instructions, prompt }) => ({ text: await askMyModel(instructions, prompt), model: 'local-1' }),
    }),
  ],
});

Gotchas #

A call that takes more than 60 seconds without answering fails.
Network access needs a permission in the manifest; without it the call fails with permission denied. lan: true allows hosts on your local network over http; a hosted service needs its exact host under network.hosts, over https.
ctx.fetch cannot reach localhost or 127.0.0.1 over http. Run the model server on a .local name or a private address.
Unknown keys in an entry are refused.

The sample below can be downloaded from the Full example section and edited into your own. It talks to an Ollama server.

The handler #

generate gets instructions, prompt and, when the caller wants structured output, schema (a JSON Schema). It returns the text, or { text, model?, tokensIn?, tokensOut? }. The engine records model and the token counts when you give them. An empty answer counts as a failure, and so does a thrown error.

Inside the engine the provider is labelled ext:<extension id>.<id>.

Settings and secrets #

A provider usually needs to know where to ask. Add a settingsPane for the server and model name and read them with ctx.settings.get(...). Keep an API key in a secret field and read it with ctx.secrets.get(...). Never write a key into the code.