Baseten

To add the model in the UI, follow these steps. The steps create the same model as the Config file tab.

1. Set your API key in the BASETEN_API_KEY environment variable. The UI checks that the variable exists when you save the model.

   export BASETEN_API_KEY='<your-api-key>'

2. Create a configuration file with an empty llm section, and start agentgateway in the same terminal. Without an llm section, the UI does not show the LLM > Models page.

   cat > config.yaml <<'EOF'
   # yaml-language-server: $schema=https://agentgateway.dev/schema/config
   llm:
     models: []
   EOF
   agentgateway -f config.yaml

3. Open LLM > Models, such as http://localhost:15000/ui/llm/models, and click Add model.

4. From the Provider dropdown, select Baseten.

5. Keep the default baseten/* in the Incoming model match field.

6. For the Provider API key, select Env var, and enter BASETEN_API_KEY.

7. For Outgoing model, keep Strip baseten/. This option removes the baseten/ prefix before the request is forwarded. To send every request to one model instead, select Explicit, and enter the model name. This option sets params.model.

   

   

8. Click Save model.

The form has no field for params.baseUrl. To override the base URL, click the Raw icon (<>) in the drawer header and add the field, or use the Config file tab.

Whether the UI can save depends on your storage mode. For more information, see Save changes from the UI.

Review the following example configuration.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "baseten/*"
    provider: baseten
    params:
      apiKey: "$BASETEN_API_KEY"
    transformation:
      model: llmRequest.model.stripPrefix("baseten/")

Review the following table to understand this configuration.