> AI agents: For documentation discovery and navigation, see [llms.txt](/llms.txt).
# AI language models in VS Code
Visual Studio Code gives you access to multiple built-in language models, each optimized for different tasks. You can switch models for chat, inline suggestions, and utility tasks, and you can add more models by bringing your own API key.
For background on how language models work, their characteristics, and how to choose the right model, see [Language models concepts](/docs/agents/concepts/language-models.md).
## Choose how to access models
Start with the account, credentials, or model you already have:
* **GitHub Copilot access**: [Set up Copilot](/docs/setup/copilot.md), then [choose a model](#change-the-model-for-chat) from the providers available through your plan.
* **A provider account or harness-specific credentials**: Follow the [Anthropic Claude](/docs/agents/run/agent-harnesses.md#claude) or [OpenAI Codex](/docs/agents/run/agent-harnesses.md#codex) requirements for that harness. Supported accounts and authentication methods depend on the integration.
* **An API key or custom endpoint**: [Add a model provider](#bring-your-own-language-model-key) in the Language Models editor. This configures model access, not a different harness.
* **A locally running model**: Add a [model provider extension](#add-a-model-provider-extension), such as Ollama. Alternatively, check the hardware and sign-in requirements for [Copilot-managed on-device models](#on-device-model-requirements).
The model provides reasoning, the harness coordinates tools and workflow, and the access provider determines authentication and billing. These choices are related, but aren't interchangeable. Compare [harness-specific access options](/docs/agents/run/agent-harnesses.md#requirements-and-setup) before reusing an account or key.
## Change the model for chat
Different models have different strengths. Use a fast model for quick edits and simple questions, and a reasoning model for complex refactoring, architectural decisions, or multi-step tasks. Depending on the [harness](/docs/agents/concepts/agent-harnesses.md) you are using, the list of available models might differ.
Use the language model picker in the chat input field to change the model for chat conversations and code editing.
If the model picker shows **Models** instead of a model name, open it to check whether workspace trust or sign-in is required. Learn more about [workspace trust and Restricted Mode](/docs/editing/workspaces/workspace-trust.md#restricted-mode).
You can further extend the list of available models by [using your own language model API key](#bring-your-own-language-model-key).
> [!NOTE]
> If you are a Copilot Business or Copilot Enterprise user, your administrator needs to enable certain models for your organization by opting in to `Editor Preview Features` in the [Copilot policy settings](https://docs.github.com/en/enterprise-cloud@latest/copilot/managing-copilot/managing-github-copilot-in-your-organization/managing-policies-for-copilot-in-your-organization#enabling-copilot-features-in-your-organization) on GitHub.com.
To compare solutions to the same coding task, use [model comparison](/docs/agents/run/compare-models.md) in the Agents window. Run one prompt with multiple models or harnesses, then review the results side by side.
### Organization-managed model defaults
Your organization can set the starting model for new conversations in both the editor and the Agents window. The model picker marks the organization's choice as **Org default**, and the Settings editor shows an indicator for `setting(chat.defaultModel)` with a **Managed by organization** explanation.
The managed default takes precedence over your saved model preference for new conversations, but you can still select another available model for the current conversation. Existing conversations keep their model. Your saved preference is preserved and applies to new conversations again when the managed default is removed.
For administrator configuration, see [Set managed model defaults](/docs/enterprise/manage-ai-settings.md#set-managed-model-defaults).
## Configure thinking effort and context size
Some models support configurable thinking effort, context size, and other parameters.These control how much reasoning the model applies to each request, and how much context the model can consider at once. For background on thinking and reasoning, see [Thinking and reasoning](/docs/agents/concepts/language-models.md#thinking-and-reasoning).
By default, VS Code sets recommended effort levels and has adaptive reasoning enabled, where the model dynamically determines how much to think based on the complexity of each request. For most use cases, the default configuration works well.
> [!TIP]
> Higher thinking effort produces more thinking tokens, which increases [AI credit](/docs/agents/concepts/language-models.md#ai-credits-and-model-costs) consumption. Only increase thinking effort for genuinely complex tasks. Learn more about [optimizing AI credit usage](/docs/agents/guides/optimize-usage.md).
To configure model parameters:
1. Open the model picker in the chat input field and select a reasoning model.
1. Select the **>** arrow that appears next to the model name to open the model configuration submenu.
> [!NOTE]
> Non-reasoning models, such as GPT-4.1 and GPT-4o, do not show the thinking effort option in the model configuration submenu.
1. Select an effort level.
The model configuration is remembered per session, so revisiting an earlier session keeps the same effort level and context size it was using. New conversations start from the effort level and context size you last set for that model.
> [!NOTE]
> The `setting(github.copilot.chat.anthropic.thinking.effort)` and `setting(github.copilot.chat.responsesApiReasoningEffort)` settings are deprecated. Configure thinking effort directly from the language model picker.
## Use auto model selection
With auto model selection, VS Code evaluates task complexity and real-time model availability to route each request to the optimal model. For background on how auto model selection works, see [Auto model selection](/docs/agents/concepts/language-models.md#auto-model-selection).
To use auto model selection, select **Auto** from the model picker in chat.
In [Copilot harness sessions](/docs/agents/run/agent-harnesses.md#copilot), **Copilot Fusion** replaces **Auto** when dynamic Auto tiers are enabled and the service offers a Fusion tier your account can run. If the picker shows **Copilot Fusion**, follow the [Fusion instructions](#use-copilot-fusion) instead.
Use **Optimize for**, when available, to choose a tier that matches your priorities. The offered tiers depend on the service and can include:
* **Efficiency** favors lower AI credit consumption.
* **Balance** balances capability and credit consumption.
* **Intelligence** favors more capable models for complex tasks.
Your organization can set a starting tier for new chats. The managed tier appears as **Default** in the **Optimize for** menu, but you can still choose a different tier. An explicit choice is preserved when the managed default changes or is removed.
You can see which model is used for generating a response by hovering over the chat response.
## Use Copilot Fusion
Copilot Fusion chooses models and a workflow to complete your task, so you don't need to select a model for each step. For background on how Fusion differs from Auto and when to use it, see [Copilot Fusion concepts](/docs/agents/concepts/language-models.md#copilot-fusion).
For details about orchestration, effort levels, availability, and AI credit usage, see [GitHub Copilot Fusion](https://docs.github.com/en/copilot/concepts/models/copilot-fusion) in the GitHub documentation.
To use Copilot Fusion:
1. Verify that `setting(chat.copilot.dynamicAutoTiers.enabled)` is enabled in the Settings editor. Changing it restarts the Copilot runtime after active requests finish.
1. Start a new chat with the [Copilot harness](/docs/agents/run/agent-harnesses.md#copilot).
1. In the model picker, select **Copilot Fusion**, then choose an available tier: **Standard**, **High**, or **Max**.
Enabling the setting doesn't grant access to Fusion tiers. If the service doesn't offer a Fusion tier that your account can run, the picker continues to show **Auto** instead.
## Use on-device models
`feature(on-device-models)`
On-device models let you run a coding model provided by Copilot, currently MAI Code 1.1 Flash, directly on your device in Copilot harness sessions. VS Code uses [llama.cpp](https://github.com/ggml-org/llama.cpp) to serve the model on your device. On-device models bring that capability to Copilot in VS Code, without you having to find, download, and serve a model yourself.
You can use the local model alongside cloud models. [Auto-model selection](#use-auto-model-selection) can route suitable requests to the local model, with fallback to a cloud model.
On-device models differ from [BYOK local models](#bring-your-own-language-model-key). Copilot provides and manages on-device models, and they require GitHub sign-in. BYOK local models, such as Ollama, work without sign-in through a provider that you configure.
### On-device model requirements
* Sign in to GitHub with Copilot access. If you use Copilot Business or Enterprise, your administrator must enable `Editor Preview Features` in the [Copilot policy settings](https://docs.github.com/en/enterprise-cloud@latest/copilot/managing-copilot/managing-github-copilot-in-your-organization/managing-policies-for-copilot-in-your-organization#enabling-copilot-features-in-your-organization).
* Use a [Copilot harness session](/docs/agents/run/agent-harnesses.md) that runs on your local machine in the Chat view or [Agents window](/docs/agents/run/agents-window.md).
* Use an NVIDIA GPU with 96 GB or more of GPU memory (VRAM). Devices with unified memory, memory that the GPU shares with the system, count toward this requirement.
* Have about 55 GB of free disk space for MAI Code 1.1 Flash. The download is 52.8 GB, and setup requires 54.8 GB of free space. The details page in the model picker shows the exact size for your device.
* Install a llama.cpp build that supports the model. VS Code does not install llama.cpp. Get a build from the [llama.cpp releases page](https://github.com/ggml-org/llama.cpp/releases). MAI Code 1.1 Flash requires llama.cpp b11658 or later. If the build is too old, the model picker shows notification that a newer version is needed.
### Set up an on-device model
1. Install a llama.cpp build for your platform and GPU.
1. Set `setting(chat.copilot.localModel.llamaServerPath)` to the absolute path of the `llama-server` executable. This setting is machine-scoped. VS Code only runs the `llama-server` executable at this path.
1. Enable `setting(chat.copilot.localModel.enabled)` `feature(on-device-models)`. When you change this setting, the local Copilot runtime restarts after active responses finish.
1. In a Copilot harness session, open the model picker and select the **On This Device** tab. Models that are not downloaded appear dimmed with a **Download** action.
1. Select the model name to open its details, review the download size, source (Azure ML registry), and license, and then select **Download**.
1. Track progress in the notice above the chat input. You can keep using Copilot while the model downloads, and select **Pause** to pause the download.
VS Code stores models in `~/.copilot/azureml-models` and shares them with other Copilot apps, such as Copilot CLI.
### Use an on-device model in chat
After the download finishes, VS Code verifies the files and loads the model. Loading can take a few minutes, and the notice above the chat input shows the progress, with a **Cancel** action. When the model is ready, select **Switch to MAI Code 1.1 Flash** in the notice, or select the model from the **On This Device** tab.
When you select an on-device model, the model picker and the footer of each response show the model name followed by **On-device**. On-device responses don't use AI credits, and the response footer shows **0 credits**.
To change how the model runs, hover over the model in the **On This Device** tab and select **Details**. On the details page, you can:
* Choose the **Effort Level**: **Low**, **Medium**, or **High**.
* Choose the **Context** size. The available sizes depend on the GPU memory of your device, and the page shows the estimated memory for the selected size. Changing the context size reloads the model after running responses finish.
* Stop or remove the model by using the icons at the top of the page.
### Use an on-device model with auto-model selection
[Auto-model selection](#use-auto-model-selection) can route requests to the active on-device model, with cloud fallback. If the on-device model can't respond, model selection falls back to a cloud model before the first output. The change takes effect immediately.
To enable the local model for auto-model selection, enable **Use in Auto** on the model's details page, or enable `setting(chat.copilot.localModel.useInAuto)`.
Auto-model selection chooses the appropriate model for each request, so many requests continue to use cloud models. Responses that use the local model are labeled **On-device**. If a model is too slow for auto-model selection on your device, it shows **Too slow for Auto**. You can still select the model directly.
> [!CAUTION]
> Local models might not use the same safety filters as GitHub Copilot-provided cloud models and might be more susceptible to malicious instructions, especially when [actions are automatically approved](/docs/agents/run/approvals.md).
### Manage on-device models
Use these commands from the Command Palette:
* **Chat: Set Up On-Device Model...**
* **Chat: Make On-Device Model Active...**
* **Chat: Start On-Device Model**
* **Chat: Stop On-Device Model**
* **Chat: Update On-Device Model**
* **Chat: Resume On-Device Model Download**
* **Chat: Cancel On-Device Model Download**
* **Chat: Retry On-Device Model Setup**
* **Chat: Remove On-Device Model...**
Only one on-device model is loaded at a time. Loading another model unloads the active model. Removing a model asks for confirmation.
### Troubleshoot on-device models
If a model can't be set up or loaded, the **On This Device** tab shows a status, and the notice above the chat input offers actions such as **Get llama.cpp**, **Open Settings**, and **Retry**.
| Status | What to do |
|--------|------------|
| **Needs llama.cpp** | Install llama.cpp and set `setting(chat.copilot.localModel.llamaServerPath)` to its `llama-server` executable. |
| **llama.cpp is too old** or **Needs a newer llama.cpp** | Install a newer llama.cpp release, update the setting, and select **Retry**. |
| **GPU driver needed** | Install or update the NVIDIA GPU driver. |
| **Not enough GPU memory** | Choose a smaller context size, or use a device with more GPU memory. |
| **GPU memory busy** | Close other apps that use the GPU, and then select **Retry**. |
| **Running in another window** | The model is already running in another VS Code window. Use it there or stop it first. |
| **Not enough disk space** | Free up disk space, and then resume the download. |
| **Download interrupted** | Check your network connection, and then resume the download. |
| **Blocked by policy** | Your organization's policy blocks the llama.cpp executable. Contact your administrator. |
## Manage language models
You can use the language models editor to view all available models, choose which models are shown in the model picker, and add more models by adding from built-in providers or from extension-provided model providers.
To open the Language Models editor, open the model picker in the Chat view and select **Manage Language Models** (gear icon) or run the **Chat: Manage Language Models** command from the Command Palette. The Language Models editor opens by default in a [modal overlay](/docs/editing/getting-started/userinterface.md#modal-editors) on top of the editor area.
The editor lists all models available to you, showing key information such as the model capabilities, context size, billing details, and visibility status. By default, models are grouped by provider, but you can also group them by visibility.
You can search and filter models by using the following options:
* Text search with the search box
* Provider: `@provider:"OpenAI"`
* Capability: `@capability:tools`, `@capability:vision`, `@capability:agent`
* Visibility: `@visible:true/false`
### Customize the model picker
You can customize which models are shown in the model picker by changing the visibility status of models in the Language Models editor. You can show or hide models from any provider.
Hover over a model in the list and select the eye icon to show or hide the model in the model picker.
### Pin favorite models
Pin models to keep them in a fixed position at the top of the model picker. Pinned models appear in a dedicated **Pinned** section and don't shift around as you use other models.
To pin or unpin a model:
1. Open the model picker in the chat input field.
1. Hover over a model and select the pin icon to add it to the **Pinned** section.
1. To unpin a model, hover over it in the **Pinned** section and select the unpin icon.
## Bring your own language model key
Bring Your Own Key (BYOK) lets you connect to any compatible model provider while still using the VS Code chat experience and tools. You can use BYOK to access models from other providers, to run models locally, or to use models that are not yet available as built-in options in VS Code.
In [Local harness sessions](/docs/agents/run/agent-harnesses.md#local) `feature(local-harness)`, BYOK models work without signing into a GitHub account and without a Copilot plan. This lets you use AI chat features with your own models, including offline scenarios with local models such as Ollama.
> [!NOTE]
> To use supported BYOK models in Copilot harness sessions in the Chat view or [Agents window](/docs/agents/run/agents-window.md), enable `setting(chat.agentHost.byokModels.enabled)` `feature(agent-host-byok-models)`. For sign-in requirements, see [Agents window accounts and provider credentials](/docs/agents/run/agents-window-configuration.md#manage-ai-providers-and-accounts).
To use provider credentials with the Anthropic Claude or OpenAI Codex harness instead, follow that [harness's setup requirements](/docs/agents/run/agent-harnesses.md#requirements-and-setup). Adding a model here does not configure authentication for every harness.
You can also use these models to [override the models used for utility tasks in VS Code](#configure-models-for-other-features) (such as title generation and intent detection).
> [!NOTE]
> Some features still require a GitHub account: semantic search, inline suggestions (code completions), and features that rely on embeddings. BYOK applies to the chat experience and utility tasks only.
VS Code provides different options to add more language models:
* [Built-in providers](#add-a-model-from-a-built-in-provider): The provider you want is already listed (Azure, Anthropic, Gemini, OpenAI, and others).
* [Extensions](#add-a-model-provider-extension): A marketplace extension provides the model, for example AI Toolkit for local models.
* [Custom endpoint](#add-a-custom-endpoint-model): You have a self-hosted, enterprise, or other endpoint that speaks Chat Completions, Responses, or Messages API.
> [!NOTE]
> If you are a Copilot Business or Enterprise user, your administrator can disable the **Bring Your Own Language Model Key in VS Code** policy in the [Copilot policy settings](https://github.com/settings/copilot/features) on GitHub.com. For more details, see the [GitHub Copilot documentation](https://docs.github.com/en/copilot/how-tos/administer-copilot/manage-for-enterprise/use-your-own-api-keys).
### Add a model from a built in provider
Pick from a set of common providers that are ready to use in VS Code. Depending on the provider, you need an API key and other configuration details like the endpoint URL.
> [!IMPORTANT]
> The built-in Ollama provider is deprecated. For local Ollama models, install the official Ollama extension from the [Ollama publisher on the Visual Studio Marketplace](https://marketplace.visualstudio.com/publishers/Ollama) instead. The extension is maintained by the Ollama team, so it can support new Ollama models and capabilities faster than the built-in provider. If you use the built-in Ollama provider for BYOK local models, install the extension and remove the built-in provider configuration to keep using Ollama models without interruption.
To configure a language model from a built-in provider:
1. Open the Language Models editor by selecting **Manage Language Models** (gear icon) from the language model picker or via the **Chat: Manage Language Models** command from the Command Palette.
1. Select **Add Models**, and then select a model provider from the list.
1. Enter a group name for the models. This is the grouping label shown in the model picker and Language Models editor.
You can change the group name later from the Language Models editor if needed.
1. Enter the provider-specific details, such as the API key or endpoint URL.
1. If the provider requires additional configuration, VS Code opens a `chatLanguageModels.json` file where you can configure the provider and model details. See the [Model configuration reference](#model-configuration-reference) for details on the configuration properties.
The following example shows an Azure OpenAI configuration that uses Entra ID authentication:
```json
[
{
"name": "Azure",
"vendor": "azure",
"models": [
{
"id": "",
"name": "GPT-5.5",
"url": "https://.openai.azure.com",
"toolCalling": true,
"vision": true,
"maxInputTokens": 200000,
"maxOutputTokens": 64000
}
]
}
]
```
1. After configuring the model, you can now select it from the model picker in chat.
For a model to be available when using agents in chat, it must support tool calling. If the model doesn't support tool calling, it won't be shown in the model picker.
### Add a model provider extension
You can install extensions from the Visual Studio Marketplace that add language model providers to VS Code. These extensions can provide access to additional cloud-hosted or locally running models. For example, the Foundry Toolkit for VS Code extension provides access to Foundry's local and cloud-hosted models.
To add a model provider extension:
1. Open the Language Models editor by selecting **Manage Language Models** (gear icon) from the language model picker or via the **Chat: Manage Language Models** command from the Command Palette.
1. Select **Install Model Providers**.
VS Code opens the Extensions view, filtered to language model provider extensions. Alternatively, open the Extensions view and search for `@tag:language-models`.
1. Select **Install** to install the extension, for example, [Foundry Toolkit for VS Code](https://aka.ms/AIToolkit).
1. Follow the extension's setup instructions to configure model access.
1. The extension's models appear in the model picker in chat and in the Language Model editor. If the models don't appear, reload VS Code.
### Add a custom endpoint model
> [!NOTE]
> It replaces the deprecated OpenAI Compatible provider and supports additional API types. The `setting(github.copilot.chat.customOAIModels)` setting is deprecated.
The Custom Endpoint provider lets you connect any compatible API endpoint to chat in VS Code. It supports three API types, which you can select per provider or per model: Chat Completions, Responses, and the Anthropic Messages API. This makes it a good fit for self-hosted models, enterprise gateways, and providers that aren't available as a built-in provider.
To add a model with the Custom Endpoint provider:
1. Open the Language Models editor by selecting **Manage Language Models** (gear icon) from the language model picker or via the **Chat: Manage Language Models** command from the Command Palette.
1. Select **Add Models**, and then select **Custom Endpoint** from the list.
1. Enter a group name for the models. This is the grouping label shown in the model picker and Language Models editor.
You can change the group name later from the Language Models editor if needed.
1. Enter a display name and API key for the endpoint.
1. Select the API type: **Chat Completions**, **Responses**, or **Messages**. Make sure the model supports this API type.
1. VS Code opens a `chatLanguageModels.json` file where you can configure the model details. Update the model properties and save the file. See the [Custom Endpoint configuration reference](#custom-endpoint-configuration-reference) for details on the configuration properties.
The following example shows a Messages API configuration for an Anthropic endpoint:
```json
[
{
"name": "Anthropic",
"vendor": "customendpoint",
"apiKey": "${input:anthropicApiKey}",
"apiType": "messages",
"models": [
{
"id": "claude-sonnet-4-6",
"name": "Claude Sonnet 4.6",
"url": "https://api.anthropic.com/v1/messages",
"toolCalling": true,
"vision": true,
"maxInputTokens": 200000,
"maxOutputTokens": 64000
}
]
}
]
```
1. After configuring the model, select it from the model picker in chat.
> [!TIP]
> If the model you added does not immediately appear in the model picker, restart VS Code.
#### Custom Endpoint configuration reference
The Custom Endpoint provider supports all of the [common model configuration properties](#model-configuration-reference), plus additional provider-level and model-level properties described in this section. The provider-level properties are set on the provider object, and the model-level properties are set on each entry of the `models` array. When a property is set on both levels, the model-level value takes precedence.
Provider-level properties (in addition to the [common provider properties](#model-configuration-reference)):
| Property | Description |
|----------|-------------|
| `vendor` | Must be `customendpoint` to use the Custom Endpoint provider. |
| `apiKey` | The API key used to authenticate requests. Store the key securely by using an input variable, for example `"apiKey": "${input:myApiKey}"`, instead of committing a raw key. |
| `apiType` | _(Optional)_ Default API type for all models in the provider: `chat-completions` (default), `responses`, or `messages`. Override it per model with the model-level `apiType`. |
| `url` | _(Optional)_ Base URL used to discover models automatically. When set, VS Code queries the endpoint for the list of available models instead of using the `models` array. Omit it to configure models explicitly with the `models` array. |
In addition to the [common model properties](#model-configuration-reference) the Custom Endpoint provider supports the following model-level properties:
| Property | Description |
|----------|-------------|
| `apiType` | _(Optional)_ Override the API type for this model (`chat-completions`, `responses`, or `messages`). Defaults to the provider-level `apiType`, or is inferred from the `url` when neither is set. |
| `contextWindow` | _(Optional)_ The model's full context window (input + output) in tokens, for example `1000000` for a 1M-token model. When set, you can omit `maxInputTokens` and VS Code derives it as `contextWindow - maxOutputTokens`. |
| `modelOptions` | _(Optional)_ An object of request parameters sent with every request to the model, such as `temperature` and `top_p`. For example, `"modelOptions": { "temperature": 0.2 }`. |
| `requestHeaders` | _(Optional)_ An object of additional HTTP headers to include with requests to this model, for example to authenticate through a gateway or vanity domain. See [Custom authentication headers](#custom-authentication-headers). |
| `statefulResponses` | _(Optional)_ Whether the Responses API server retains prior responses. Defaults to `true`. Set to `false` for stateless servers to send full conversation history with each request. Applies only to the Responses API. See [Stateful Responses API endpoints](#stateful-responses-api-endpoints). |
##### Endpoint URL resolution
The `url` you provide for a model is resolved based on its API type:
* If the URL already contains an explicit API path (`/chat/completions`, `/responses`, or `/messages`), it is used as-is.
* Otherwise, VS Code appends the path for the model's API type. If the URL doesn't already end in a version segment such as `/v1`, VS Code inserts `/v1` first. For example, with the Responses API type, `https://my-host.example.com` resolves to `https://my-host.example.com/v1/responses`.
To avoid ambiguity, provide the full endpoint URL including the API path, as shown in the examples.
##### Stateful Responses API endpoints
Responses API models default to `statefulResponses: true`, sending only new history with `previous_response_id`. For stateless servers, set `statefulResponses` to `false` to omit that ID and send full history, including tool calls and results, avoiding missing-tool-call errors.
For a stateless server, add these properties to its model entry in `chatLanguageModels.json`:
```json
{
"apiType": "responses",
"statefulResponses": false
}
```
This option describes the server's capabilities and does not change its data retention policy. When `zeroDataRetentionEnabled` is `true`, `statefulResponses` is ignored and requests send full conversation history without `previous_response_id`.
##### Custom authentication headers
By default, the Custom Endpoint provider infers the authentication header from the API type and URL. For the Messages API, it sends `x-api-key`. For Azure OpenAI URLs, it sends `api-key`. Otherwise, it sends `Authorization: Bearer `.
You can override the authentication header with `requestHeaders`. When you supply a well-known auth header (such as `Authorization`, `api-key`, `x-api-key`, `x-goog-api-key`, or `apikey`), VS Code does not also send the default inferred auth header, so the endpoint doesn't receive conflicting credentials.
To keep your API key in secret storage while still using it in a custom header, use the `${apiKey}` token in the header value. VS Code replaces it with the configured `apiKey` at request time:
```json
[
{
"name": "Gateway",
"vendor": "customendpoint",
"apiKey": "${input:myApiKey}",
"models": [
{
"id": "my-model",
"name": "My Model",
"url": "https://gateway.example.com/v1/chat/completions",
"toolCalling": true,
"vision": false,
"maxInputTokens": 128000,
"maxOutputTokens": 16000,
"thinking": true,
"supportsReasoningEffort": ["low", "medium", "high"],
"reasoningEffortFormat": "chat-completions",
"modelOptions": {
"temperature": 0.2,
"top_p": 0.9
},
"requestHeaders": {
"Authorization": "Bearer ${apiKey}",
"Ocp-Apim-Subscription-Key": "my-subscription-key"
}
}
]
}
]
```
## Update model provider details
To update the details of a model provider you configured previously:
1. Select **Manage Language Models** (gear icon) from the language model picker in the Chat view or run the **Chat: Manage Language Models** command from the Command Palette.
1. In the Language Models editor, select the gear icon next to the model provider you want to update.
1. Update the provider details, such as the API key or endpoint URL.
## Configure models for other features
In addition to the main chat model, you can configure which model is used for inline chat, inline suggestions, Voice Mode, and background utility tasks.
### Change the model for inline chat
You can configure a default language model for editor inline chat. This enables you to use a different model for inline chat than for chat conversations.
To configure the default model for inline chat, use the `setting(inlineChat.defaultModel)` setting. The setting lists all available models from the model picker.
If you change the model during an inline chat session, the selection persists for the remainder of the session. After you reload VS Code, the model resets to the value specified in the `setting(inlineChat.defaultModel)` setting.
### Change the model for inline suggestions
To change the language model that is used for generating inline suggestions in the editor:
1. Select **Configure Inline Suggestions...** from the Chat menu in the VS Code title bar.
1. Select **Change Completions Model...**, and then select one of the models from the list.
> [!NOTE]
> The list of available models might vary and change over time. When no alternative models are available, the option to change the model is not available.
>
> If you are a Copilot Business or Enterprise user, your Administrator needs to enable certain models for your organization by opting in to `Editor Preview Features` in the [Copilot policy settings](https://docs.github.com/en/enterprise-cloud@latest/copilot/managing-copilot/managing-github-copilot-in-your-organization/managing-policies-for-copilot-in-your-organization#enabling-copilot-features-in-your-organization) on GitHub.com.
### Change the model for Voice Mode
To use an OpenAI Live model for spoken agent conversations, [configure Voice Mode with your own OpenAI key](/docs/configure/accessibility/voice.md#use-your-own-openai-key). `feature(voice-mode)` The voice model handles the spoken conversation separately from the model selected for coding tasks in chat.
### Change the model for utility tasks
In addition to the main chat model, VS Code uses lightweight models in the background for utility tasks such as generating titles, creating commit messages and pull request descriptions, and detecting intent. The model you select for a chat or agent session does not control these utility tasks. By default, these tasks use built-in utility models provided by GitHub Copilot. You can override which model is used for these tasks with any available model, including [BYOK](#bring-your-own-language-model-key) and extension-provided models.
There are two settings for utility models, depending on the type of task:
* `setting(chat.utilityModel)`: Override the model used for general utility flows, such as generating titles and summaries, settings search, and Git review.
* `setting(chat.utilitySmallModel)`: Override the model used for fast, lightweight utility flows, such as commit messages, pull request titles and descriptions, rename suggestions, branch name generation, prompt categorization, and intent detection. A fast and inexpensive model is recommended for this setting.
Both settings default to **Default**, which uses the built-in utility model from GitHub Copilot.
If you use BYOK models without signing into a GitHub account, the built-in utility models are not available. VS Code shows a notification in the Chat view that prompts you to configure utility models. Set `setting(chat.utilityModel)` and `setting(chat.utilitySmallModel)` to a BYOK model to enable utility features like title generation, commit message creation, and pull request description creation.
#### Configure the default utility model for BYOK models
When you select a [BYOK](#bring-your-own-language-model-key) model as the main agent model, use the `setting(chat.byokUtilityModelDefault)` setting to control which model built-in utility flows use by default. This setting accepts the following values:
* **None**: Do not use a default utility model.
* **Main Agent Model**: Use the selected BYOK main agent model for utility flows.
* **GitHub Copilot**: Use the default GitHub Copilot utility models.
> [!NOTE]
> By default, utility flows use GitHub Copilot utility models when a Copilot token is available. If no Copilot token is available, configure `setting(chat.utilityModel)` and `setting(chat.utilitySmallModel)`, or select **Main Agent Model**, to enable utility features.
This setting has no effect when the main agent model is provided by GitHub Copilot. A specific model configured in `setting(chat.utilityModel)` or `setting(chat.utilitySmallModel)` takes precedence over this default.
## Model configuration reference
When you add BYOK model, you can configure the model properties in the `chatLanguageModels.json` file. The configuration has two levels: provider-level and model-level.
Depending on the provider, some provider and model properties might be required while others are optional. For example, some providers only require an API key and endpoint URL and discover the available models automatically, while others require you to specify the details for each model.
The provider-level properties include:
| Property | Description |
|----------|-------------|
| `vendor` | The provider of the model, for example `azure`, `openai`, `customendpoint` |
| `name` | The display name (group name) of the provider shown in the UI. |
| `models` | _(Optional)_ An array of model configurations provided by this provider. |
Each model in the `models` array supports the following properties:
| Property | Description |
|----------|-------------|
| `id` | Model identifier sent to the API. For example, for Foundry this is the deployment name. |
| `name` | Display name shown in the model picker. |
| `url` | Full endpoint URL for the model. |
| `apiType` | _(Optional)_ Override the API type per model (`chat-completions`, `responses`, or `messages`). Defaults to the provider-level `apiType`. |
| `toolCalling` | Set to `true` if the model supports tool calling. |
| `vision` | Set to `true` if the model supports image inputs. |
| `maxInputTokens` | Maximum number of input tokens the model accepts. Together with `maxOutputTokens`, this defines the model's context window. |
| `maxOutputTokens` | Maximum number of output tokens the model generates. Together with `maxInputTokens`, this defines the model's context window. |
| `editTools` | _(Optional)_ An array of edit tools the model supports. If not configured, the editor tries multiple edit tools and picks the best one. Possible values: `find-replace`, `multi-find-replace`, `apply-patch`, `code-rewrite`. |
| `thinking` | _(Optional)_ Set to `true` if the model supports thinking capabilities. Defaults to `false`. |
| `streaming` | _(Optional)_ Set to `true` if the model supports streaming responses. Defaults to `true`. |
| `zeroDataRetentionEnabled` | _(Optional)_ Set to `true` if Zero Data Retention (ZDR) is enabled for this endpoint. When enabled, `previous_response_id` is not sent in requests via the Responses API. Defaults to `false`. |
| `supportsReasoningEffort` | _(Optional)_ An array of reasoning effort levels the model accepts (for example, `["low", "medium", "high"]`). When set, a **Thinking Effort** picker is shown in the model picker. Common levels are `minimal`, `low`, `medium`, `high`. |
| `reasoningEffortFormat` | _(Optional)_ Body shape used to forward reasoning effort to the model. `chat-completions` sends a top-level `reasoning_effort` string, `responses` sends a nested `reasoning.effort` object, and `messages` sends `output_config.effort`. When unset, the format follows the URL. |
| `requestHeaders` | _(Optional)_ An object of additional HTTP headers to include with requests to this model. Certain reserved headers (forbidden, forwarding, and internal headers) are not allowed and are ignored if present. |
> [!NOTE]
> The sum of `maxInputTokens` and `maxOutputTokens` must not exceed the model's context window. VS Code uses the sum of these two values as the model's total context window, for example to show context usage in the Chat view. Typically, you set `maxInputTokens` to the model's context window size minus `maxOutputTokens`. Check your provider's model documentation for the context window size.
## Frequently asked questions
### How do I enable bring your own model key for Copilot Business or Copilot Enterprise?
If you are a Copilot Business or Enterprise user, your organization administrator must enable the **Bring Your Own Language Model Key in VS Code** policy in the [Copilot policy settings](https://github.com/settings/copilot/features) on GitHub.com. After the policy is enabled, you can use your own API keys to add models, just like individual plan users. For more details, see the [GitHub Copilot documentation](https://docs.github.com/en/copilot/how-tos/administer-copilot/manage-for-enterprise/use-your-own-api-keys).
### Can I use locally hosted models with Copilot in VS Code?
You have two options to use locally hosted models in chat:
* **BYOK local models**: Use [bring your own model key](#bring-your-own-language-model-key) (BYOK) with a model provider that supports connecting to a local model. You can use a built-in model provider that supports local models or install an extension from the [Visual Studio Marketplace](https://marketplace.visualstudio.com/search?term=tag%3Alanguage-models&target=VSCode&category=All%20categories&sortBy=Relevance), such as [AI Toolkit for VS Code with Foundry Local](https://aka.ms/AIToolkit). BYOK local models work without a GitHub account, a Copilot plan, or an internet connection.
* **On-device models (Experimental)**: Use a [Copilot-managed model](#use-on-device-models) that runs on supported NVIDIA hardware. On-device models require GitHub sign-in.
To get the full set of utility features with a BYOK local model, such as title generation and commit messages, configure `setting(chat.utilityModel)` and `setting(chat.utilitySmallModel)` to point to the local model.
Currently, you cannot connect to a local model for inline suggestions. VS Code provides an extension API [`InlineCompletionItemProvider`](/api/references/vscode-api.md#InlineCompletionItemProvider) that enables extensions to contribute a custom completion provider. You can get started with our [Inline Completions sample](https://github.com/microsoft/vscode-extension-samples/blob/main/inline-completions).
> [!NOTE]
> Some features require a GitHub account and internet connectivity: semantic search, inline suggestions (code completions), and features that rely on embeddings. These features are not available through BYOK models.
### Can I use a local model without an internet connection?
Yes, you can use a BYOK local model completely offline. Add a local model provider such as Ollama by using the **Chat: Manage Language Models** command, select the model in chat, and start using it. To also enable utility features like title generation and commit messages, set `setting(chat.utilityModel)` and `setting(chat.utilitySmallModel)` to a local model. Features that depend on the GitHub Copilot service, such as semantic search, inline suggestions, and embeddings, are not available offline.
### Can I use a local model without a Copilot plan?
Yes, you can use BYOK models, including local models, without a Copilot plan and without signing into a GitHub account. Add a model with the **Chat: Manage Language Models** command and select it in chat. Features that depend on the GitHub Copilot service, such as semantic search, inline suggestions, and embeddings, require a Copilot plan.
## Related resources
* [Language models concepts](/docs/agents/concepts/language-models.md)
* [Available language models in GitHub Copilot](https://docs.github.com/en/copilot/using-github-copilot/ai-models/changing-the-ai-model-for-copilot-chat?tool=vscode)
* [Choosing the right AI model for your task](https://docs.github.com/en/copilot/using-github-copilot/ai-models/choosing-the-right-ai-model-for-your-task)
* [Security considerations for AI in VS Code](/docs/agents/run/security.md)