- Find the code or prompt logic behind a trace failure.
- Read pull requests, reviews, comments, and CI results.
- Investigate CI failures using GitHub Actions logs and artifacts.
- Write a fix and regression test, push a branch, and open a pull request.
Connect GitHub
You need:- The Manage settings organization permission. Organization owners have it by default.
- Ownership of the personal GitHub account, or approval from a GitHub organization owner. Organization members can request approval during installation.
1
Connect an installation
Go to Settings > Integrations and click Connect GitHub.In GitHub, authorize access. Back in Braintrust, choose how to connect:
- Use an existing installation: Select the installation for the GitHub account or organization you want to connect, then click Connect. If it’s marked Requires GitHub org owner, a GitHub organization owner must connect it.
- Connect another GitHub account or organization: Click Install in a new organization. If all listed installations are already connected, click Install in the notice instead.
If the installation needs approval
If the installation needs approval
Integrations shows Pending and Needs org owner approval.
- Click Copy approval link and share it with a GitHub organization owner. Owners can also click Approve [organization] on GitHub.
- After approval, reload Integrations to finish connecting.
2
Verify access and set the default
Back in Integrations, click the installation’s row. Under Repositories, confirm that the repositories you need are listed, then click Set as default and Save.The installation shows a Default badge. Projects use it unless they select a different installation. Organization-level Loop threads always use the default installation, regardless of attached projects or their settings.
3
Use the code in Loop
Start a new Loop thread in the project and ask a question that combines traces with source code:If Loop asks you to authorize GitHub, connect your GitHub account to continue.Loop clones up to five authorized repositories into
github/<owner>/<repo>. Ask it to clone others as needed. See checkout limitations.Connect your GitHub account
Loop uses your connected GitHub account in conversations, subject to repository access and permissions. When Loop displays an Authorize GitHub card:- Click Authorize.
- Complete the connection in the popup, using a browser signed in to your own Braintrust account. API keys and OAuth bearer tokens can’t complete it.
- Return to the thread. Loop continues automatically.
Change GitHub settings for a project
Projects inherit the organization’s default installation. With the Manage settings organization permission, you can select another installation or disable GitHub for a project. In your project, go to Settings > Integrations. The Project usage column marks the installation the project uses as In use.- Use a different installation: Click the installation’s row. Under Project usage, turn on Use this installation, then click Save.
- Return to the organization default: Open the selected installation, turn off Use this installation, then click Save.
- Turn GitHub off for the project: Click Disable GitHub in [project name]. Loop threads in the project don’t get repository access until you click Enable GitHub in [project name], which returns the project to the organization default.
Investigate CI failures
Loop can find GitHub Actions runs and download their logs and artifacts to investigate a failure:Write changes through Loop
Loop can implement a fix and regression test, push a branch, and open a pull request using your connected GitHub account. To make changes, you need:- Braintrust access: Read permission on the project, or organization membership for an organization-level thread.
- GitHub access: Permission for the requested action from both your account and the GitHub App installation.
How write access works
Automation runs, including Patterns runs, are read-only. In interactive threads:- Actions: Loop can push code and make repository or organization REST requests, including pull request updates and reviews. Your account and the GitHub App installation must permit each action.
- Repositories: Loop can write to repositories where your account has write access and the thread’s GitHub connection allows access, including repositories it hasn’t cloned.
- Branches: Braintrust doesn’t restrict branch names or block default-branch or force pushes. Use GitHub branch protection rules or rulesets to restrict them.
- Revoking access: Change GitHub permissions or remove repository access to stop writes. This does not undo completed operations or delete existing branches and pull requests.
Access and security
Loop uses your connected GitHub account for reads and writes in conversations. Repository access must be allowed by all three:- Your GitHub account
- The GitHub App installation
- The project’s GitHub connection, or the organization’s default installation for organization-level threads
- Credentials: GitHub tokens are not exposed to the model or sandbox.
- Downloads: Actions log and artifact downloads recheck your repository read access. Requests to GitHub’s storage hosts do not include your GitHub credentials.
- Cloned files: Background checkouts can use the GitHub App installation’s credentials. Your personal GitHub permissions do not restrict access to files already cloned into a sandbox.
Manage connections
Change or remove the organization default
Change or remove the organization default
In Integrations settings, click an installation’s row. Click Set as default to make it the default, or Remove as default to stop providing a default installation, then click Save.These changes affect organization-level threads and projects that use the organization default. Projects that selected a different installation are unchanged. Without a default, organization-level threads and projects without a selected installation can’t use GitHub.
Change repository access or disconnect GitHub
Change repository access or disconnect GitHub
To change which repositories Braintrust can access, click Manage on GitHub in an installation’s row in Integrations settings, or click the row and then Manage repository access on GitHub below the Repositories list.If Manage on GitHub doesn’t appear, for example when Braintrust can’t reach GitHub to check the installation, open the Braintrust GitHub App on GitHub and click Configure to change its repository access.To remove an installation from Braintrust, click Disconnect installation in its row, or click the row and then Delete. Confirm the disconnection when prompted. The app remains installed on GitHub.
Troubleshoot a missing repository
Troubleshoot a missing repository
In the project’s Integrations settings, check that the installation marked In use is active and that the repository appears in its Repositories list. If the repository is authorized but was not automatically cloned, ask Loop to clone it by its
owner/repo name.Start a new thread after changing the connection so Loop creates a new sandbox and prepares the checkouts. A failed checkout does not prevent the rest of Loop from working. Ask Loop to check whether the repository is available before continuing a source-code task.Troubleshoot a blocked push or pull request
Troubleshoot a blocked push or pull request
When Braintrust blocks a push or pull request, Loop reports the reason in the thread and what to change. Common causes:
- Insufficient permissions: Check that you have Read permission on the project and that your connected GitHub account and the GitHub App installation permit the operation. GitHub branch protections still apply.
- The thread is organization-level: Check that you are a member of the Braintrust organization and that the repository is in its default installation.
- Your GitHub account isn’t connected: Connect your GitHub account when Loop prompts you.
- The run is an automation: Automation runs, including Patterns runs, are read-only. Ask for the change in an interactive thread.
- The repository isn’t part of the project’s GitHub connection: Ask someone with the Manage settings organization permission to add the repository to the installation the project uses. Repository access is changed on GitHub, which can require a GitHub organization owner.
Limitations
- Up to five automatic checkouts. Loop automatically clones the first five authorized repositories, ordered by GitHub repository ID. Ask Loop to clone additional authorized repositories.
- Scoped REST access. Loop can use REST endpoints under
/repos/<owner>/<repo>/for connected repositories and/orgs/<owner>/for their owning organizations, subject to GitHub permissions. GraphQL is not supported. - Shallow checkouts. Each checkout starts with the latest default-branch commit, without tags or earlier history. Ask Loop to fetch additional history or updates as needed.
Next steps
- Explore what Loop can do for more investigation workflows.
- Connect external MCP servers to add context from issue trackers and other services.