Get Started with Braintrust

What is Braintrust?

Braintrust is the AI observability platform helping teams measure, evaluate, and improve AI in production. Learn more at https://www.braintrust.dev/

Prerequisites

Before you begin, ensure you have:

Cerebras API Key - Get a free API key here.
Braintrust API Key - Visit Braintrust and create an account or log in.
- Go to Settings > AI Providers and add your Cerebras API key. Then generate a Braintrust API key.
Python 3.7 or higher

Configure Braintrust

Install required dependencies

Run the following:

pip install autoevals braintrust openai

Configure environment variables

Create a .env file in your project directory:

CEREBRAS_API_KEY=your-cerebras-api-key-here
BRAINTRUST_API_KEY=your-braintrust-api-key-here

# Optional: If self-hosting Braintrust
# BRAINTRUST_API_URL=your-braintrust-api-url

Initialize the client

Set up the Cerebras client with Braintrust wrapping:

import os
import openai
import braintrust

# Wrap the OpenAI client to enable automatic tracing
client = braintrust.wrap_openai(
    openai.OpenAI(
        api_key=os.getenv("CEREBRAS_API_KEY"),
        base_url="https://api.cerebras.ai/v1",
        default_headers={"X-Cerebras-3rd-Party-Integration": "braintrust"}
    )
)

Start logging

Initialize a logger to automatically track all your model calls:

  import os
  import openai
  import braintrust

  # Wrap the OpenAI client to enable automatic tracing
  client = braintrust.wrap_openai(
      openai.OpenAI(
          api_key=os.getenv("CEREBRAS_API_KEY"),
          base_url="https://api.cerebras.ai/v1",
          default_headers={"X-Cerebras-3rd-Party-Integration": "braintrust"}
      )
  )

  # Initialize logger for your project
  braintrust.init_logger("My Cerebras Project")

  # Make a simple completion call
  response = client.chat.completions.create(
      model="gpt-oss-120b",
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is the capital of Nevada?"},
      ],
      extra_headers={
          "X-Cerebras-3rd-Party-Integration": "braintrust"
      }
  )

  print(response.choices[0].message.content)

All calls are automatically logged to Braintrust with metrics like latency, token usage, and time to first token.

View your logs

Open the Braintrust dashboard
Navigate to your project
View detailed logs with metrics
Reproduce and tweak prompts directly in the UI

Run evaluations

Create evaluations to test your model’s performance:

  import os
  import openai
  from braintrust import wrap_openai, Eval
  from autoevals import Factuality

  # Wrap the OpenAI client to enable automatic tracing
  client = wrap_openai(
      openai.OpenAI(
          api_key=os.getenv("CEREBRAS_API_KEY"),
          base_url="https://api.cerebras.ai/v1",
          default_headers={"X-Cerebras-3rd-Party-Integration": "braintrust"}
      )
  )

  Eval(
      "My Cerebras Project",
      data=[
          {"input": "What is 100-94?", "expected": "6"},
          {"input": "What is the square root of 16?", "expected": "4"},
      ],
      task=lambda input: client.chat.completions.create(
          model="gpt-oss-120b",
          messages=[
              {
                  "role": "system",
                  "content": "You are a helpful assistant. Provide only the answer.",
              },
              {"role": "user", "content": input},
          ],
      ).choices[0].message.content,
      scores=[
          Factuality(
              model="gpt-oss-120b",
              api_key=os.environ["CEREBRAS_API_KEY"]
          )
      ],
  )

You can check these evaluations in the Braintrust dashboard:

Available Models

Cerebras offers several high-performance models for your Braintrust evaluations:

Model	Parameters	Best For
gpt-oss-120b	120B	Largest model for the most demanding tasks
zai-glm-4.7	357B	Advanced 357B parameter model with strong reasoning capabilities

Simply change the model parameter in your API calls to switch between models.

Next Steps

Explore the Braintrust documentation
Migrate to GLM4.7: Ready to upgrade? Follow our migration guide to start using our latest model
Try out different Cerebras models
Set up custom evaluation metrics
Build production monitoring dashboards

Troubleshooting

API Key Issues

Verify your keys are correctly set in environment variables
Check that Cerebras key is added to Braintrust’s AI providers

Import Errors

Ensure all packages are installed: pip install autoevals braintrust openai
Use a virtual environment to avoid conflicts

Connection Issues

Verify the base URL: https://api.cerebras.ai/v1
Check your network connection and firewall settings

Integrations

Agentic Frameworks

AI Development Kits

Coding Tools

Containerization

Observability & Evaluation

LLM Integration Tools

LLM Application Frameworks

Document Processing

Real-Time Audio

Multi-LLM Management

No-Code/Low-Code Platforms

What is Braintrust?

Prerequisites

Configure Braintrust

Available Models

Next Steps

Troubleshooting

API Key Issues

Import Errors

Connection Issues

​What is Braintrust?

​Prerequisites

​Configure Braintrust

​Available Models

​Next Steps

​Troubleshooting

​API Key Issues

​Import Errors

​Connection Issues

What is Braintrust?

Prerequisites

Configure Braintrust

Available Models

Next Steps

Troubleshooting

API Key Issues

Import Errors

Connection Issues