Skip to main content

​
What is Braintrust?

Braintrust is the AI observability platform helping teams measure, evaluate, and improve AI in production. Learn more at https://www.braintrust.dev/

​
Prerequisites

Before you begin, ensure you have:
  • Cerebras API Key - Get a free API key here.
  • Braintrust API Key - Visit Braintrust and create an account or log in.
    • Go to Settings > AI Providers and add your Cerebras API key. Then generate a Braintrust API key.
  • Python 3.7 or higher

​
Configure Braintrust

1

Install required dependencies

Run the following:
pip install autoevals braintrust openai
2

Configure environment variables

Create a .env file in your project directory:
CEREBRAS_API_KEY=your-cerebras-api-key-here
BRAINTRUST_API_KEY=your-braintrust-api-key-here

# Optional: If self-hosting Braintrust
# BRAINTRUST_API_URL=your-braintrust-api-url
3

Initialize the client

Set up the Cerebras client with Braintrust wrapping:
import os
import openai
import braintrust

# Wrap the OpenAI client to enable automatic tracing
client = braintrust.wrap_openai(
    openai.OpenAI(
        api_key=os.getenv("CEREBRAS_API_KEY"),
        base_url="https://api.cerebras.ai/v1",
    )
)
4

Start logging

Initialize a logger to automatically track all your model calls:
  import os
  import openai
  import braintrust

  # Wrap the OpenAI client to enable automatic tracing
  client = braintrust.wrap_openai(
      openai.OpenAI(
          api_key=os.getenv("CEREBRAS_API_KEY"),
          base_url="https://api.cerebras.ai/v1",
      )
  )

  # Initialize logger for your project
  braintrust.init_logger("My Cerebras Project")

  # Make a simple completion call
  response = client.chat.completions.create(
      model="llama3.1-8b",
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is the capital of Nevada?"},
      ],
      extra_headers={
          "X-Cerebras-3rd-Party-Integration": "braintrust"
      }
  )

  print(response.choices[0].message.content)
All calls are automatically logged to Braintrust with metrics like latency, token usage, and time to first token.
5

View your logs

  • Open the Braintrust dashboard
  • Navigate to your project
  • View detailed logs with metrics
  • Reproduce and tweak prompts directly in the UI view logs tweak the prompt
6

Run evaluations

Create evaluations to test your model’s performance:
  import os
  import openai
  from braintrust import wrap_openai, Eval
  from autoevals import Factuality

  # Wrap the OpenAI client to enable automatic tracing
  client = wrap_openai(
      openai.OpenAI(
          api_key=os.getenv("CEREBRAS_API_KEY"),
          base_url="https://api.cerebras.ai/v1",
      )
  )

  Eval(
      "My Cerebras Project",
      data=[
          {"input": "What is 100-94?", "expected": "6"},
          {"input": "What is the square root of 16?", "expected": "4"},
      ],
      task=lambda input: client.chat.completions.create(
          model="llama3.1-8b",
          messages=[
              {
                  "role": "system",
                  "content": "You are a helpful assistant. Provide only the answer.",
              },
              {"role": "user", "content": input},
          ],
      ).choices[0].message.content,
      scores=[
          Factuality(
              model="llama3.3-70b",
              api_key=os.environ["CEREBRAS_API_KEY"]
          )
      ],
  )
You can check these evaluations in the Braintrust dashboard:dashboard

​
Next Steps

  • Explore the Braintrust documentation
  • Try out differen Cerebras models
  • Set up custom evaluation metrics
  • Build production monitoring dashboards

​
Troubleshooting

​
API Key Issues

  • Verify your keys are correctly set in environment variables
  • Check that Cerebras key is added to Braintrust’s AI providers

​
Import Errors

  • Ensure all packages are installed: pip install autoevals braintrust openai
  • Use a virtual environment to avoid conflicts

​
Connection Issues