> ## Documentation Index
> Fetch the complete documentation index at: https://inference-docs.cerebras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Get Started with LiteLLM

> Learn how to use LiteLLM as a unified interface to call Cerebras models alongside 100+ other LLM providers with a single API.

## What is LiteLLM?

LiteLLM is a lightweight Python package that provides a unified interface for calling 100+ LLM providers (OpenAI, Azure, Anthropic, Cohere, Cerebras, Replicate, PaLM, and more) using the OpenAI format. With LiteLLM, you can easily switch between different LLM providers, including Cerebras, without changing your code structure.

## Prerequisites

Before you begin, ensure you have:

* **Cerebras API Key** - Get a free API key [here](https://cloud.cerebras.ai/?utm_source=3pi_litellm\&utm_campaign=integrations).
* **Python 3.7 or higher** - LiteLLM requires a modern Python environment.
* **pip package manager** - For installing the LiteLLM library.

## Configure LiteLLM

<Steps>
  <Step title="Install LiteLLM">
    Install the LiteLLM package using pip:

    ```bash theme={null}
    pip install litellm
    ```

    This will install LiteLLM and all its dependencies, including the OpenAI SDK which LiteLLM uses under the hood.
  </Step>

  <Step title="Set up your environment variables">
    Create a `.env` file in your project directory to securely store your API key:

    ```bash theme={null}
    CEREBRAS_API_KEY=your-cerebras-api-key-here
    ```

    Alternatively, you can export the environment variable in your terminal:

    ```bash theme={null}
    export CEREBRAS_API_KEY=your-cerebras-api-key-here
    ```
  </Step>

  <Step title="Make your first request with LiteLLM">
    LiteLLM provides a simple `completion()` function that works across all providers. Here's how to call Cerebras models:

    <CodeGroup>
      ```python Python theme={null}
      import os
      from litellm import completion

      # Set your Cerebras API key
      os.environ["CEREBRAS_API_KEY"] = os.getenv("CEREBRAS_API_KEY")

      # Make a completion request to Cerebras
      response = completion(
          model="cerebras/gpt-oss-120b",
          messages=[
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "What is the capital of France?"}
          ],
          api_base="https://api.cerebras.ai/v1",
          custom_llm_provider="cerebras",
          extra_headers={
              "X-Cerebras-3rd-Party-Integration": "litellm"
          }
      )

      print(response.choices[0].message.content)
      ```

      ```javascript JavaScript theme={null}
      import Anthropic from '@anthropic-ai/sdk';
      import { OpenAI } from 'openai';

      // Note: LiteLLM's JavaScript support is through direct OpenAI SDK usage
      const openai = new OpenAI({
        apiKey: process.env.CEREBRAS_API_KEY,
        baseURL: 'https://api.cerebras.ai/v1',
        defaultHeaders: {
          'X-Cerebras-3rd-Party-Integration': 'litellm'
        }
      });

      const response = await openai.chat.completions.create({
        model: 'gpt-oss-120b',
        messages: [
          { role: 'system', content: 'You are a helpful assistant.' },
          { role: 'user', content: 'What is the capital of France?' }
        ]
      });

      console.log(response.choices[0].message.content);
      ```

      ```bash cURL theme={null}
      curl https://api.cerebras.ai/v1/chat/completions \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer $CEREBRAS_API_KEY" \
        -H "X-Cerebras-3rd-Party-Integration: litellm" \
        -d '{
          "model": "gpt-oss-120b",
          "messages": [
            {
              "role": "system",
              "content": "You are a helpful assistant."
            },
            {
              "role": "user",
              "content": "What is the capital of France?"
            }
          ]
        }'
      ```
    </CodeGroup>

    The `cerebras/` prefix tells LiteLLM to route the request to Cerebras, and the integration header ensures proper tracking and support.
  </Step>

  <Step title="Use streaming responses">
    LiteLLM supports streaming responses, which is useful for real-time applications where you want to display tokens as they're generated:

    <CodeGroup>
      ```python Python theme={null}
      import os
      from litellm import completion

      os.environ["CEREBRAS_API_KEY"] = os.getenv("CEREBRAS_API_KEY")

      # Enable streaming with stream=True
      response = completion(
          model="cerebras/gpt-oss-120b",
          messages=[
              {"role": "user", "content": "Write a short poem about artificial intelligence."}
          ],
          api_base="https://api.cerebras.ai/v1",
          custom_llm_provider="cerebras",
          stream=True,
          extra_headers={
              "X-Cerebras-3rd-Party-Integration": "litellm"
          }
      )

      # Process the stream
      for chunk in response:
          if chunk.choices[0].delta.content:
              print(chunk.choices[0].delta.content, end="")
      ```

      ```javascript JavaScript theme={null}
      import { OpenAI } from 'openai';

      const openai = new OpenAI({
        apiKey: process.env.CEREBRAS_API_KEY,
        baseURL: 'https://api.cerebras.ai/v1',
        defaultHeaders: {
          'X-Cerebras-3rd-Party-Integration': 'litellm'
        }
      });

      const stream = await openai.chat.completions.create({
        model: 'gpt-oss-120b',
        messages: [
          { role: 'user', content: 'Write a short poem about artificial intelligence.' }
        ],
        stream: true
      });

      for await (const chunk of stream) {
        process.stdout.write(chunk.choices[0]?.delta?.content || '');
      }
      ```

      ```bash cURL theme={null}
      curl https://api.cerebras.ai/v1/chat/completions \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer $CEREBRAS_API_KEY" \
        -H "X-Cerebras-3rd-Party-Integration: litellm" \
        -d '{
          "model": "gpt-oss-120b",
          "messages": [
            {
              "role": "user",
              "content": "Write a short poem about artificial intelligence."
            }
          ],
          "stream": true
        }'
      ```
    </CodeGroup>

    Streaming is particularly powerful with Cerebras's fast inference speeds, allowing you to deliver near-instantaneous responses to your users.
  </Step>

  <Step title="Try different Cerebras models">
    Cerebras offers several high-performance models optimized for different use cases. Here's how to use them with LiteLLM:

    ```python theme={null}
    import os
    from litellm import completion

    os.environ["CEREBRAS_API_KEY"] = os.getenv("CEREBRAS_API_KEY")

    # Try Llama 3.1 8B for faster responses
    response_8b = completion(
        model="cerebras/gpt-oss-120b",
        messages=[{"role": "user", "content": "Hello!"}],
        api_base="https://api.cerebras.ai/v1",
        custom_llm_provider="cerebras",
        extra_headers={"X-Cerebras-3rd-Party-Integration": "litellm"}
    )

    # Try GPT-OSS 120B for most capable responses
    response_120b = completion(
        model="cerebras/gpt-oss-120b",
        messages=[{"role": "user", "content": "Write a detailed analysis."}],
        api_base="https://api.cerebras.ai/v1",
        custom_llm_provider="cerebras",
        extra_headers={"X-Cerebras-3rd-Party-Integration": "litellm"}
    )
    ```

    Choose the model that best fits your latency, cost, and capability requirements.
  </Step>
</Steps>

## Advanced Features

### Using LiteLLM's Router for Load Balancing

LiteLLM's Router allows you to load balance across multiple models or providers, including Cerebras. This is useful for distributing traffic and implementing fallback strategies:

```python theme={null}
import os
from litellm import Router

os.environ["CEREBRAS_API_KEY"] = os.getenv("CEREBRAS_API_KEY")

# Define model configurations
model_list = [
    {
        "model_name": "cerebras-llama",
        "litellm_params": {
            "model": "cerebras/gpt-oss-120b",
            "api_base": "https://api.cerebras.ai/v1",
            "custom_llm_provider": "cerebras",
            "extra_headers": {"X-Cerebras-3rd-Party-Integration": "litellm"}
        }
    },
    {
        "model_name": "cerebras-qwen",
        "litellm_params": {
            "model": "cerebras/gpt-oss-120b",
            "api_base": "https://api.cerebras.ai/v1",
            "custom_llm_provider": "cerebras",
            "extra_headers": {"X-Cerebras-3rd-Party-Integration": "litellm"}
        }
    }
]

# Initialize router
router = Router(model_list=model_list)

# Make a request - router will automatically load balance
response = router.completion(
    model="cerebras-llama",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

### Fallback and Retry Logic

LiteLLM supports automatic fallbacks between providers, which is useful for building resilient applications:

```python theme={null}
import os
from litellm import completion

os.environ["CEREBRAS_API_KEY"] = os.getenv("CEREBRAS_API_KEY")

# Define fallback models
response = completion(
    model="cerebras/gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello!"}],
    api_base="https://api.cerebras.ai/v1",
    custom_llm_provider="cerebras",
    fallbacks=["cerebras/gpt-oss-120b", "cerebras/gpt-oss-120b"],
    extra_headers={"X-Cerebras-3rd-Party-Integration": "litellm"}
)
```

If the primary model fails or is unavailable, LiteLLM will automatically try the fallback models in order.

### Cost Tracking and Budgets

LiteLLM includes built-in cost tracking to help you monitor your API usage:

```python theme={null}
import os
from litellm import completion, completion_cost

os.environ["CEREBRAS_API_KEY"] = os.getenv("CEREBRAS_API_KEY")

response = completion(
    model="cerebras/gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello!"}],
    api_base="https://api.cerebras.ai/v1",
    custom_llm_provider="cerebras",
    extra_headers={"X-Cerebras-3rd-Party-Integration": "litellm"}
)

# Calculate the cost of this completion
cost = completion_cost(completion_response=response)
print(f"Cost: ${cost}")
```

## Why Use LiteLLM with Cerebras?

* **Unified Interface**: LiteLLM provides a consistent API across 100+ providers, making it easy to experiment with different models or migrate between providers without rewriting code.
* **Production-Ready Features**: Built-in support for retries, fallbacks, load balancing, and cost tracking.
* **Observability**: Integrate with logging and monitoring tools to track your LLM usage and performance.
* **Speed Meets Flexibility**: Combine Cerebras's industry-leading inference speed with LiteLLM's flexible routing and management capabilities.

## FAQ

<Accordion title="Can I use LiteLLM Proxy with Cerebras?">
  Yes! LiteLLM Proxy allows you to create a centralized gateway for all your LLM requests. You can configure Cerebras as one of your providers in the proxy configuration file:

  ```yaml theme={null}
  model_list:
    - model_name: cerebras-llama
      litellm_params:
        model: cerebras/gpt-oss-120b
        api_base: https://api.cerebras.ai/v1
        custom_llm_provider: openai
        extra_headers:
          X-Cerebras-3rd-Party-Integration: litellm
  ```

  Learn more in the [LiteLLM Proxy documentation](https://docs.litellm.ai/docs/proxy/quick_start?utm_source=cerebras\&utm_medium=integration_page\&utm_campaign=3pi_litellm).
</Accordion>

<Accordion title="How do I handle rate limits with LiteLLM?">
  LiteLLM provides built-in retry logic with exponential backoff. You can configure this behavior:

  ```python theme={null}
  from litellm import completion

  response = completion(
      model="cerebras/gpt-oss-120b",
      messages=[{"role": "user", "content": "Hello!"}],
      api_base="https://api.cerebras.ai/v1",
      custom_llm_provider="cerebras",
      num_retries=3,
      extra_headers={"X-Cerebras-3rd-Party-Integration": "litellm"}
  )
  ```

  You can also use the Router to distribute load across multiple API keys or models.
</Accordion>

<Accordion title="Can I use LiteLLM with async/await in Python?">
  Yes! LiteLLM supports async operations using `acompletion()`:

  ```python theme={null}
  import os
  import asyncio
  from litellm import acompletion

  os.environ["CEREBRAS_API_KEY"] = os.getenv("CEREBRAS_API_KEY")

  async def main():
      response = await acompletion(
          model="cerebras/gpt-oss-120b",
          messages=[{"role": "user", "content": "Hello!"}],
          api_base="https://api.cerebras.ai/v1",
          custom_llm_provider="cerebras",
          extra_headers={"X-Cerebras-3rd-Party-Integration": "litellm"}
      )
      print(response.choices[0].message.content)

  asyncio.run(main())
  ```

  This is particularly useful for building high-performance applications that need to handle multiple concurrent requests.
</Accordion>

## Available Models

LiteLLM supports all Cerebras models through the unified interface:

| Model            | Parameters | Best For                                                         |
| ---------------- | ---------- | ---------------------------------------------------------------- |
| **gpt-oss-120b** | 120B       | Largest model for the most demanding tasks                       |
| **zai-glm-4.7**  | 357B       | Advanced 357B parameter model with strong reasoning capabilities |

Use the `cerebras/` prefix with the model name (e.g., `cerebras/gpt-oss-120b`) when calling through LiteLLM.

## Next Steps

* Explore the [LiteLLM documentation](https://docs.litellm.ai/?utm_source=cerebras\&utm_campaign=litellm) for advanced features like caching, budgets, and custom callbacks
* Learn about [Cerebras's available models](/models/overview) and their capabilities
* Set up [LiteLLM Proxy](https://docs.litellm.ai/docs/proxy/quick_start?utm_source=cerebras\&utm_campaign=litellm) for centralized LLM management
* **Migrate to GLM4.7**: Ready to upgrade? Follow our [migration guide](https://inference-docs.cerebras.ai/resources/glm-47-migration) to start using our latest model

## Troubleshooting

### Authentication Errors

If you encounter authentication errors, verify that:

* Your `CEREBRAS_API_KEY` environment variable is set correctly
* The API key is valid and hasn't expired
* You're using the correct `api_base` URL: `https://api.cerebras.ai/v1`

### Model Not Found Errors

Ensure you're using the correct model name format:

* Use `cerebras/gpt-oss-120b` (with the `cerebras/` prefix)
* Check the [available models](/models/overview) to confirm the model name
* Note that model names are case-sensitive

### Rate Limiting

If you hit rate limits:

* Implement exponential backoff using LiteLLM's built-in retry logic with `num_retries`
* Consider using the Router for load balancing across multiple API keys

## Additional Resources

* [LiteLLM GitHub Repository](https://github.com/BerriAI/litellm?utm_source=cerebras\&utm_campaign=litellm)
* [Cerebras API Reference](/api-reference/chat-completions)
* [LiteLLM Supported Providers](https://docs.litellm.ai/docs/providers?utm_source=cerebras\&utm_campaign=litellm)
* [LiteLLM Discord Community](https://discord.gg/wuPM9dRgDw?utm_source=cerebras\&utm_campaign=litellm)