> ## Documentation Index
> Fetch the complete documentation index at: https://inference-docs.cerebras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Get Started with Portkey

> Learn how to use Portkey's AI gateway to route, monitor, and manage Cerebras Inference API calls with advanced features like caching, load balancing, and fallbacks.

## What is Portkey?

Portkey is an AI gateway that provides a unified interface to manage, monitor, and optimize your LLM applications. It offers features like request routing, caching, load balancing, fallbacks, and comprehensive observability—all while maintaining compatibility with the OpenAI SDK.

With Portkey, you can:

* **Monitor and trace** all your Cerebras API calls in real-time
* **Cache responses** to reduce costs and improve latency
* **Implement fallbacks** to other providers for reliability
* **Load balance** across multiple API keys or models
* **Set budgets and rate limits** to control spending
* **Analyze usage patterns** with detailed analytics

Learn more at [Portkey](https://portkey.ai?utm_source=cerebras\&utm_campaign=portkey).

## Prerequisites

Before you begin, ensure you have:

* **Cerebras API Key** - Get a free API key [here](https://cloud.cerebras.ai/?utm_source=3pi_portkey\&utm_campaign=integrations).
* **Portkey Account** - Visit [Portkey](https://app.portkey.ai?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey) and create a free account.
* **Portkey API Key** - After signing up, generate your Portkey API key from the dashboard under **Settings** > **API Keys**.
* **Python 3.11 or higher**

## Configure Portkey

<Steps>
  <Step title="Install the Portkey SDK">
    Install the Portkey SDK for your preferred language. The SDK provides a drop-in replacement for the OpenAI client with additional gateway features.

    <CodeGroup>
      ```bash Python theme={null}
      pip install portkey-ai openai python-dotenv
      ```
    </CodeGroup>
  </Step>

  <Step title="Configure environment variables">
    Create a `.env` file in your project directory to store your Portkey API key:

    ```bash theme={null}
    PORTKEY_API_KEY=your-portkey-api-key-here
    ```

    <Note>
      With Portkey's new integration, you no longer need to create virtual keys. Simply use the `@cerebras/` prefix with your model name, and Portkey will route requests directly to Cerebras.
    </Note>
  </Step>

  <Step title="Initialize the Portkey client">
    Set up the Portkey client with your API key. The client is compatible with the OpenAI SDK interface, making it easy to integrate into existing code.

    <CodeGroup>
      ```python Python theme={null}
      import os
      from portkey_ai import Portkey

      # Initialize Portkey client
      portkey = Portkey(
          api_key=os.getenv("PORTKEY_API_KEY")
      )
      ```
    </CodeGroup>

    <Note>
      Portkey automatically routes requests to Cerebras when you use the `@cerebras/` model prefix. No additional configuration needed!
    </Note>
  </Step>

  <Step title="Make your first request">
    Now you can make requests to Cerebras through Portkey's gateway. Use the `@cerebras/` prefix with your model name. All requests are automatically logged, monitored, and can leverage Portkey's advanced features.

    <CodeGroup>
      ```python Python theme={null}
      import os
      from portkey_ai import Portkey

      portkey = Portkey(
          api_key=os.getenv("PORTKEY_API_KEY")
      )

      # Make a chat completion request using @cerebras/ prefix
      response = portkey.chat.completions.create(
          model="@cerebras/gpt-oss-120b",
          messages=[
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "Explain quantum computing in simple terms."}
          ],
          max_tokens=500
      )

      print(response.choices[0].message.content)
      ```
    </CodeGroup>

    After running this code, you'll see the request appear in your Portkey dashboard with detailed logs, latency metrics, and token usage.
  </Step>
</Steps>

## Use Portkey with the OpenAI SDK

If you prefer to use the standard OpenAI SDK, you can route requests through Portkey by configuring the base URL and headers. This approach gives you Portkey's observability features while maintaining your existing OpenAI SDK code.

<CodeGroup>
  ```python Python theme={null}
  import os
  from openai import OpenAI
  from dotenv import load_dotenv

  load_dotenv()

  # Initialize OpenAI client with Portkey gateway
  client = OpenAI(
      api_key="portkey",  # Dummy key when using Portkey
      base_url="https://api.portkey.ai/v1",
      default_headers={
          "x-portkey-api-key": os.getenv("PORTKEY_API_KEY"),
          "X-Cerebras-3rd-Party-Integration": "Portkey"
      }
  )

  # Make a request using @cerebras/ prefix
  response = client.chat.completions.create(
      model="@cerebras/gpt-oss-120b",
      messages=[
          {"role": "user", "content": "What are the benefits of AI gateways?"}
      ]
  )

  print(response.choices[0].message.content)
  ```

  ```bash cURL theme={null}
  curl https://api.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -H "X-Cerebras-3rd-Party-Integration: Portkey" \
    -d '{
      "model": "@cerebras/gpt-oss-120b",
      "messages": [
        {
          "role": "user",
          "content": "What are the benefits of AI gateways?"
        }
      ]
    }'
  ```
</CodeGroup>

## Advanced Features

### Response Caching

Portkey can cache responses to reduce costs and improve latency for repeated queries. When you enable caching, identical requests return cached responses instantly without calling the Cerebras API.

<CodeGroup>
  ```python Python theme={null}
  import os
  from portkey_ai import Portkey
  from dotenv import load_dotenv

  load_dotenv()

  portkey = Portkey(
      api_key=os.getenv("PORTKEY_API_KEY")
  )

  # Enable simple caching for this request
  response = portkey.with_options(
      cache="simple",
      cache_force_refresh=False
  ).chat.completions.create(
      model="@cerebras/gpt-oss-120b",
      messages=[{"role": "user", "content": "What is the capital of France?"}]
  )

  print(response.choices[0].message.content)
  ```
</CodeGroup>

Portkey supports two cache modes:

* **Simple cache**: Caches based on the exact request parameters
* **Semantic cache**: Uses embeddings to match semantically similar requests

### Fallback Configuration

Set up automatic fallbacks to other providers or models if Cerebras is unavailable. This ensures your application remains resilient even during outages or rate limiting.

To configure fallbacks:

1. Log in to your [Portkey dashboard](https://app.portkey.ai?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey)
2. Navigate to **Configs** and create a new configuration
3. Add multiple providers (e.g., Cerebras as primary, OpenAI as fallback)
4. Set the strategy mode to "fallback"
5. Save the configuration and use its ID in your code

Learn more about [fallback configurations](https://docs.portkey.ai/docs/product/ai-gateway/fallbacks?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey) in the Portkey documentation.

### Load Balancing

Distribute requests across multiple API keys or providers to increase throughput and avoid rate limits. Portkey supports multiple load balancing strategies including round-robin, weighted, and priority-based distribution.

To configure load balancing:

1. Log in to your [Portkey dashboard](https://app.portkey.ai?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey)
2. Navigate to **Configs** and create a new configuration
3. Add multiple targets with the same or different providers
4. Set the strategy mode to "loadbalance"
5. Configure weights for each target (optional)
6. Save the configuration and use its ID in your code

Learn more about [load balancing strategies](https://docs.portkey.ai/docs/product/ai-gateway/load-balancing?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey) in the Portkey documentation.

### Request Tracing with Metadata

Add custom metadata to track requests across your application. This helps you analyze usage patterns, debug issues, and attribute costs to specific users or features.

<CodeGroup>
  ```python Python theme={null}
  import os
  from portkey_ai import Portkey
  from dotenv import load_dotenv

  load_dotenv()

  portkey = Portkey(
      api_key=os.getenv("PORTKEY_API_KEY")
  )

  response = portkey.with_options(
      metadata={
          "user_id": "user_123",
          "session_id": "session_456",
          "environment": "production",
          "feature": "chat_assistant"
      }
  ).chat.completions.create(
      model="@cerebras/gpt-oss-120b",
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```
</CodeGroup>

You can filter and analyze requests by metadata in the Portkey dashboard, making it easy to track usage by user, feature, or environment.

### Budget and Rate Limits

Set spending limits and rate limits to control costs and prevent unexpected charges. Configure these in your Portkey dashboard under **Settings** > **Budgets**.

```python Python theme={null}
import os
from portkey_ai import Portkey
from dotenv import load_dotenv

load_dotenv()

portkey = Portkey(
    api_key=os.getenv("PORTKEY_API_KEY")
)

# Budget limits are configured in the Portkey dashboard
# Requests will be rejected if limits are exceeded
response = portkey.chat.completions.create(
    model="@cerebras/gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

## Supported Cerebras Models

All current Cerebras models are available through Portkey:

| Model                    | Parameters | Best For                       |
| ------------------------ | ---------- | ------------------------------ |
| `@cerebras/gpt-oss-120b` | 120B       | Most capable open-source model |
| `@cerebras/gpt-oss-120b` | 8B         | Fast responses, simple tasks   |

Learn more about each model in the [Cerebras models documentation](/models).

## Monitoring and Analytics

Portkey's dashboard provides comprehensive insights into your Cerebras API usage. Access your dashboard at [app.portkey.ai](https://app.portkey.ai?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey).

### Request Logs

View all requests with full details including:

* Complete prompts and responses
* Token usage and costs
* Latency and performance metrics
* Custom metadata
* Error messages and stack traces

### Performance Metrics

Track key performance indicators:

* **Latency**: P50, P95, and P99 response times
* **Throughput**: Requests per second and tokens per second
* **Success Rate**: Percentage of successful requests
* **Cache Hit Rate**: Percentage of requests served from cache

### Cost Analytics

Monitor spending across:

* Different models and providers
* Time periods (hourly, daily, monthly)
* Users, features, or environments (via metadata)
* API keys and virtual keys

### Custom Dashboards

Create custom views filtered by:

* Metadata fields (user\_id, environment, etc.)
* Model or provider
* Time range
* Success or error status
* Cache hits or misses

## Troubleshooting

<AccordionGroup>
  <Accordion title="Model Not Found or Invalid">
    If you see an error about an invalid model:

    * Ensure you're using the `@cerebras/` prefix with your model name (e.g., `@cerebras/gpt-oss-120b`)
    * Check that the model name is spelled correctly
    * Verify the model is available in the [Cerebras models documentation](/models)
    * Confirm your Portkey API key is valid and active
  </Accordion>

  <Accordion title="Requests Not Appearing in Dashboard">
    If requests aren't showing up in your Portkey dashboard:

    * Confirm your `PORTKEY_API_KEY` is correct and active
    * Check that you're using the Portkey client or routing through the Portkey gateway
    * Verify your network allows outbound connections to `api.portkey.ai`
    * Check the browser console for any CORS or network errors
    * Wait a few seconds—there may be a slight delay in log processing
  </Accordion>

  <Accordion title="Rate Limit Errors">
    If you're hitting rate limits:

    * Check your Cerebras account's rate limits in the [Cerebras dashboard](https://cloud.cerebras.ai?utm_source=3pi_portkey\&utm_campaign=integrations)
    * Implement request queuing or exponential backoff retry logic
    * Use Portkey's load balancing feature to distribute requests across multiple API keys
    * Consider upgrading your Cerebras plan for higher rate limits
  </Accordion>

  <Accordion title="Cache Not Working">
    If caching isn't reducing your costs:

    * Ensure you're using identical request parameters for cache hits (model, messages, temperature, etc.)
    * Check that `cache_force_refresh` is set to `False`
    * Verify caching is enabled in your Portkey organization settings
    * Review cache analytics in the dashboard to see hit rates
    * Consider using semantic cache for similar but not identical queries
  </Accordion>

  <Accordion title="Fallback Not Triggering">
    If fallbacks aren't working as expected:

    * Verify your fallback configuration includes valid virtual keys
    * Check that the primary provider is actually failing (not just slow)
    * Review the fallback strategy mode (`fallback` vs `loadbalance`)
    * Check logs in the dashboard to see which provider handled each request
  </Accordion>

  <Accordion title="High Latency">
    If you're experiencing slower response times:

    * Check the Portkey status page for any ongoing issues
    * Review latency metrics in the dashboard to identify patterns
    * Consider enabling caching for frequently asked questions
    * Verify your network connection and geographic location relative to Portkey's servers
    * Try using a different Cerebras model (smaller models are faster)
  </Accordion>
</AccordionGroup>

<Note>
  For additional support, visit [Portkey's documentation](https://docs.portkey.ai?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey), contact their [support team](https://portkey.ai/support?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey), or join their [Discord community](https://discord.gg/portkey?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey).
</Note>

## Next Steps

Now that you have Portkey set up with Cerebras, explore these advanced features:

* **[Prompt Templates](https://docs.portkey.ai/docs/product/prompt-library?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey)** - Create reusable prompt templates with variables
* **[Guardrails](https://docs.portkey.ai/docs/product/guardrails?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey)** - Filter inputs and outputs for safety and compliance
* **[Continuous Logging](https://docs.portkey.ai/docs/product/observability/logs?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey)** - Export logs to your data warehouse
* **[A/B Testing](https://docs.portkey.ai/docs/product/ai-gateway/multimodal-capabilities/vision?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey)** - Compare different models or prompts
* **[Cerebras Models](/models)** - Explore all available Cerebras models and their capabilities

For more examples and use cases, check out [Portkey's cookbook](https://docs.portkey.ai/docs/integrations/libraries/langchain-python?utm_source=cerebras_docs\&utm_medium=integration_page\&utm_campaign=3pi_portkey).