To get started with a free API key, click here.
CePO (Cerebras Planning & Optimization) is a framework that adds advanced reasoning capabilities to the Llama family of models by utilizing test-time compute. This approach enables Llama to address complex reasoning tasks that can be difficult for standard one-shot or instruct models. CePO is implemented using Cerebras inference, which currently supports llama3.3-70b at 2,200 reasoning tokens/s. This level of inference speed enables efficient test-time computation for more sophisticated reasoning tasks.

​
How CePO Works

CePO demonstrates how additional test-time computation can improve Llama’s reasoning. The process involves four main stages:
  1. Planning: The LLM produces a plan to solve a given problem step by step.
  2. Execution: The LLM executes the plan multiple times, generating multiple responses.
  3. Analysis: The model analyzes the responses to detect inconsistencies across executions, helping catch and correct mistakes.
  4. Best-of-N: Responses are evaluated within a Best-of-N framework that includes a structured confidence scoring mechanism.

​
Getting Started with CePO

1

Step 1: Prerequisites

CePO is built on the popular, open-source OptiLLM library. To get started, install OptiLLM and make sure you have the latest version of the Cerebras Inference SDK installed.
pip install --upgrade cerebras_cloud_sdk 
pip install --upgrade optillm
Next, configure your API key, which can be found in our developer platform.
export CEREBRAS_API_KEY='your_api_key_here'
2

Step 2: Run OptiLLM with CePO

Finally, run the OptiLLM script with CePO
optillm \
  --base-url https://api.cerebras.ai \
  --approach cepo
If you would like to print intermediate states in the OptiLLM log, you can optionally add:
--cepo_print_output true

​
Continued Research

Further work on CePO includes:
  • More advanced prompting frameworks that leverage comparative reasoning.
  • Synthetic data optimized for inference-time computation.
  • Enhanced verification mechanisms for complex reasoning chains.
If you have questions or would like to discuss your findings, please join the #research channel in our discord community.

​
Read More

For more information on the implementation details and results, see the full CePO announcement.