Braintrust Integration
This guide demonstrates how to integrate Braintrust with CrewAI using OpenTelemetry for comprehensive tracing and evaluation. By the end of this guide, you will be able to trace your CrewAI agents, monitor their performance, and evaluate their outputs using Braintrust’s powerful observability platform.What is Braintrust? Braintrust is an AI evaluation and observability platform that provides comprehensive tracing, evaluation, and monitoring for AI applications with built-in experiment tracking and performance analytics.
Get Started
We’ll walk through a simple example of using CrewAI and integrating it with Braintrust via OpenTelemetry for comprehensive observability and evaluation.Step 1: Install Dependencies
Step 2: Set Up Environment Variables
Setup Braintrust API keys and configure OpenTelemetry to send traces to Braintrust. You’ll need a Braintrust API key and your OpenAI API key.Step 3: Initialize OpenTelemetry with Braintrust
Initialize the Braintrust OpenTelemetry instrumentation to start capturing traces and send them to Braintrust.Step 4: Create a CrewAI Application
We’ll create a CrewAI application where two agents collaborate to research and write a blog post about AI advancements, with comprehensive tracing enabled.Step 5: View Traces in Braintrust
After running your crew, you can view comprehensive traces in Braintrust through different perspectives:- Trace
- Timeline
- Thread

Step 6: Evaluate via SDK (Experiments)
You can also run evaluations using Braintrust’s Eval SDK. This is useful for comparing versions or scoring outputs offline. Below is a Python example using theEval
class with the crew we created above:
Key Features of Braintrust Integration
- Comprehensive Tracing: Track all agent interactions, tool usage, and LLM calls
- Performance Monitoring: Monitor execution times, token usage, and success rates
- Experiment Tracking: Compare different crew configurations and models
- Automated Evaluation: Set up custom evaluation metrics for crew outputs
- Error Tracking: Monitor and debug failures across your crew executions
- Cost Analysis: Track token usage and associated costs
Version Compatibility Information
- Python 3.8+
- CrewAI >= 0.86.0
- Braintrust >= 0.1.0
- OpenTelemetry SDK >= 1.31.0
References
- Braintrust Documentation - Overview of the Braintrust platform
- Braintrust CrewAI Integration - Official CrewAI integration guide
- Braintrust Eval SDK - Run experiments via the SDK
- CrewAI Documentation - Overview of the CrewAI framework
- OpenTelemetry Docs - OpenTelemetry guide
- Braintrust GitHub - Source code for Braintrust SDK