Agent Monitoring with OpenLIT
Quickly start monitoring your Agents in just a single line of code with OpenTelemetry.
OpenLIT Overview
OpenLIT is an open-source tool that makes it simple to monitor the performance of AI agents, LLMs, VectorDBs, and GPUs with just one line of code.
It provides OpenTelemetry-native tracing and metrics to track important parameters like cost, latency, interactions and task sequences. This setup enables you to track hyperparameters and monitor for performance issues, helping you find ways to enhance and fine-tune your agents over time.
OpenLIT Dashboard
Features
- Analytics Dashboard: Monitor your Agents health and performance with detailed dashboards that track metrics, costs, and user interactions.
- OpenTelemetry-native Observability SDK: Vendor-neutral SDKs to send traces and metrics to your existing observability tools like Grafana, DataDog and more.
- Cost Tracking for Custom and Fine-Tuned Models: Tailor cost estimations for specific models using custom pricing files for precise budgeting.
- Exceptions Monitoring Dashboard: Quickly spot and resolve issues by tracking common exceptions and errors with a monitoring dashboard.
- Compliance and Security: Detect potential threats such as profanity and PII leaks.
- Prompt Injection Detection: Identify potential code injection and secret leaks.
- API Keys and Secrets Management: Securely handle your LLM API keys and secrets centrally, avoiding insecure practices.
- Prompt Management: Manage and version Agent prompts using PromptHub for consistent and easy access across Agents.
- Model Playground Test and compare different models for your CrewAI agents before deployment.
Setup Instructions
Deploy OpenLIT
Git Clone OpenLIT Repository
Start Docker Compose
From the root directory of the OpenLIT Repo, Run the below command:
Install OpenLIT SDK
Initialize OpenLIT in Your Application
Add the following two lines to your application code:
Example Usage for monitoring a CrewAI Agent:
Refer to OpenLIT Python SDK repository for more advanced configurations and use cases.
Visualize and Analyze
With the Agent Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your Agent’s performance, behavior, and identify areas of improvement.
Just head over to OpenLIT at 127.0.0.1:3000
on your browser to start exploring. You can login using the default credentials
- Email:
user@openlit.io
- Password:
openlituser
OpenLIT Dashboard
Was this page helpful?