Monitor and evaluate CrewAI agent performance using Patronus AI’s comprehensive evaluation platform for LLM outputs and agent behaviors.
PatronusEvalTool
, which allows agents to select the most appropriate evaluator and criteria:
PatronusPredefinedCriteriaEvalTool
, which uses predefined evaluator and criteria:
PatronusLocalEvaluatorTool
, which uses custom function evaluators:
PatronusEvalTool
does not require any parameters during initialization. It automatically fetches available evaluators and criteria from the Patronus API.
PatronusPredefinedCriteriaEvalTool
accepts the following parameters during initialization:
[{"evaluator": "judge", "criteria": "contains-code"}]
.PatronusLocalEvaluatorTool
accepts the following parameters during initialization:
PatronusEvalTool
and PatronusPredefinedCriteriaEvalTool
, the following parameters are required when calling the tool:
PatronusLocalEvaluatorTool
, the same parameters are required, but the evaluator and gold answer are specified during initialization.