​
The CrewAI Approach to LLM Selection

Rather than prescriptive model recommendations, we advocate for a thinking framework that helps you make informed decisions based on your specific use case, constraints, and requirements. The LLM landscape evolves rapidly, with new models emerging regularly and existing ones being updated frequently. What matters most is developing a systematic approach to evaluation that remains relevant regardless of which specific models are available.

This guide focuses on strategic thinking rather than specific model recommendations, as the LLM landscape evolves rapidly.

​
Quick Decision Framework

1

Analyze Your Tasks

Begin by deeply understanding what your tasks actually require. Consider the cognitive complexity involved, the depth of reasoning needed, the format of expected outputs, and the amount of context the model will need to process. This foundational analysis will guide every subsequent decision.

2

Map Model Capabilities

Once you understand your requirements, map them to model strengths. Different model families excel at different types of work; some are optimized for reasoning and analysis, others for creativity and content generation, and others for speed and efficiency.

3

Consider Constraints

Factor in your real-world operational constraints including budget limitations, latency requirements, data privacy needs, and infrastructure capabilities. The theoretically best model may not be the practically best choice for your situation.

4

Test and Iterate

Start with reliable, well-understood models and optimize based on actual performance in your specific use case. Real-world results often differ from theoretical benchmarks, so empirical testing is crucial.

​
Core Selection Framework

​
a. Task-First Thinking

The most critical step in LLM selection is understanding what your task actually demands. Too often, teams select models based on general reputation or benchmark scores without carefully analyzing their specific requirements. This approach leads to either over-engineering simple tasks with expensive, complex models, or under-powering sophisticated work with models that lack the necessary capabilities.

  • Simple Tasks represent the majority of everyday AI work and include basic instruction following, straightforward data processing, and simple formatting operations. These tasks typically have clear inputs and outputs with minimal ambiguity. The cognitive load is low, and the model primarily needs to follow explicit instructions rather than engage in complex reasoning.

  • Complex Tasks require multi-step reasoning, strategic thinking, and the ability to handle ambiguous or incomplete information. These might involve analyzing multiple data sources, developing comprehensive strategies, or solving problems that require breaking down into smaller components. The model needs to maintain context across multiple reasoning steps and often must make inferences that aren’t explicitly stated.

  • Creative Tasks demand a different type of cognitive capability focused on generating novel, engaging, and contextually appropriate content. This includes storytelling, marketing copy creation, and creative problem-solving. The model needs to understand nuance, tone, and audience while producing content that feels authentic and engaging rather than formulaic.

​
b. Model Capability Mapping

Understanding model capabilities requires looking beyond marketing claims and benchmark scores to understand the fundamental strengths and limitations of different model architectures and training approaches.

​
Strategic Configuration Patterns

​
a. Multi-Model Approach

Use different models for different purposes within the same crew to optimize both performance and cost.

The most sophisticated CrewAI implementations often employ multiple models strategically, assigning different models to different agents based on their specific roles and requirements. This approach allows teams to optimize for both performance and cost by using the most appropriate model for each type of work.

Planning agents benefit from reasoning models that can handle complex strategic thinking and multi-step analysis. These agents often serve as the “brain” of the operation, developing strategies and coordinating other agents’ work. Content agents, on the other hand, perform best with creative models that excel at writing quality and audience engagement. Processing agents handling routine operations can use efficient models that prioritize speed and cost-effectiveness.

Example: Research and Analysis Crew

from crewai import Agent, Task, Crew, LLM

# High-capability reasoning model for strategic planning
manager_llm = LLM(model="gemini-2.5-flash-preview-05-20", temperature=0.1)

# Creative model for content generation
content_llm = LLM(model="claude-3-5-sonnet-20241022", temperature=0.7)

# Efficient model for data processing
processing_llm = LLM(model="gpt-4o-mini", temperature=0)

research_manager = Agent(
    role="Research Strategy Manager",
    goal="Develop comprehensive research strategies and coordinate team efforts",
    backstory="Expert research strategist with deep analytical capabilities",
    llm=manager_llm,  # High-capability model for complex reasoning
    verbose=True
)

content_writer = Agent(
    role="Research Content Writer",
    goal="Transform research findings into compelling, well-structured reports",
    backstory="Skilled writer who excels at making complex topics accessible",
    llm=content_llm,  # Creative model for engaging content
    verbose=True
)

data_processor = Agent(
    role="Data Analysis Specialist", 
    goal="Extract and organize key data points from research sources",
    backstory="Detail-oriented analyst focused on accuracy and efficiency",
    llm=processing_llm,  # Fast, cost-effective model for routine tasks
    verbose=True
)

crew = Crew(
    agents=[research_manager, content_writer, data_processor],
    tasks=[...],  # Your specific tasks
    manager_llm=manager_llm,  # Manager uses the reasoning model
    verbose=True
)

The key to successful multi-model implementation is understanding how different agents interact and ensuring that model capabilities align with agent responsibilities. This requires careful planning but can result in significant improvements in both output quality and operational efficiency.

​
b. Component-Specific Selection

The manager LLM plays a crucial role in hierarchical CrewAI processes, serving as the coordination point for multiple agents and tasks. This model needs to excel at delegation, task prioritization, and maintaining context across multiple concurrent operations.

Effective manager LLMs require strong reasoning capabilities to make good delegation decisions, consistent performance to ensure predictable coordination, and excellent context management to track the state of multiple agents simultaneously. The model needs to understand the capabilities and limitations of different agents while optimizing task allocation for efficiency and quality.

Cost considerations are particularly important for manager LLMs since they’re involved in every operation. The model needs to provide sufficient capability for effective coordination while remaining cost-effective for frequent use. This often means finding models that offer good reasoning capabilities without the premium pricing of the most sophisticated options.

​
Task Definition Framework

​
a. Focus on Clarity Over Complexity

Effective task definition is often more important than model selection in determining the quality of CrewAI outputs. Well-defined tasks provide clear direction and context that enable even modest models to perform well, while poorly defined tasks can cause even sophisticated models to produce unsatisfactory results.

​
b. Task Sequencing Strategy

Sequential task dependencies are essential when tasks build upon previous outputs, information flows from one task to another, or quality depends on the completion of prerequisite work. This approach ensures that each task has access to the information and context it needs to succeed.

Implementing sequential dependencies effectively requires using the context parameter to chain related tasks, building complexity gradually through task progression, and ensuring that each task produces outputs that serve as meaningful inputs for subsequent tasks. The goal is to maintain logical flow between dependent tasks while avoiding unnecessary bottlenecks.

Sequential dependencies work best when there’s a clear logical progression from one task to another and when the output of one task genuinely improves the quality or feasibility of subsequent tasks. However, they can create bottlenecks if not managed carefully, so it’s important to identify which dependencies are truly necessary versus those that are merely convenient.

​
Optimizing Agent Configuration for LLM Performance

​
a. Role-Driven LLM Selection

Generic agent roles make it impossible to select the right LLM. Specific roles enable targeted model optimization.

The specificity of your agent roles directly determines which LLM capabilities matter most for optimal performance. This creates a strategic opportunity to match precise model strengths with agent responsibilities.

Generic vs. Specific Role Impact on LLM Choice:

When defining roles, think about the specific domain knowledge, working style, and decision-making frameworks that would be most valuable for the tasks the agent will handle. The more specific and contextual the role definition, the better the model can embody that role effectively.

# ✅ Specific role - clear LLM requirements
specific_agent = Agent(
    role="SaaS Revenue Operations Analyst",  # Clear domain expertise needed
    goal="Analyze recurring revenue metrics and identify growth opportunities",
    backstory="Specialist in SaaS business models with deep understanding of ARR, churn, and expansion revenue",
    llm=LLM(model="gpt-4o")  # Reasoning model justified for complex analysis
)

Role-to-Model Mapping Strategy:

  • “Research Analyst” → Reasoning model (GPT-4o, Claude Sonnet) for complex analysis
  • “Content Editor” → Creative model (Claude, GPT-4o) for writing quality
  • “Data Processor” → Efficient model (GPT-4o-mini, Gemini Flash) for structured tasks
  • “API Coordinator” → Function-calling optimized model (GPT-4o, Claude) for tool usage

​
b. Backstory as Model Context Amplifier

Strategic backstories multiply your chosen LLM’s effectiveness by providing domain-specific context that generic prompting cannot achieve.

A well-crafted backstory transforms your LLM choice from generic capability to specialized expertise. This is especially crucial for cost optimization - a well-contextualized efficient model can outperform a premium model without proper context.

Context-Driven Performance Example:

# Context amplifies model effectiveness
domain_expert = Agent(
    role="B2B SaaS Marketing Strategist",
    goal="Develop comprehensive go-to-market strategies for enterprise software",
    backstory="""
    You have 10+ years of experience scaling B2B SaaS companies from Series A to IPO. 
    You understand the nuances of enterprise sales cycles, the importance of product-market 
    fit in different verticals, and how to balance growth metrics with unit economics. 
    You've worked with companies like Salesforce, HubSpot, and emerging unicorns, giving 
    you perspective on both established and disruptive go-to-market strategies.
    """,
    llm=LLM(model="claude-3-5-sonnet", temperature=0.3)  # Balanced creativity with domain knowledge
)

# This context enables Claude to perform like a domain expert
# Without it, even it would produce generic marketing advice

Backstory Elements That Enhance LLM Performance:

  • Domain Experience: “10+ years in enterprise SaaS sales”
  • Specific Expertise: “Specializes in technical due diligence for Series B+ rounds”
  • Working Style: “Prefers data-driven decisions with clear documentation”
  • Quality Standards: “Insists on citing sources and showing analytical work”

​
c. Holistic Agent-LLM Optimization

The most effective agent configurations create synergy between role specificity, backstory depth, and LLM selection. Each element reinforces the others to maximize model performance.

Optimization Framework:

# Example: Technical Documentation Agent
tech_writer = Agent(
    role="API Documentation Specialist",  # Specific role for clear LLM requirements
    goal="Create comprehensive, developer-friendly API documentation",
    backstory="""
    You're a technical writer with 8+ years documenting REST APIs, GraphQL endpoints, 
    and SDK integration guides. You've worked with developer tools companies and 
    understand what developers need: clear examples, comprehensive error handling, 
    and practical use cases. You prioritize accuracy and usability over marketing fluff.
    """,
    llm=LLM(
        model="claude-3-5-sonnet",  # Excellent for technical writing
        temperature=0.1  # Low temperature for accuracy
    ),
    tools=[code_analyzer_tool, api_scanner_tool],
    verbose=True 
)

Alignment Checklist:

  • ✅ Role Specificity: Clear domain and responsibilities
  • ✅ LLM Match: Model strengths align with role requirements
  • ✅ Backstory Depth: Provides domain context the LLM can leverage
  • ✅ Tool Integration: Tools support the agent’s specialized function
  • ✅ Parameter Tuning: Temperature and settings optimize for role needs

The key is creating agents where every configuration choice reinforces your LLM selection strategy, maximizing performance while optimizing costs.

​
Practical Implementation Checklist

Rather than repeating the strategic framework, here’s a tactical checklist for implementing your LLM selection decisions in CrewAI:

Audit Your Current Setup

What to Review:

  • Are all agents using the same LLM by default?
  • Which agents handle the most complex reasoning tasks?
  • Which agents primarily do data processing or formatting?
  • Are any agents heavily tool-dependent?

Action: Document current agent roles and identify optimization opportunities.

Implement Crew-Level Strategy

Set Your Baseline:

# Start with a reliable default for the crew
default_crew_llm = LLM(model="gpt-4o-mini")  # Cost-effective baseline

crew = Crew(
    agents=[...],
    tasks=[...],
    memory=True
)

Action: Establish your crew’s default LLM before optimizing individual agents.

Optimize High-Impact Agents

Identify and Upgrade Key Agents:

# Manager or coordination agents
manager_agent = Agent(
    role="Project Manager",
    llm=LLM(model="gemini-2.5-flash-preview-05-20"),  # Premium for coordination
    # ... rest of config
)

# Creative or customer-facing agents  
content_agent = Agent(
    role="Content Creator",
    llm=LLM(model="claude-3-5-sonnet"),  # Best for writing
    # ... rest of config
)

Action: Upgrade 20% of your agents that handle 80% of the complexity.

Validate with Enterprise Testing

Once you deploy your agents to production:

  • Use CrewAI Enterprise platform to A/B test your model selections
  • Run multiple iterations with real inputs to measure consistency and performance
  • Compare cost vs. performance across your optimized setup
  • Share results with your team for collaborative decision-making

Action: Replace guesswork with data-driven validation using the testing platform.

​
When to Use Different Model Types

Reasoning models become essential when tasks require genuine multi-step logical thinking, strategic planning, or high-level decision making that benefits from systematic analysis. These models excel when problems need to be broken down into components and analyzed systematically rather than handled through pattern matching or simple instruction following.

Consider reasoning models for business strategy development, complex data analysis that requires drawing insights from multiple sources, multi-step problem solving where each step depends on previous analysis, and strategic planning tasks that require considering multiple variables and their interactions.

However, reasoning models often come with higher costs and slower response times, so they’re best reserved for tasks where their sophisticated capabilities provide genuine value rather than being used for simple operations that don’t require complex reasoning.

​
Common CrewAI Model Selection Pitfalls

​
Testing and Iteration Strategy

Start Simple

Begin with reliable, general-purpose models that are well-understood and widely supported. This provides a stable foundation for understanding your specific requirements and performance expectations before optimizing for specialized needs.

Measure What Matters

Develop metrics that align with your specific use case and business requirements rather than relying solely on general benchmarks. Focus on measuring outcomes that directly impact your success rather than theoretical performance indicators.

Iterate Based on Results

Make model changes based on observed performance in your specific context rather than theoretical considerations or general recommendations. Real-world performance often differs significantly from benchmark results or general reputation.

Consider Total Cost

Evaluate the complete cost of ownership including model costs, development time, maintenance overhead, and operational complexity. The cheapest model per token may not be the most cost-effective choice when considering all factors.

Focus on understanding your requirements first, then select models that best match those needs. The best LLM choice is the one that consistently delivers the results you need within your operational constraints.

​
Enterprise-Grade Model Validation

For teams serious about optimizing their LLM selection, the CrewAI Enterprise platform provides sophisticated testing capabilities that go far beyond basic CLI testing. The platform enables comprehensive model evaluation that helps you make data-driven decisions about your LLM strategy.

Advanced Testing Features:

  • Multi-Model Comparison: Test multiple LLMs simultaneously across the same tasks and inputs. Compare performance between GPT-4o, Claude, Llama, Groq, Cerebras, and other leading models in parallel to identify the best fit for your specific use case.

  • Statistical Rigor: Configure multiple iterations with consistent inputs to measure reliability and performance variance. This helps identify models that not only perform well but do so consistently across runs.

  • Real-World Validation: Use your actual crew inputs and scenarios rather than synthetic benchmarks. The platform allows you to test with your specific industry context, company information, and real use cases for more accurate evaluation.

  • Comprehensive Analytics: Access detailed performance metrics, execution times, and cost analysis across all tested models. This enables data-driven decision making rather than relying on general model reputation or theoretical capabilities.

  • Team Collaboration: Share testing results and model performance data across your team, enabling collaborative decision-making and consistent model selection strategies across projects.

Go to app.crewai.com to get started!

The Enterprise platform transforms model selection from guesswork into a data-driven process, enabling you to validate the principles in this guide with your actual use cases and requirements.

​
Key Principles Summary

Task-Driven Selection

Choose models based on what the task actually requires, not theoretical capabilities or general reputation.

Capability Matching

Align model strengths with agent roles and responsibilities for optimal performance.

Strategic Consistency

Maintain coherent model selection strategy across related components and workflows.

Practical Testing

Validate choices through real-world usage rather than benchmarks alone.

Iterative Improvement

Start simple and optimize based on actual performance and needs.

Operational Balance

Balance performance requirements with cost and complexity constraints.

Remember: The best LLM choice is the one that consistently delivers the results you need within your operational constraints. Focus on understanding your requirements first, then select models that best match those needs.

​
Current Model Landscape (June 2025)

Snapshot in Time: The following model rankings represent current leaderboard standings as of June 2025, compiled from LMSys Arena, Artificial Analysis, and other leading benchmarks. LLM performance, availability, and pricing change rapidly. Always conduct your own evaluations with your specific use cases and data.

​
Leading Models by Category

The tables below show a representative sample of current top-performing models across different categories, with guidance on their suitability for CrewAI agents:

These tables/metrics showcase selected leading models in each category and are not exhaustive. Many excellent models exist beyond those listed here. The goal is to illustrate the types of capabilities to look for rather than provide a complete catalog.

Best for Manager LLMs and Complex Analysis

ModelIntelligence ScoreCost ($/M tokens)SpeedBest Use in CrewAI
o370$17.50FastManager LLM for complex multi-agent coordination
Gemini 2.5 Pro69$3.44FastStrategic planning agents, research coordination
DeepSeek R168$0.96ModerateCost-effective reasoning for budget-conscious crews
Claude 4 Sonnet53$6.00FastAnalysis agents requiring nuanced understanding
Qwen3 235B (Reasoning)62$2.63ModerateOpen-source alternative for reasoning tasks

These models excel at multi-step reasoning and are ideal for agents that need to develop strategies, coordinate other agents, or analyze complex information.

​
Selection Framework for Current Models

​
Key Considerations for Model Selection

  • Performance Trends: The current landscape shows strong competition between reasoning-focused models (o3, Gemini 2.5 Pro) and balanced models (Claude 4, GPT-4.1). Specialized models like DeepSeek R1 offer excellent cost-performance ratios.

  • Speed vs. Intelligence Trade-offs: Models like Llama 4 Scout prioritize speed (2,600 tokens/s) while maintaining reasonable intelligence, whereas models like o3 maximize reasoning capability at the cost of speed and price.

  • Open Source Viability: The gap between open-source and proprietary models continues to narrow, with models like Llama 4 Maverick and DeepSeek V3 offering competitive performance at attractive price points. Fast inference providers particularly shine with open-source models, often delivering better speed-to-cost ratios than proprietary alternatives.

Testing is Essential: Leaderboard rankings provide general guidance, but your specific use case, prompting style, and evaluation criteria may produce different results. Always test candidate models with your actual tasks and data before making final decisions.

​
Practical Implementation Strategy

1

Start with Proven Models

Begin with well-established models like GPT-4.1, Claude 3.7 Sonnet, or Gemini 2.0 Flash that offer good performance across multiple dimensions and have extensive real-world validation.

2

Identify Specialized Needs

Determine if your crew has specific requirements (coding, reasoning, speed) that would benefit from specialized models like Claude 4 Sonnet for development or o3 for complex analysis. For speed-critical applications, consider fast inference providers like Groq alongside model selection.

3

Implement Multi-Model Strategy

Use different models for different agents based on their roles. High-capability models for managers and complex tasks, efficient models for routine operations.

4

Monitor and Optimize

Track performance metrics relevant to your use case and be prepared to adjust model selections as new models are released or pricing changes.