← all ideas

Splitting Large Models for Efficiency

The idea

The idea is to not use a single, large AI model for every task but instead break down workflows into smaller, more efficient tasks. Instead of employing the largest, most expensive model for every job, consider whether smaller, more affordable models can handle specific pieces of the workflow. By doing this, you can save both time and money without sacrificing the quality of your work.

Why it works

• Breaking down large models into smaller ones allows you to save on both latency and cost. Smaller models are less computationally intensive and can be more cost-effective.
• When you have specific tasks that smaller models can handle efficiently, you can automate them without relying on the most complex models for everything. This leads to faster processing times and reduced expenses.
• By identifying which parts of your workflow can be handled by smaller models, you can optimize your AI workflow. This process reduces the workload on your AI models, allowing them to focus on the most critical tasks.
• Smaller models can be used for repetitive or less complex tasks, freeing up the more powerful models for more demanding or critical operations. This results in a more efficient use of AI resources.
• When you break down a large task into smaller, more manageable parts, you can better assess and optimize the use of different models for specific tasks, leading to a more efficient and cost-effective AI system.

The playbook

To start implementing the idea of splitting large models into smaller ones, follow these steps:

1. Assess Your Workflow: Begin by thoroughly examining your current workflow to identify tasks that can be automated. Break down your tasks into smaller, more manageable pieces. This allows you to determine if each piece can be handled by a smaller, more efficient model rather than relying on a single, large model for everything.

1. Evaluate Model Efficiency: After assessing your workflow, evaluate each piece to see if a smaller model can perform the task effectively without compromising on quality. If a smaller model can handle a piece of the workflow, consider switching to a more lightweight model. This not only saves time and money but also ensures your workflow remains efficient and cost-effective.

1. Iterate and Refine: Once you've identified and switched to smaller models, continue to monitor and refine your workflow. As you gain more experience and familiarity with different models, you may find even more opportunities to split tasks further or optimize your workflow even more.

1. Monitor Performance and Costs: Keep a close eye on the performance and costs of your workflow. As you switch to smaller models, pay attention to how these changes affect your workflow's speed and your expenses. This will help you make informed decisions and ensure you are getting the most out of your model investments.

1. Stay Updated with AI Trends: Finally, stay informed about the latest advancements and trends in AI. The landscape is constantly evolving, and new models and techniques are being developed all the time. Staying updated can help you make the most of your workflow and continue to optimize it for efficiency and cost-effectiveness.

Where people get it wrong

The founder often advises using one of the largest AI models like O3 or Claude Opus 4.1 for every single task, thinking it will be the best solution. However, this is a common mistake, and it’s important to avoid it. Instead, ask yourself: Can a smaller, more efficient model accomplish this task effectively? Breaking down your workflow into smaller, more manageable pieces can also help. Consider automating specific tasks with a smaller model, or look for ways to divide large tasks into smaller, more lightweight parts.

Do this next

1. Break Down Your Workflow: Start by analyzing your current workflow and identifying where you are using large models for every task. Break down each task into smaller, more manageable components.

1. Evaluate Each Task: For each component, assess whether a smaller, more efficient model could accomplish the task just as well. Consider the impact on latency and cost for each part of your workflow.

1. Implement Smaller Models: Begin by implementing smaller models for the most critical and time-consuming tasks. As you see what works, gradually replace larger models with more efficient ones.

1. Monitor and Refine: Continuously monitor the performance of your new workflow. As you gain experience and understanding, refine your approach to optimize efficiency even further.