Can we modify this approach to get LLMs that are good at specific programming languages or frameworks? That seems to be where local LLMs could really shine.
LLMs shine through emergent behaviour. Finding an LLM that does Rails doesn't know poetry is like finding a Rails human developer who doesn't have a hobby e.g. basketball. So what if they play basketball? They can code too!
Then it might need a new type of architecture to work. I’m not attached to LLMs. If a new model comes out that can do only the things I want it do it, then great
This has been the standard approach for more complex LLM deployments for a while now in our shop.
Using different models across iterations is also something I've found useful in my own experiments. It's like getting a fresh pair of eyes.