You should consider using local models instead of cloud ones. That both reduces the power you give to companies and your reliance on them remaining free.

Language translation is only as good as the data set. There’s a ton of TS/js out there (and a lot of it is horrible) so LLMs tend to be able to spit out working js (but the security properties are at best questionable). They tend to be less good at python, and obscure things like lisp are more likely to just be totally wrong. (IMHO, the entire paradigm of having machines try to make code to be read by other machines is nonsense because code is for people. That’s literally the point.)

This is also true for natural languages.

Finding things is really just about encodings, so if you’re building something like a RAG then you kind of don’t even need to bother with the inference. I’ve build half-RAGs before just to get semantic search and then just skip the inference part after I have my doc list.

This is, of course, less useful if you’re looking for something on the internet and you don’t have a local search engine. But I’m not sure that an LLM is better than just using a search engine. That’s just my take, though. Perhaps I just haven’t experienced it.

Yeah, there’s definitely a bit more nuance. I think most folks can just skip LLMs, but there are some use cases in tech.

The thing about local models is that inference is cheap. They’re trying to charge for inference to cover the cost of training, and training is what uses all the resources. If you refuse to pay for inference, then the economic model collapses. IMHO, that’s the critical part.