A lot of people want to use small models (< 10 GB) to drive assistants. The needs here are different from those associated with large models.

For assistant use, we want models that are small, quick, and reliable rather than those that can solve world-changing problems and write entire codebases.

To try to understand which models are currently (Nov 2026) best, I created EverydayBench, a benchmark suite tuned towards everyday tasks such as summarization, translation, and image analysis (e.g., reading a bus timetable). This benchmark suite intentionally excludes coding and long-horizon tasks. I then ran it on LFM 3 Omni, Gemma E4B, and Nemotron 3.5 E4B.

At only 2.3 GB, LFM 3 Omni Q8 performed as well as the larger Gemma E4B Q8 on summarization and was good enough on image analysis. It did well on analysis of short audio snippets (1 and 5 min), but failed on 30 min snippets. LFM also couldn’t match Gemma on translation.

In the end, that’s not bad for a model that takes less than a third of the space of Nemotron 3.5 Q4KM and Gemma E4B. However, you have to know that LFM’s translation capabilities don’t match up.

  • zackhow@programming.dev
    link
    fedilink
    English
    arrow-up
    10
    ·
    2 days ago

    I’ve tried a bunch of 2B-12B models to use in home assistant to support the voice pipeline and found Gemma E4B worked best. No real benchmarks, just based on vibes from manual testing. Felt it gave a good balance on executing commands correctly and response time.

      • zackhow@programming.dev
        link
        fedilink
        English
        arrow-up
        2
        ·
        11 hours ago

        Two things:fully local voice assistant and text descriptions from my reolink doorbell. I would say it is damn close to replacing Google Home/Alexa with the biggest weakness being the VAD on the voice satellites.

        • rando@lemmy.ml
          link
          fedilink
          English
          arrow-up
          1
          ·
          3 hours ago

          Which voice satellite are u using? I’m working with atom echo but speaker is quite small so I’m trying to figure out pushing audio out somewhere else.

          • zackhow@programming.dev
            link
            fedilink
            English
            arrow-up
            1
            ·
            2 hours ago

            I have an atom, HA VPE, and Third Reality Voice. You are correct that the atom speaker sucks. The HA VPE is better but the TRV has the best speaker of the three that I’ve used. With that said I like the VPE more due to LED ring. I create a lot of timers and the LED ring makes it easy to see how much time is left on the timer without having to use a command.