Like if I type “I have two appl…” for example, often it will suggest “apple” singular instead of plural. Just a small example, but it is really bad at predicting which variant of a word should come after the previous

  • bassomitron@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    edit-2
    11 months ago

    The kind of local/offline LLMs that would work on your phone would not be very good quality. There’s been amazing progress in quantization of LLMs to get them working on weaker GPUs with lower VRAM and CPUs, so maybe it’ll occur, but I’m not an expert.

    I also don’t foresee them linking it up to a cloud-based LLM as that’d be a shit load of queries and extremely expensive.

    • astraeus@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      11 months ago

      OpenAI is probably already handling a significant amount of queries, I think for daily use the LLM should simply initialize a word map based on user history and then update it semi-occasionally, like once a week or two. Most people don’t drastically change their vocabulary in the course of a few weeks

      • EatYouWell@lemmy.world
        link
        fedilink
        arrow-up
        2
        arrow-down
        1
        ·
        11 months ago

        We’re talking about orders of magnitude more queries if we start offloading predective text like that.