• iminahurry@discuss.tchncs.de
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    Anybody know of a good guide for hosting this with GPU? Every guide seems to be talking about running it on CPU when one would expect the opposite to be true. I have not been able to use my RTX 3060 for this so far.

    • eu8@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Take my answer with a grain of salt, but I’m pretty sure if you have a GPU you can just run the same models and it should work more efficiently for you. The only difference for you is you can run some of the larger models.