Wondering about services to test on either a 16gb ram “AI Capable” arm64 board or on a laptop with modern rtx. Only looking for open source options, but curious to hear what people say. Cheers!

  • kata1yst@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    20
    arrow-down
    2
    ·
    2 days ago

    I use OLlama & Open-WebUI, OLlama on my gaming rig and Open-WebUI as a frontend on my server.

    It’s been a really powerful combo!

    • kiol@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      1
      ·
      2 days ago

      Would you please talk more about it. I forgot about Open-webui, but intending to start playing with. Honestly, what do you actually do with it?

      • mac@lemm.ee
        link
        fedilink
        English
        arrow-up
        4
        ·
        edit-2
        1 day ago

        I have Linkwarden pointed at my ollama deployment, so it auto tags links that I archive which is nice.

        I’ve seen other people send images captured on their security cameras on frigate to ollama to get it to describe the image

        There’s a bunch of other use cases I’ve thought of for coding projects, but haven’t started on any of them yet

      • Lucy :3@feddit.org
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 day ago

        Sex chats. For other uses, just simple searches are better 99% of the time. And for the 1%, something like the Kagis FastGPT helps to find the correct keywords.