Like does content get mirrored to my instance and then mine distributes it ala BitTorrent or is my instance it’s own thing?

  • terribleplan@lemmy.nrd.li
    link
    fedilink
    English
    arrow-up
    25
    ·
    1 year ago

    Broadly, yes. The way federation works means anything any user on your instance is interested in will be sent to you once (at least posts/comments/votes/etc). Whenever someone on your instance views that thing that is a request that would otherwise be made to another instance. This does, however, increase the load of federation on servers hosting popular communities, as now they have to send each post/comment/vote/whatever to your instance. Unlike bit torrent there is only one place responsible for sending you all of the content that exists in a community, so the fediverse doesn’t get p2p-style network effects where every peer/sever helps even a bit.

    A single user instance is a little inefficient, unless you are actually looking at most/all of the content your instance receives, in which case it is probably a wash. The ideal for how federation is implemented in ActivityPub would be many similarly-sized (in terms of user count) instances with the most popular communities being spread out among them.

    Sadly right now the most popular instances (lemmy.world, lemmy.ml, lemmynsfw.com, kbin.social) are both where users and communities are, so the real gains to help those instances (several of which continue to struggle under the load) are really only medium and larger sized instances.

    • acqrs@acqrs.co.uk
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Isn’t a semi-significant amount of the per instance load in the database interactions though? I don’t know if that’s still true after some of the optimizations in 0.18, but by having your own instance, even 1 person instance, you no longer load the database of another instance with calculating hot/active/whatever for things you want to see, and you don’t load it when you do pulls.

      • terribleplan@lemmy.nrd.li
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 year ago

        There is certainly an inflection point. I am not sure where it falls. If you rarely use Lemmy the other servers are expending effort federating to you for no reason, so for a heavy single user it is probably a positive. I imagine it also varies based on how you browse as “new” is probably “lighter” than “hot” or “active”.

        • acqrs@acqrs.co.uk
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          Sure, but the server calculates the hot/active every 15 minutes regardless of whether or not you browse it. It’s calculated on a timed job so it’s always fresh and ready for everyone.

          • terribleplan@lemmy.nrd.li
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            I was unaware that was the case, that’s pretty expensive and inefficient to do it for every user (because each user’s “Subscribed” feed is likely different) every 15 minutes regardless of how recently they’ve visited. The inflection point is probably lower for the benefits of even single-user instances in that case.

  • thereddevil@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    16
    ·
    1 year ago

    Your instance is it’s own thing and will only push data that’s from communities created on your instance to others.

    However, since your instance will have limited users, it will consume fewer resources and your browsing experience would be faster.

    I’m not sure of where the limits are on the pushing side of things. If people from a lot of instances are subscribed to a particular community in an instance, there is a lot of outgoing data from there. This is something I’m curious about.

  • Wander@yiffit.net
    link
    fedilink
    English
    arrow-up
    12
    ·
    1 year ago

    Text gets mirrored but not images. In general it helps, but not by much unless you host for several hundred users.

    • rs5th@lemmy.scottlabs.io
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 year ago

      Link thumbnails do get mirrored. My understanding is the front end of Lemmy is pretty heavy for the big instances and the burden of federating to another instance is pretty small. One thing I’ve noticed on my instance is that sometimes inbound federation can be pretty annoyingly slow.

      • Dusty@l.dustybeer.com
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        1 year ago

        Not only annoyingly slow, but I tend to get a massive influx of posts from one community all at once. It fills my entire page with that single community. It’s been my biggest annoyance so far.

  • sab@lemmy.world
    link
    fedilink
    English
    arrow-up
    14
    arrow-down
    2
    ·
    1 year ago

    In effect, not really.

    All the communities you’re subscribed to will now also have to push all their updates (posts, comments, upvotes) to your server, even when you’re not interacting with Lemmy.

    As someone else mentioned, it would only be efficient once you have a decent (hard to pinpoint) amount of users on your server.

    • Rikudou_Sage@lemmings.world
      link
      fedilink
      English
      arrow-up
      0
      arrow-down
      4
      ·
      1 year ago

      They’ll do it once, though. Then every time you view it, you’re helping the bigger servers by serving it from your instance.

      • sab@lemmy.world
        link
        fedilink
        English
        arrow-up
        5
        ·
        1 year ago

        Yes. Once for every post, comment and vote.

        So say you have your own personal instance, and you use that to follow community news on lemmy.world. If throughout the day that community receives 10 new topics, 50 comments and 100 upvotes, it would have to make 160 calls to your server.

        So when you decide to read those 10 topics (if you even read all of them), you would then make roughly 10 api calls.

        You would be saving those last mentioned 10 calls by using your own instance, but at the cost of 160 calls made throughout the day.

        • Rikudou_Sage@lemmings.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          1 year ago

          So you need just 15 more users on your instance to break even, if you have 17 in total, you’ve saved 10 calls.

          • sab@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            ·
            edit-2
            1 year ago

            So you need just 15 more users on your instance to break even, if you have 17 in total, you’ve saved 10 calls.

            In this particular example, yes. But only if those 15 people subscribe to the exact same communities. If they don’t, the calculation gets even more complicated.

            I’m not sure why exactly you’re opposed to federation when that’s one of the biggest points of fediverse.

            Some people seem to be under the impression that setting up their own personal server is relieving the pressure on the network. What I trying to get across is that’s not the case, unless it’s being used by a reasonable amount of people.

            I can’t tell you what the sweet spot is - but my guess would be that it’s only going to be at least several dozen, more if their interests (subscriptions) don’t overlap very well.

        • ashe@lemmy.starless.one
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          The thing is that when you interact with the remote server directly it’s not 10 api calls, it’s 10 full-blown HTML webpages that have to be served to you, which are way bigger than REST API calls.

          • sab@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            ·
            1 year ago

            Generally speaking, lemmy is much more cpu bound than it is bound by bandwidth - so the added bytes don’t matter that much. The example above was just for 1 community. Now imagine the user is subscribed to a dozen communities, but doesn’t even browse lemmy that day. That’s probably thousands of api calls made to keep his server on sync, and 0 requests saved.

            Like the big instances have literally hundreds of thousands of workers running in order to get all the updates out. If one of those calls fails, it gets put back into the queue for retry.

            OP asked if having his server added to the lemmiverse would alleviate the load “Like with torrent”. That is demonstrably not the case - it only adds more workload on the other servers, with a break even point that’s highly variable. Yes, your server will be nice and snappy, but the origin servers have to pay the price - death by a thousand papercuts synchronisation calls.

            • ashe@lemmy.starless.one
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              1 year ago

              Yeah you’re right, I just felt the need to point out that API calls are not really comparable to serving a full website.

  • Thief@lemmy.myserv.one
    link
    fedilink
    English
    arrow-up
    7
    ·
    1 year ago

    Yes but only if you have a lot of users eventually. So your server can serve up hundreds of requests from several different federated servers (eg: maybe some content from beehaw, other content from lemmy.world) instead of 1 server (maybe lemmy.world) having to serve that same content 100 times from all over the place.

    If its only you and no other users, then not.

  • Brandon@lemm.ee
    link
    fedilink
    English
    arrow-up
    4
    ·
    1 year ago

    I think I’m going to host my own. I have a bunch of good, short domains I could set it up with.

  • das@lemellem.dasonic.xyz
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    It will take part of the load off, but the server still has a push updates to every instance subscribed to one of its communities. But I would guess that is much less of a demand than having an active user using the instance.

  • Kururin@talk.kururin.tech
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Should be. One instance hosting lot of community vs one instance hosting one community is less load. But your instance have to be popular enough to make a dent in the popular instances users.