This site is currently struggling to handle the amount of new users. I have already upgraded the server, but it will go down regardless if half of Reddit tries to join.

However Lemmy is federated software, meaning you can interact seamlessly with communities on other instances like beehaw.org or lemmy.one. The documentation explains in more detail how this works. Use the instance list to find one where you can register. Then use the Community Browser to find interesting communities. Paste the community url into the search field to follow it.

You can help other Reddit refugees by inviting them to the same Lemmy instance where you joined. This way we can spread the load across many different servers. And users with similar interests will end up together on the same instances. Others on the same instance can also automatically see posts from all the communities that you follow.

Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements. That way the server will only go down sooner.

  • aksdb@feddit.deEnglish
    1·
    2 years ago

    I think lemmy will be bitten in the ass by not having considered clustering/horizontal scaling from the start. Federation alone as a scaling mechanism is only feasible for “nerds”. But if the network wants to grow, we will need a few scale-able large hosted instances. And if their only choice is to scale vertically, there will be a hard limit (unless we put a good old Mainframe somewhere ^^).

    Another downside of this design is: you can’t run it with high availability. If there’s only one process per instance, updating it will mean the whole instance is down. Sure, if all goes well this downtime is under a second. But if it doesn’t go well or if a migration is needed, this might quickly become hours.

    • PriorProject@lemmy.worldEnglish
      1·
      2 years ago

      I think you probably underestimate how far one can get with “vertical” scaling. Here’s the dockerfile: https://raw.githubusercontent.com/LemmyNet/lemmy/release/v0.17/docker/prod/docker-compose.yml

      • It includes 4 different containers… so there’s a way to scale out to 4 machines right away. Maybe not every container is doing an equal amount of work… but there’s some amount of immediately available machine-splitting.
      • I’m no expert, but I believe that at least the lemmy and lemmy-ui containers are stateless. If so, they’re horizontally scalable already.
      • Postgres then would likely be the main bottleneck. But postgres offers read-replicas, so again the write-load and the read-load can be hosted on separate machines. And if there’s enough read-load, you can have many replicas.

      Other comments from the admins have shown that lemmy.ml today is running on a single eight-core box and it’s currently hosting 30k registered users and over 1k active. So how much more compute capacity can we throw at “vertical” scaling on the current software architecture?

      • Just by going to a bigger single box, we can get 128 cores with no problem, a 16x bump in capacity. Does that get us to at least to 300k registered + 10k active?
      • Splitting the containers onto 4 separate machines. Does that get us 2x more?
      • Adding PG read-replicas and additional lemmy/lemm-ui containers would allow us to expand our instance footprint to maybe 6 physical machines should get us another 2x or more in performance.

      Conservatively, that’s 100x the computing capacity of the current hardware and could potentially support 1m registered users and 50k active. Now, I don’t REALLY expect this to be possible today, there will be many software bottlenecks found along the way to scaling a single instance this large. But my point is that there’s already a medium amount of horizontal scalability built into lemmy, and if the software doesn’t fall over for algorithmic reasons (which is will at first), the current infrastructure architecture allows quite a lot of growth. There’s plenty of time between now and a federation of million user instances to adopt a truly distributed storage backend if needed.

    • federico3@lemmy.mlEnglish
      1·
      2 years ago

      Indeed. If a big instance like lemmy.ml was to be shut down all the communities would be lost. This is simply not sustainable. Why would users put effort building a community if it could be gone at any time?

      • aksdb@feddit.deEnglish
        1·
        2 years ago

        That however would be a different problem. A horizontally scaled instance would be able to cope with more users, but if it shuts down for monetary, personal, or whatever reason, it’s still down.

        Protecting a community from this is what the decentralized part is for. That is already in place.

        (Although there is a middle ground where you could design the system in a way that one instance is mirrored and load-balanced across different hosters. That would actually also be quite interesting to have. But that’s another layer of complexity on top.)

        • d3Xt3r@lemmy.mlEnglish
          0·
          2 years ago

          Protecting a community from this is what the decentralized part is for. That is already in place.

          What? How is it solved exactly? If say lemmy.ml is down, what’s the point of other servers existing, if most of the content and users are here? Like, I created a few new communities on lemmy.ml, which don’t exist on say Beehaw because for some strange reason, the Beehaw admins don’t allow users to create communities. So how is going to Beehaw help me, if lemmy.ml is unavailable? Okay, so you tell me I should go to a different server then. Maybe even make a new server. Done and done. But there’s very few to zero users on that server, so those new communities and content created there might as well not exist. Also, even though Lemmy is federated, the homepage defaults to “local”, so all the new users coming in may miss out on all the other federated communities, and, if I’m reading this correctly, the federation isn’t even a fully automatic process, and some admins may even choose to put there server in a whitelist mode. All of it makes the whole “advantage” of federation, or at least Lemmy’s version of it, seem kind of pointless.

          It’s like saying, “Hey, Gmail is down so you should just use Hotmail instead.” Okay, so I can still send and receive emails, but I can’t access any of my old emails for context, and none of my contacts can reach me using my Gmail address, and none of my filters, address book and other content is available so I may not even be able to reach out to my contacts and let them know what my new email is.

          IMO the way the way the federation should’ve been designed is to use something like blockchain technology, so every instance basically has all the content and there’s only one source of truth for user accounts and data (distributed ledger), or maybe even just implement the whole thing as a plain old high-availability cluster with load balancing.

          Unless I’m missing something fundamental, I don’t see how this decentralization is of any use if the content isn’t there.

        • GuyDudeman@lemmy.mlEnglish
          0·
          2 years ago

          I still don’t quite understand how the community is replicated…

          Are you saying that if Lemmy.ml/tiki exists and someone creates Beehaw.org/tiki that they are the same community? They would show the same posts and comments?

          Or are they completely separate communities that would just have the same name… users could subscribe to both if they wanted, but the posts and comments would be stuck on their respective instances?

          Or - Is it the case that Lemmy.ml’s tiki community and posts and comments are also stored on Beehaw.org somehow?

          If I deleted the tiki community on Lemmy.ml, would users from both communities lose their posts and comments from the Lemmy.ml instance of that community?

          • Mac@lemmy.worldEnglish
            0·
            2 years ago

            The Tiki community should simply run a Tiki server, no? Problem solved.

        • federico3@lemmy.mlEnglish
          0·
          2 years ago

          you could design the system in a way that one instance is mirrored and load-balanced across different hosters

          That’s exactly what I meant. Horizontal replication shares a lot of building blocks with federation. NNTP had peering/replication and worked quite well for a protocol designed in 1986.

    • _NetNomad @ DXC@forum.dxcomplex.comEnglish
      1·
      2 years ago

      i’ve been saying we need a COBOL/CICS implementation of ActivityPub for YEARS and it’s always the same “where the hell am i supposed to get a 3270 in 2023” and “what do you mean i can’t shitpost during the batch window”

  • Ruud@lemmy.worldEnglish
    1·
    2 years ago

    Lemmy.world is a new server, accepting signups. You’re welcome there.

  • lightrush@lemmy.caEnglish
    1·
    2 years ago

    Point us to where the coin slot is. E.g. Patreon. We insert coin 🪙, you upgrade.

  • Bilb!@lemmy.mlEnglish
    1·
    2 years ago

    I’m going to set up a general purpose instance tomorrow with the intention of handling a relatively large number of users. The main problem is choosing a domain!

  • anji@lemmy.anji.nl
    1·
    2 years ago

    Sadly, I feel like the Fediverse, based on ActivityPub, was fundamentally designed wrong for scaling potential. I do like Fedi and I like ActivityPub, but I think instances should not have to be responsible for all of this:

    • Owning user accounts
    • Exclusively host communities
    • Serving local and remote users webpages and media
    • Never going down, as this results in users and content becoming unavailable

    Because servers “own” the user accounts and communities it’s not trivial for users to switch to a different instance, and as instances scale their costs go up slightly exponentially.

    I wish the Fediverse from the beginning was a truly distributed content replication platform, usenet-style or Matrix-style, and every instance would add additional capacity to the network instead of hosting specific communities or users.

    I guess it’s a bit too late for a redesign now… Perhaps decentralized identifiers will take us there in some form in the future.

  • comfy@lemmy.mlEnglish
    1·
    2 years ago

    @nutomic@lemmy.ml It might be a good idea to default the Communities page to All instead of Local, to help push users into discovering other instances and promote them.

    • Provoked Gamer@lemmy.caEnglish
      0·
      2 years ago

      I agree because this way, new users will learn what and how to use other instances. Plus, it also helps with finding more content, especially if the user picked an instance without many people which makes there be less communities and content they can check out on first glance.

      • Red Army Dog Cooper@lemmy.mlEnglish
        0·
        2 years ago

        I disagree because it makes the more narrowly focused topic or theme based instances more daluted, makes everything blur together more, I also see it as a detrament to the smaller intances because they will now there local comunity will have less traffic

            • gkd@lemmy.mlEnglish
              0·
              2 years ago

              I think a client that might select a server for you by default (hopefully a trusted one of course) would make things way more easy to understand for the average user. Then making it easier to add or view communities on other instances.

  • cecirdr@lemmy.mlEnglish
    0·
    2 years ago

    I’m a noob. I created an account on beehaw and on lemmy.ml. That’s because I see communities on one instance that I’m interested in and a different community on another instance. So if there’s a technology community on both, how do I get to see all the technology posts without having to have two accounts?

    This is really confusing for noobs like me. I’d just like to see one community to technology, one for Science, one for nintendo etc. I don’t care it it’s spread out amongst different servers to divvy up the load, but from the user side, it needs to be seamlessly integrated.

    I’m still learning how all this works though. But I don’t know how many folks that are more casual than me will be willing to figure it out. I hope they do though! It’ll be worth it to leave reddit in the rearview mirror!

    Edit: lawdy, I just figured it out. Local vs all on the communities list. It was right in front of my face. good grief!

  • kunday@lemmy.mlEnglish
    0·
    2 years ago

    New user,how do I donate / tip to help you peeps cover server costs? It wasn’t directly obvious how to do it; apologize if it’s a big button right on a page that I missed.

  • papasfritas@lemmy.mlEnglish
    0·
    2 years ago

    is there some kind of status page to have a look at and see how things are going? I cannot make any comments to a specific community at the moment and wondering why.

    EDIT: Figured it out, when I tried to leave a comment via Jerboa I got an error “Language not allowed” and so I selected a language on the desktop site and then my comment went through. Note that this error does not appear on desktop site so I had no idea what was going on and why my comment was not going through

  • d3Xt3r@lemmy.mlEnglish
    0·
    2 years ago

    Edit: If you moderate a large subreddit, do not link your users directly to lemmy.ml in your announcements

    How/which URL should we link to then? Now is the best time to get users to switch to Lemmy so we need to make it as newbie friendly as possible. Already the application process has put off some people (I do like that bit though, keeps away the low effort folks). Thanks.

  • sunaurus@lemm.eeEnglish
    0·
    2 years ago

    If anybody else is lost and wants a basic general-purpose home for their account, https://lemm.ee is on good hardware and open for signups.

  • Zagaroth@lemmy.mlEnglish
    0·
    2 years ago

    So, might I recommend having a button on the top bar that shows us the instances we’ve subscribed to, and maybe a quick link to the list of available instances? People like easy navigation, having to do multiple bookmarks or navigate through finding a link to the list of servers is not easy navigation.