Hi Beeple!

Here’s a vague version of events :

  • 11PM EST: Lemmy.world got hacked

  • 12:20AM EST: Blahaj.zone got hacked

  • 12:25AM EST: I shut down the server

  • 12:30AM EST: I make announcements to tell people about this

  • 12:45AM EST: I have an idea of what the problem is but there is no fix

  • 2:20AM EST: I go to sleep

  • 8:50AM EST: The server is booted back up, steps are applied to mitigate issues (Rotating JWTs, Clearing DB of the source of vulnerability, deleting custom emoji), UI is updated with the fix, CSP and other security options are applied

  • 11:40AM EST: We start testing things to make sure are working And well, now here we are.

If you have issues logging in or using an app:

  1. Log out if you somehow are still logged in

  2. Clear all cache, site data, etc.

  3. Hard refresh Beehaw using CTRL+F5

  4. Log back in.

If you still have issues, write to us at support@beehaw.org

To be clear : We have not been hacked as far as we know, we were completely unaffected. This was done preemptively.

Oh yeah, in case, you haven’t, this is a good opportunity and reminder to follow us on Mastodon as the communication line was still up despite Beehaw being down : https://hachyderm.io/@beehaw

  • jarfil@beehaw.org
    link
    fedilink
    English
    arrow-up
    5
    ·
    edit-2
    1 year ago

    We had no messages on our database that had the vulnerability

    This is interesting. I actually commented about the use of emojis/emotes a couple days ago on a post on !foss@beehaw.org made by a federated user from lemmy.one, that has since been removed (😕), but I still have the bookmarked comment in which I copied the raw embed for the remote emote image in the federated comment I was responding to.

    Do I understand it correctly, that the latest fixes to stop the code injection, will still allow remote image embedding, so something like an “emote picker extension to embed animated GIFs from a remote server and/or remote instance’s emoji list” would still be doable and wouldn’t pose any risk?

    Or would such picker still have to include measures to prevent offering embeds with potential exploits?

    • Lionir [he/him]@beehaw.orgOPM
      link
      fedilink
      English
      arrow-up
      9
      ·
      1 year ago

      Remote image embedding is not the issue, remote custom emojis would not have been an issue either. The issue, from my understanding, is that the way local emojis are rendered allowed for an XSS exploit.

      You can look at the PR which fixed this issue if you have a better understanding of these things than me : https://github.com/LemmyNet/lemmy-ui/pull/1897/

      I believe such a picker would be fine.

      • jarfil@beehaw.org
        link
        fedilink
        English
        arrow-up
        5
        ·
        edit-2
        1 year ago

        I see, so the prior emoji handling rendered content directly from the comment, instead of making sure it was strictly what was defined for the local emoji; that was a weird choice. Now they’ve also added a sanitizer wrapper to all of it in: https://github.com/LemmyNet/lemmy-ui/pull/1906

        I guess the only downside of a picker that used the non-emoji image renderer, would be the loss of emoji CSS formatting.

        • tanglisha [she/her]@beehaw.org
          link
          fedilink
          English
          arrow-up
          5
          ·
          1 year ago

          From what I can tell the whole point to the css class/formatting was controlling the size of the emojis. Depending on where they came from, I could see some being of random size and shape. Admins might not have the time or know-how to shrink them down, so css seems like a reasonable compromise as long as the files aren’t huge.

          I’m kind of bothered that the only fix seems to be on the frontend. Unfortunately, I haven’t been able to stick with Rust long enough to take a reasonable crack at figuring out how to help on the backend. Input and output sanitization should ideally be handled in both places.

          • jarfil@beehaw.org
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            Lemmy’s backend is kind of curious, in that it does the bare minimum to move content around and add some metadata fields.

            For example, did you know that “deleting” a comment, only marks a “deleted: true” field, while the comment is still pushed in full to the frontend? Same thing happens with banned/mod removed comments, they just get marked as “removed: true” but otherwise still get pushed to the client, and the user can still edit them.

            All the display processing is done in the frontend, or whichever app you happen to use.