Hi,

We had a DDOS attack against old.lemmy.today which overloaded the entire server for about 20 minutes. I had to turn off old.lemmy.today temporarily and will look through logs and probably block more chinese ip ranges, since thats where it came from.

Always something to do. :) Keeping this as sticky for a while to inform everyone.

Edit: I added more ip blocks for Chinese networks and hopefully it will keep future bots away. Bringing up the old.lemmy.today user interface again…

Edit 2: Had to take it down again due to massive bot scraping. Currently down.

  • freudian_slop@programming.dev
    16·
    5 days ago

    I don’t have an account there but I am always happy when fediverse instance admins are transparent about their problems. This will help other instances too.

  • MyOpinion@lemmy.todayEnglish
    13·
    5 days ago

    When I was running a web server this was a common occurrence. Most of the DDOS attacks I experienced came from China.

      • YiddishMcSquidish@lemmy.todayEnglish
        7·
        5 days ago

        This is a genuine question, but why doesn’t this instance? Is it because we would miss out on some content? I mean you are much more in touch with the technicalities, but do we have a mandarin speaking user base you’re not trying to alienate?

        • mrmanager@lemmy.todayOPM
          9·
          5 days ago

          I think we are unfortunantly heading in that direction of blocking more and more of them…

          And no, no mandarin user base. :)

      • mehquestion@lemmy.world
        31·
        4 days ago

        First, thank you for all the hard work you do.

        Second, as someone still struggling to understand lemmy, one of the reasons I’m not a fan of old.lemmy.world is that I have to turn on cloudflare.

        I have noscript running and I have cloudlflare blocked.

        Your response implies otherwise that you don’t have cloudflare running either.

        So why am I always greeted with cloudflare when I visit old.lemmy.world?

        Edit: Oh I misread the title. Its old.lemmy.today that you’re talkinga bout, not old.lemmy.world.

        My goof. Please ignore my post, but do take my gratitude.

        Edit 2: So I went over to old.lemmy.today. It is tempting to switch. I like the default interface and it has content from a lot of places (on quick glance). One of the reasons I’m not a huge fan of old.slrpnk.net is that I can’t figure out how to access content from other instances. I seem tempted to switch.

        • mschae@discuss.mschae23.deEnglish
          2·
          3 days ago

          One of the reasons I’m not a huge fan of old.slrpnk.net is that I can’t figure out how to access content from other instances.

          old.slrpnk.net should still have posts from other instances, as far as I can tell. They only show local stuff by default, so you have to set “listing” to “all” to see them (the small toggle under the yellow tagline). If you want to primarily interact with posts from other instances, lemmy.today is probably the better choice though either way. slrpnk.net looks like it mostly focuses on local communities.

          I have noscript running and I have cloudlflare blocked.

          The good thing about mlmym (the software behind old.lemmy.today, old.lemmy.world, and so on) is that it’s entirely usable without JavaScript :)

          So as long as there isn’t something like cloudflare in the way that does require JS, it should work fine to use it that way.

          Looking at it right now though, @mrmanager@lemmy.today it looks like old.lemmy.today is erroring with 502?

          • mrmanager@lemmy.todayOPM
            3·
            3 days ago

            Yeah, I had to take it down overnight and currently experimenting with anubis for it… will be a bit unstable today… :)

        • mrmanager@lemmy.todayOPM
          4·
          4 days ago

          You are very welcome if you do. Its the same content in old.lemmy.today as lemmy.today, just a different look. :)

          I dont like cloudflare so doing my best to avoid it if I can.

            • mrmanager@lemmy.todayOPM
              3·
              3 days ago

              Ok now its working and seemingly blocking bots, but we will see over time how well this works. Like you said, im sure its possible to bypass but maybe it improves the situation somewhat still.

            • mrmanager@lemmy.todayOPM
              3·
              3 days ago

              Its not really working yet, it keeps popping up the “seeing if you are a bot” message all the time.

              But yeah, working on it. If it can prevent dumb bots, its a win for sure.

          • mehquestion@lemmy.world
            2·
            3 days ago

            Hi I just wanted to follow up on something

            Since last evening, I can’t access old.lemmy.today (I can still access lemmy.today but that’s not an interface I like).

            Are you having bot issues again or is there something I messed up on my end?

            • mrmanager@lemmy.todayOPM
              3·
              3 days ago

              Now it should work well again. I put it behind anubis and we will see if it improves the situation with the bots.

              • mehquestion@lemmy.world
                1·
                2 days ago

                I’m so sorry to constantly pester you, but I have another question.

                I’ve been enjoying old.lemmy.today for the past few days. The biggest praise I can give it: I haven’t missed reddit at all, and the smaller community actually makes it feel like an upgrade. Like the reddit of old.

                But I wanted to make an account on Lemmy.today to make a post, however to register it seems I need an email address.

                I don’t want to give an email address. I could give a disposable one from one of those sites, but that would be obvious (to what it sounds like is a human curation process).

                I know you said its to ward off bots, I’m wondering if its a temporary pause, or if its a permanent policy?

                • mrmanager@lemmy.todayOPM
                  2·
                  2 days ago

                  It’s fine to not give an email address. It’s just for the email to let you know that you were approved or denied. So if you put something random there and check if you can login later, it’s all good.

                  I think it’s even optional to give it… You can try registering without. :)

            • mrmanager@lemmy.todayOPM
              3·
              3 days ago

              It was down overnight and today im working on trying to get anubus to work with it, so it will be unstable… :)

      • mrmanager@lemmy.todayOPM
        7·
        5 days ago

        Very good, I pulled the latest version.

        Thanks for your work on this, and the code you added to forward ip numbers was super useful today to see where these botnets are coming from. Really appreciate it.

      • CalcProgrammer1@lemmy.todayEnglish
        4·
        5 days ago

        Thanks to both of you for keeping mlmym alive. It’s the best way to use Lemmy and I moved to lemmy.today primarily because they have it as an available interface.

      • somebody_to_love@lemmy.today
        2·
        5 days ago

        Thanks to both of you!

        Now that you’re both here, I have a question/request. What I love most from the “old Reddit” interface is using RES to navigate through posts with the keyboard: “h” to hide, j/k as up/down, “l” to open in a new tab, etc. Makes browsing super fast and efficient.

        Is this something you’d consider? I’m not asking for a full RES reimplementation, of course. Just the keyboard navigation.

        • mschae@discuss.mschae23.deEnglish
          2·
          4 days ago

          I know of the lemmy keyboard navigation addon that reimplements this feature for lemmy-ui and mlmym. I haven’t tested it myself though, and it’s been last updated over a year ago, so I don’t know if it still works. I have thought about including the feature directly in mlmym (could actually make use of the keyboard navigation setting that lemmy already has to toggle it), but if I do, that’ll only happen after I’m done with the refactor for the v4 API. So it will probably take a while 😅

          edit: opened an issue for it (#27) so I don’t forget

  • db0@lemmy.dbzer0.com
    5·
    4 days ago

    Look into pow protection. The Chinese botnets come from a practically infinite range at the moment

  • ThunderComplex@lemmy.todayEnglish
    5·
    5 days ago

    Thanks for keeping the instance running. I happened to get a error message smth about the instance being broken and remembering that disk upgrade post from a couple days ago I feared the DB got nuked for a sec hehe.

    • mrmanager@lemmy.todayOPM
      5·
      4 days ago

      It’s manual at this point since it hasn’t been a huge problem so far, but we will see how it evolves. Perhaps it gets worse as the instance grows bigger.

  • Wren@lemmy.todayEnglish
    1·
    5 days ago

    Thanks for the heads up. I got a brief error message a while ago and figured that’s what happened. Good job keeping the instance going!

    • mrmanager@lemmy.todayOPM
      4·
      5 days ago

      Thank you! These ddos attacks are annoying but not so common after all. Last time was like 3 months ago or something.