The End of Gmane?

In 2002, I grew annoyed with not finding the obscure technical information I was looking for, so I started Gmane, the mailing list archive. All technical discussion took place on mailing lists those days, and archiving those were, at best, spotty and with horrible web interfaces.

The past few weeks, the Gmane machines (and more importantly, the company I work for, who are graciously hosting the servers) have been the target of a number of distributed denial of service attacks. Our upstream have been good about helping us filter out the DDoS traffic, but it’s meant serious downtime where we’ve been completely off the Internet.

Of course, there are ways to try to mitigate all this: I’m moving the Gmane servers off of my employer’s net, and I’m putting Cloudflare in front of the Gmane web servers.

But I ask myself: Is this fun any more?

Running a mailing list archive means, of course, that people want stuff removed from the archive, so I’ve apparently been sued in India (along with Google and Yahoo) (and I’m never going there: I might be sentenced. I don’t know). And I’m the Internet Help Desk, which is nice, but confusing. And all the threats of “legal action” are, well, something.

And now the DDoS stuff, which I have no idea why is happening, but I can only assume that somebody is angry about something.

Probably me being a wise ass.

So… it’s been 14 years… I’m old now. I almost threw up earlier tonight because I’m so stressed about the situation. I should retire and read comic books and watch films. Oh, and the day job. Work, work, work. Oh, and Gnus.

I’m thinking about ending Gmane, at least as a web site. Perhaps continue running the SMTP-to-NNTP bridge? Perhaps not? I don’t want to make 20-30K mailing lists start having bouncing addresses, but I could just funnel all incoming mail to /dev/null, I guess…

The nice thing about a mailing list archive (with NNTP and HTTP interfaces) is that it enables software maintainers to say (whenever somebody suggests using Spiffy Collaboration Tool of the Month instead of yucky mailing lists) is “well, just read the stuff on Gmane, then”. I feel like I’m letting down a generation here. And despite what I rambled about in that paragraph up there, I’ve had many fun interactions with people because of Gmane. And lots and lots and lots of appreciative feedback over the years.

*sigh*

But there’s The Mail Archive. Those guys are doing a good job. If The Mail Archive had been as good in 2002 as it is now, I probably wouldn’t have started Gmane.

I’m open to ideas here. If somebody else wants to take over the concept, I can FedEx you a disk containing the archive (as an NNTP spool). I’ve written a lot of software for Gmane, but it’s all quite site specific and un-documented. And the web interface was written in, like, 2004, so it’s way way way un-Web 2.0-ey and shiny. You’re probably better off implementing this stuff from scratch.

Oh, and along with the spool you’ll get the gmane.conf file which has the mailing list->NNTP mapping.

I can’t really recommend the job, though. It sucks.

[Update: See this comment.]

127 thoughts on “The End of Gmane?”

  1. Thank you for gmane. This is the only window to NNTP before to get started with my own NNTP client.

    The best solution to me would be a local NNTP to maildir conversion tool. Just like a ~/News I could access from my mail reader. A small config file to tell which news group I want to download. Like nnmaildir as standalone. This would be a pretty acceptable replacement to me.

    Thank you for all this work on Gmane, kept along with the other projects!

  2. Sounds like this could take a collaborative effort. I’ll volunteer to throw in and help somehow, whether it’s working on the actual tech, or just donating money, or $whatever.

  3. I would be very interested in taking over this, or at least getting a copy of the entire news spool. What email address may be chat on? kevin dot bowling at kev009 dot com for me.

    I work at one of the largest CDNs and have a lot of experience with scalability, DDOS, etc.
    I run a public Usenet service and would be interested in backfilling the gmane data and doing new development on a web UI like Google Groups, but not crappy.

  4. At my first opportunity, I’d like to donate to help defray your costs (both hosting, and stress-reduction beverages). I don’t know how yet because GMane is DDoSed 😦

    If you get enough money would you be able to keep NNTP–Email gateway running? With volunteers maybe?

  5. Have you considered setting up Cloudflare DDOS protection? They list advanced ddos protection as a feature of their 200USD/mo business plan but as GMANE is such an important service it may be worth chatting with their sales team to see what’s available for what budget.

  6. Hey, I just googled for donation page and it’s down: http://gmane.org/donate.php I also noticed it was cached by CloudFlare, were you using CloudFlare during the attack? Big thank you, I used Gmane here and there randomly for passive reading.

  7. Thank you for all your work and effort.

    I was never a heavy gmane or mailing list user. But the web interface will be missed, because it made all those mailing list searchable and accessible via google. I can not remember the countless times that a long winded google search ended in a helpful gmane link.

    For thins: thank you!

  8. I can’t volunteer to help you but if you set up a donation system or a crowdfunding campaign I will gladly give some money.

  9. Thank you for GMANE! I have a GMANE t-shirt I wear quite regularly, though I can’t remember how or why I got it. Back in the day, I ran Thunderbird and added all my favorite mailing lists (mostly Plone & Python related). I’ve stopped doing that lately, due to laziness & getting olderness. But I still have a great deal of respect and admiration for GMANE and what you’ve accomplished with it. I’m encouraged to see others are interested in helping and I think it should continue if there is still a demand for it. Not sure how to handle the financials, but it might be nice to see a community of administrative volunteers grow around this awesome service. In any event, your frustration and exhaustion are totally understandable; I wish you the best whatever you decide! Good luck & thanks again.

  10. Perhaps making the base news spool available via bittorrent? I know I would like a personal, searchable archive, of gmane, and it would be good to have the data widely distributed.

    And, since I’ve never had the chance to say this before: Thank you very much for gmane, and gnus. I’ve been using your tools for decades now. You’ve always been a hero of mine.

    I hope if you put these tasks down due to adversity and exhaustion, that someone else(s) will take the time to take them up for you, and you can move on to less stressful things. I had no idea it was you, and you alone, holding up this portion of the sky.

    http://esr.ibiblio.org/?p=4196

  11. > And the web interface was written in, like, 2004, so it’s way way way un-Web 2.0-ey and shiny.

    I always thought the gmane web interface is a god-send. Simple, clutter-free and rather fast. If possible I would definitely like to have it available somewhere (I would be willing to donate money/time for it too!). Please let me know if you want to make it accessible somewhere!

    And, of course, thanks for spending so much time and effort on gmane over all these years!

  12. Thanks for all your dedication put into Gmane!

    The mail to NNTP gateway is **really** useful – I really wouldn’t want to subscribe to all mailing lists I’m reading through NNTP… The web archive isn’t important to me.

  13. gmane is a valuable resource, worth preserving.

    Could you make a new post sketching out the scale needed to host an http-version of the site? Average daily bandwidth? Average number of connections per second? How many takedown notices per week?

  14. I’ve gotten a lot of use out of Gnus and gwene.org over the years. Thank you for keeping it up for so long, people really do appreciate your work.

    1. Thank you for the great service. I am a total fan of gmane both the web interface and the newsgroups. Would be happy to help keeping it alive. I run a small training business and I can’t offer much time (guess I could help subscribing people) but I would be willing to pay something if that helps.

      PS: also my website is under dos attacks which is no fun

  15. Lars you are pathetic. We have mailed you several times to remove messages from your archive. You did nothing. Zero. Nada. Not one response from you. We will continue our fight against you in court of law. Poor!

    1. Comments like these show why it’s important to have public archives of public lists. You can’t just go around and bully people to remove stuff that has been said in public.

  16. I’m feeling lost, how am I going to read my daily mailing lists now 😦

    Thank your for your service all these years.

  17. Would love to get involved to keep gmane running. Stuff like this should be preserved! Can we get a project site set-up and crowd fund it?

  18. Thank you so much for your work. I used (and use) the nntp gateway to a bunch of mailing lists since gmanes beginnings. I never was aware that this is a one man show, my respect! I wish you all best and hope there is a future for gmane.

  19. Regardless of whatever you decide on: THANK YOU!

    It’s been an amazing privilege to enjoy the magic that is the mailing list NNTP gateway.

    Given my use case, it’d be amazing if you could continue running the NNTP gateway. (Or, perhaps, open-source it. If the code is in such a state that I could run my own, I’d also be really happy. Given the messiness of SMTP/NNTP, I’m assuming it’s pretty… let’s say “hackish” 🙂 in certain places.)

  20. Lars,

    I join the chorus, and offer my sincerest gratitude for providing this service over the past many years. Glad to hear the smtpnntp relay will remain up.

    To whomever ultimately takes on the responsibility of the web services, I humbly request that you do not make it shiny and modern, and leave it just as it is.

  21. I hope you do find a way to keep both the NNTP and web interfaces online for a long time. I expect you (or some other person or organization) could find some reasonable crowd funding if necessary.

    However it pans out, though, thank you for all that you’ve put into it. Even considering the crazy amount of work you put into things like Gnus, this had to be a huge effort (and — 14 years?!), and apparently a massive headache at times. 😦 But such an incredibly useful tool! It’s hard to imagine setting up an email environment where I’m not pulling a couple dozen lists in via gmane…

  22. I keep googling something, clicking a gmane link, then shouting FUCK! NOT AGAIN!

    Please bring back gmane before those links I keep following on Google disappear as well. It’s INSANE to take the site offline when it is as important, if not MORE IMPORTANT, then Wikipedia to the open source developer community.

    1. Thank you for taking it this far. Please, please do something to bring it back. You can make a kickstarter to pay for help and services to make this happen if you don’t have any other options. GMane is the center of the universe. We will gladly toss a few bucks your way to bring it back.

  23. Are there different plans to restore the Gwene web page? The feeds are working (NNTP) and they get updated, but I can’t find a way to add a new feed.

      1. I saw your announcement in gmane.discuss – thanks for bringing the Gwene web page back, and thank you very much for making Gwene possible!

        Maybe I broke something… I tried to add some feeds after the site was back up but before you posted the announcement. Now they are not available over NNTP (“411 No such group” for gwene.pt.maisfutebol and gwene.pt.publico.inimigo), but they can’t be added either (the web page says they have already been subscribed).

  24. Lars, first, thanks for all you’ve done. I’m shocked to hear you’ve had legal problems, although maybe I shouldn’t be. It seems to me that it’s very important that these messages not be lost to history (and that includes the ones subject to legal action!). Have you considered breaking the archive into (somewhat) smaller pieces and creating torrent files? The best way to preserve it might be to have a lot of copies running around.

  25. I’ll just chime in here too. Gmane has been by goto place to find information on mailing lists for a long time. I’ve really grown up with it. Mainly been a website user thou, so very sad to see that go. Big thanx for running it this far. I really hope it get’s picked up and continued .

  26. Thank you! I have been using gmane for many years and had no idea it was run by one person. You have done a great service to *so* many people. I am really sorry it has taken such a toll on you and cannot fault you at all for leaving it.

    BTW, I really like the frames thread-oriented UI and hate the other new one. But I fondly remember reading USENET with trn, so I may be an outlier.

    Anyway, thank you very, very much.

  27. As of today, when I click on a google search link that points to gmane.org, it comes back with “Server not found.” Is this just a temporary issue, or is gmane.org no longer serving pages?

      1. Our club (clubs.dba) uses gmane for its forums. It has confirmed that its postings are still being sent, but newsreaders now say “no new headers available” so our messages are not being distributed. However, the forum of another club I subscribe to, which also uses gmane, is still working.

        Our IT person says he can’t fix this because he can’t contact gmane any more to resolve the fault.

        What can we do about this, please?

        Adrian

  28. I have used gmane.org for many years as a valuable resource to browse and search mailing lists. Please rethink you decision not to bring back the web interface, it will be sorely missed by the community.

  29. Thanks so much for the amazing amount of work and reliability you’ve provided to the community over the years, I was never aware this was all the work of one person.

    You’ve made numerous searches for obscure configuration changes immensely easier. Whatever happens to gmane now, thank you!

  30. How sad to consider the prospect of life without nntp://new.gmane.org. Monetize it! Or let someone else do it. Micropayments? Subscriptions? Donation? I’m in! (please, no adware though). Maybe feedly.com could find the right way to monetize it and keep it alive…. just a thought.

  31. If you make clear how to contribute and where to do a little money transfer I would be happy to do that.

    You are definitely my heroes and I would love if I could ease you task in some way.

  32. Gmane has enabled my career for a decade. I use it for surge periods, interspersed with long periods away. It is vitally important for mailing lists such as cygwin’s. However, your health should come first. Thank you for what you have done for…the world.

    1. Lars, as everybody has said: thanks for Gmane! Your health (and your free time, if any) is more important than anything else. You already did *tons* of good work. I wish you will forget the bitterness!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s