In 2002, I grew annoyed with not finding the obscure technical information I was looking for, so I started Gmane, the mailing list archive. All technical discussion took place on mailing lists those days, and archiving those were, at best, spotty and with horrible web interfaces.
The past few weeks, the Gmane machines (and more importantly, the company I work for, who are graciously hosting the servers) have been the target of a number of distributed denial of service attacks. Our upstream have been good about helping us filter out the DDoS traffic, but it’s meant serious downtime where we’ve been completely off the Internet.
Of course, there are ways to try to mitigate all this: I’m moving the Gmane servers off of my employer’s net, and I’m putting Cloudflare in front of the Gmane web servers.
But I ask myself: Is this fun any more?
Running a mailing list archive means, of course, that people want stuff removed from the archive, so I’ve apparently been sued in India (along with Google and Yahoo) (and I’m never going there: I might be sentenced. I don’t know). And I’m the Internet Help Desk, which is nice, but confusing. And all the threats of “legal action” are, well, something.
And now the DDoS stuff, which I have no idea why is happening, but I can only assume that somebody is angry about something.
Probably me being a wise ass.
So… it’s been 14 years… I’m old now. I almost threw up earlier tonight because I’m so stressed about the situation. I should retire and read comic books and watch films. Oh, and the day job. Work, work, work. Oh, and Gnus.
I’m thinking about ending Gmane, at least as a web site. Perhaps continue running the SMTP-to-NNTP bridge? Perhaps not? I don’t want to make 20-30K mailing lists start having bouncing addresses, but I could just funnel all incoming mail to /dev/null, I guess…
The nice thing about a mailing list archive (with NNTP and HTTP interfaces) is that it enables software maintainers to say (whenever somebody suggests using Spiffy Collaboration Tool of the Month instead of yucky mailing lists) is “well, just read the stuff on Gmane, then”. I feel like I’m letting down a generation here. And despite what I rambled about in that paragraph up there, I’ve had many fun interactions with people because of Gmane. And lots and lots and lots of appreciative feedback over the years.
*sigh*
But there’s The Mail Archive. Those guys are doing a good job. If The Mail Archive had been as good in 2002 as it is now, I probably wouldn’t have started Gmane.
I’m open to ideas here. If somebody else wants to take over the concept, I can FedEx you a disk containing the archive (as an NNTP spool). I’ve written a lot of software for Gmane, but it’s all quite site specific and un-documented. And the web interface was written in, like, 2004, so it’s way way way un-Web 2.0-ey and shiny. You’re probably better off implementing this stuff from scratch.
Oh, and along with the spool you’ll get the gmane.conf file which has the mailing list->NNTP mapping.
I can’t really recommend the job, though. It sucks.
[Update: See this comment.]
Thank you for gmane. This is the only window to NNTP before to get started with my own NNTP client.
The best solution to me would be a local NNTP to maildir conversion tool. Just like a ~/News I could access from my mail reader. A small config file to tell which news group I want to download. Like nnmaildir as standalone. This would be a pretty acceptable replacement to me.
Thank you for all this work on Gmane, kept along with the other projects!
Sounds like this could take a collaborative effort. I’ll volunteer to throw in and help somehow, whether it’s working on the actual tech, or just donating money, or $whatever.
I would be very interested in taking over this, or at least getting a copy of the entire news spool. What email address may be chat on? kevin dot bowling at kev009 dot com for me.
I work at one of the largest CDNs and have a lot of experience with scalability, DDOS, etc.
I run a public Usenet service and would be interested in backfilling the gmane data and doing new development on a web UI like Google Groups, but not crappy.
At my first opportunity, I’d like to donate to help defray your costs (both hosting, and stress-reduction beverages). I don’t know how yet because GMane is DDoSed😦
If you get enough money would you be able to keep NNTP–Email gateway running? With volunteers maybe?
Thanks a lot for your work during all these years, it has been very important to me.
Would storing the archives on IPFS be an idea, and let anybody mirror or fetch them?
Have you considered setting up Cloudflare DDOS protection? They list advanced ddos protection as a feature of their 200USD/mo business plan but as GMANE is such an important service it may be worth chatting with their sales team to see what’s available for what budget.
Hey, I just googled for donation page and it’s down: http://gmane.org/donate.php I also noticed it was cached by CloudFlare, were you using CloudFlare during the attack? Big thank you, I used Gmane here and there randomly for passive reading.
Thank you for all your work and effort.
I was never a heavy gmane or mailing list user. But the web interface will be missed, because it made all those mailing list searchable and accessible via google. I can not remember the countless times that a long winded google search ended in a helpful gmane link.
For thins: thank you!
I can’t volunteer to help you but if you set up a donation system or a crowdfunding campaign I will gladly give some money.
Thank you for GMANE! I have a GMANE t-shirt I wear quite regularly, though I can’t remember how or why I got it. Back in the day, I ran Thunderbird and added all my favorite mailing lists (mostly Plone & Python related). I’ve stopped doing that lately, due to laziness & getting olderness. But I still have a great deal of respect and admiration for GMANE and what you’ve accomplished with it. I’m encouraged to see others are interested in helping and I think it should continue if there is still a demand for it. Not sure how to handle the financials, but it might be nice to see a community of administrative volunteers grow around this awesome service. In any event, your frustration and exhaustion are totally understandable; I wish you the best whatever you decide! Good luck & thanks again.
I will be sad to see GMANE go, I’m a long standing NNTP user and GMANE has been invaluable to me all of these years.
Perhaps making the base news spool available via bittorrent? I know I would like a personal, searchable archive, of gmane, and it would be good to have the data widely distributed.
And, since I’ve never had the chance to say this before: Thank you very much for gmane, and gnus. I’ve been using your tools for decades now. You’ve always been a hero of mine.
I hope if you put these tasks down due to adversity and exhaustion, that someone else(s) will take the time to take them up for you, and you can move on to less stressful things. I had no idea it was you, and you alone, holding up this portion of the sky.
http://esr.ibiblio.org/?p=4196
> And the web interface was written in, like, 2004, so it’s way way way un-Web 2.0-ey and shiny.
I always thought the gmane web interface is a god-send. Simple, clutter-free and rather fast. If possible I would definitely like to have it available somewhere (I would be willing to donate money/time for it too!). Please let me know if you want to make it accessible somewhere!
And, of course, thanks for spending so much time and effort on gmane over all these years!
Thanks for all your dedication put into Gmane!
The mail to NNTP gateway is **really** useful – I really wouldn’t want to subscribe to all mailing lists I’m reading through NNTP… The web archive isn’t important to me.
Have you evaluated something like that?
https://github.com/uriel-fanelli/tribes
U.
gmane is a valuable resource, worth preserving.
Could you make a new post sketching out the scale needed to host an http-version of the site? Average daily bandwidth? Average number of connections per second? How many takedown notices per week?
I’ve gotten a lot of use out of Gnus and gwene.org over the years. Thank you for keeping it up for so long, people really do appreciate your work.
I am using Gmane many years now every day. Thank you very much for your work. I hope somebody will takeover, otherwise I have a problem.
Thank you for the great service. I am a total fan of gmane both the web interface and the newsgroups. Would be happy to help keeping it alive. I run a small training business and I can’t offer much time (guess I could help subscribing people) but I would be willing to pay something if that helps.
PS: also my website is under dos attacks which is no fun
Lars you are pathetic. We have mailed you several times to remove messages from your archive. You did nothing. Zero. Nada. Not one response from you. We will continue our fight against you in court of law. Poor!
Comments like these show why it’s important to have public archives of public lists. You can’t just go around and bully people to remove stuff that has been said in public.
John B. *you* are pathetic…
I’m feeling lost, how am I going to read my daily mailing lists now😦
Thank your for your service all these years.
Would love to get involved to keep gmane running. Stuff like this should be preserved! Can we get a project site set-up and crowd fund it?
As gmane.org serves as a kind of DMI (Digital Mailings Identifier System) please don’t shut of the server.
Links to gmane i.e. http://article.gmane.org/gmane.comp.gnu.mingw.w64.general/7700 can’t be referenced anymore. At least it should be possible get back datetime and topic of references to gmane.
For me as well: many thanks for the all the hard work you spend for this useful project.
Thank you so much for your work. I used (and use) the nntp gateway to a bunch of mailing lists since gmanes beginnings. I never was aware that this is a one man show, my respect! I wish you all best and hope there is a future for gmane.
Thank you so much for all your hard work on gmane over the years.
Regardless of whatever you decide on: THANK YOU!
It’s been an amazing privilege to enjoy the magic that is the mailing list NNTP gateway.
Given my use case, it’d be amazing if you could continue running the NNTP gateway. (Or, perhaps, open-source it. If the code is in such a state that I could run my own, I’d also be really happy. Given the messiness of SMTP/NNTP, I’m assuming it’s pretty… let’s say “hackish”
in certain places.)
I also have found the gmane archive a very valuable tool for the Electric Vehicle cause, and I sincerely hope there are people that can help to bring the evdl.org archive [
http://news.gmane.org/gmane.recreation.cars.evdl
] back online.
Bruce {EVangel} Parmenter
brucedp.150m.com
(Near Silicon Valley, south of SF, CA USA)
Lars,
I join the chorus, and offer my sincerest gratitude for providing this service over the past many years. Glad to hear the smtpnntp relay will remain up.
To whomever ultimately takes on the responsibility of the web services, I humbly request that you do not make it shiny and modern, and leave it just as it is.
I hope you do find a way to keep both the NNTP and web interfaces online for a long time. I expect you (or some other person or organization) could find some reasonable crowd funding if necessary.
However it pans out, though, thank you for all that you’ve put into it. Even considering the crazy amount of work you put into things like Gnus, this had to be a huge effort (and — 14 years?!), and apparently a massive headache at times.😦 But such an incredibly useful tool! It’s hard to imagine setting up an email environment where I’m not pulling a couple dozen lists in via gmane…
I keep googling something, clicking a gmane link, then shouting FUCK! NOT AGAIN!
Please bring back gmane before those links I keep following on Google disappear as well. It’s INSANE to take the site offline when it is as important, if not MORE IMPORTANT, then Wikipedia to the open source developer community.