As previously discussed, the gmane.org domain was no longer viable, and the NNTP server has now moved to news.gmane.io.
Likewise, mailing list subscriptions have been moved from m.gmane.org to m.gmane-mx.org.
As of this writing, neither service is up, because I’m doing the final resync before restarting the services on a new server. I expect the services to be back up again about 21:00 GMT (January 15th 2020), so don’t panic before that time.
DNS changes may also take some time to propagate.
[Edit at 15:30 GMT: I had misremembered how long the rsync took, so we’re now live six hours ahead of schedule. This Shouldn’t Possibly Happen. I mean, a computer project not being late. Anyway, both Gmane and Gwene feeds are now processing, but the news-to-mail bits aren’t up yet.]
OK, service announcement done, so I thought I’d write a bit about what happened the last week:
First of all, thank you for all the nice comments and PVT EMAILs of support. I wasn’t quite sure whether to continue running the NNTP server, but getting some feedback helps.
So then I started moving 15K mailing lists from a subscription on gmane.org to the new domain, and that was… er… interesting? The process works like this:
The Gmane configuration is a file that has one of these entries per list:
gmane.test gmane-test@quimby.gnus.org Testing the Gmane hierarchy mailman gt-gmane-test validated=2020-01-13 transfer=done crosspost-posting=no
I’ve got an Emacs mode to do a the maintenance work (subscribing, unsubscribing and the like), so I utilised that to write a function that would send out two unsubscription messages (because lists may be subscribed as @gmane.org or @m.gmane.org for historical reasons and there’s no record of which one) and one subscription message for the new @m.gmane-mx address. This bit is fully automated, so I could just sit there watching Emacs send out messages at a somewhat speedy clip. (Well, sending out all the messages took 12 hours in total due to how it’s done: It’s actually doing RPC via NNTP so that the messages are sent directly from the MTA instead of from me, because that looks less spammy.)
(While looking at Emacs doing this bit, I watched Witcher, which was surprisingly entertaining… in parts, and really, really tedious for the rest of the time. And, since it was Netflix, it looked cheap and shoddy.)
I did this in 1K batches, because when I’ve triggered this bit, all messages for the group in question go to a special gmane.admin group, so that I can see all the error messages and stuff, but most importantly: The “reply to confirm subscription” messages, which I then have to respond to (from Gnus). That’s semi-scriptable, but when the “please reply” message is in Chinese, I have to kinda guess.
Then after that, the “Welcome to foo” messages start pouring in, and again, handling the ones in English is fine, but then there’s all the other languages. I know Willkommen, and I can guess at bem-vinda and bienvenu, but Japanese is not my forte, so more guessing is involved.
So this took two days, and for the second day I watched both seasons of Fleabag (the first one is really fun and original and weird, and in the second one they removed everything that was interesting about it and made it into a normal boring dramedy, which explains why it’s on all the “best of” lists of 2019, and even won the Golden Globe. Well played!).
Fun bits from this process: If you’re sending out mail to email addresses that may not longer exist, you’ll end up being branded a spammer, but that wasn’t really much of a surprise.
What did surprise me was that Sourceforge has made it impossible to sign up to mailing lists via email, so we either had to abandon the 2K Sourceforge lists (I know! So many! I had no idea) or do something… semimanually. So I wrote a little bit to open Firefox on each list URL, which put the unique gmane-mx.org into the X selection, so doing each list was “Super-s Right-Mouse TAB TAB SPACE TAB SPACE RET Super-TAB”, and I could do one in five seconds without moving my hands from the keyboard (it’s got build in mouse buttons).
But then it turns out that if you do that a lot, Sourceforge will sic the old-fashioned “click on the three palms” captcha on you. Which took the throughput way, way down.
So for the first half of the Sourceforge lists, I utilised the power of crowdsourcing, and sent off 100 lists each to people who volunteered to do this mindless and boring bit. (Some came back for seconds!) Thank you all again for volunteering.
For the second half, I discovered the wonderful world of captcha solvers, and after signing up with a free wit.ai API key, it worked pretty reliably. So I did most of the second day myself, since it just added a new “TAB RET” bit to each list. (I watched the Alan Bennett at the BBC box set for this part. It’s quite extraordinary. I particularly loved the one with Mrs Bucket in the hospital… and the one with the photographer in the churchyard… and the one with the guy who retires, is unhappy about the retirement, and then gets a stroke and dies.)
Uhm… Any other observations? Ah, right; the IETF MTAs rate-limited me to about one email per ten minutes, so it took four days for all the un/resubscription messages to get through, and some probably just timed out, so I should do another sweep of those bits.
OK… that’s it? Now I just have to wait for the rsync and the DNS changes, which is why I’m going on for this long.
Let’s hear it for TV and wine; two invaluable companions when doing boring semi-manual labour.
Thanks, again, from another daily nntp user. Let me just reiterate what lots of others have said: you’ve done a great service to a large community, you don’t owe us anything, you didn’t fuck up, and we owe you more thanks than we’ve given you so far (and money, if you find a way for us to help on that score).
Hi
I am using GNU emacs 27 (from master).
I just changed in my setting
(setq gnus-secondary-select-methods
‘(
; (nntp “news.mozilla.org”) ;
(nntp “news.gmane.io”) ;
but it does not work, could you provide somewhere the correct setting please.
thanks
Uwe Brauer
That looks correct, but as the post says: The server is down, and will remain that way until sometime this night.
oops, sorry, reading sometimes is difficult.
thanks for your efforts.
Dear Lars,
many, many, many thanks for keeping Gmane alive! I’ll happily wait until the “news.gmane.org to news.gmane.io” transition is done! I have one question, though: news.gwene.org is offline as well now, so do I have to replace it with news.gwene.io in analogy to gmane?
No, news.gwene.org will still be a valid domain name, and it will continue to point to the same news server as the Gmane one.
Thanks for the clarification, Lars!
I was going to suggest Selenium to automate the part up to the captcha, but figured it wouldn’t help much because the captcha requires interaction anyway. Together with automated captcha solvers, this task might have been completely automatable! Possibly even in time comparable to how long it took you to do all of it manually.
Thanks Lars, ALWAYS^^^
fan Byung-Hee
I’ve news.gmane.io working in Gnus now and I see the latest messages from hamlib-developer are present in the NNTP feed. Thanks so much, Lars!
This community is far from dead. 🙂
Thanks!
Working fine.
Cheers,
Steffen
thank you so much for this service, i’ve used it for years and it helps me each day.
Thanks for all your work! It is very much appreciated.
I don’t see any mails coming in for the Postgres mailings lists though.
Yeah, I don’t know what’s up with the Postgres mailing lists. Have they changed to some list manager that doesn’t allow signing up via email? It seems like no known method for interacting with those lists work now.
It seems that they have a web-based signup form where you have to create a “postgresql.org community account”. Probably one account for each email address, so looks like more captcha solving?
https://www.postgresql.org/list/
Sounds like if somebody wants those lists back again, they’ll have to do the work…
Nothing has changed regarding that as far as I know
It used to be possible to sign up to mailing lists without using the Postgres web interface. If you’re willing to do the sign-ups, I can send you a list of the mailing lists/addresses to be singed up. Mail me at larsi@gnus.org.
PostgreSQL did replace the old majordomo by a custom PGLister software. You can find the announcement here: https://wiki.postgresql.org/wiki/PGLister_Announce
I’ve contacted the webmaster (CC to you) to evaluate possible solutions.
Long time news.gmane.org user here (mostly for Boost). It seemed to be limping along ok until yesterday, but now after stumbling across this blog (in search of why) I’m getting the same result from both news.gmane.org and news.gmane.io — “connection refused”. (And they also both resolve to the same IP: 195.159.176.226.) Is it down again? Do I just need to wait for a DNS update? Or am I just completely failing at parsing UTC times?
BTW — thanks muchly for the NNTP gateway — I’ve always preferred that over mailing lists.
Oh, fun. It seems like NetworkSolutions finally fixed the problem where they wouldn’t let me change the DNS servers for gmane.io. Without telling me. And I had forgotten to update the DNS settings over at Cloudflare. Should be fixed now, although the DNS changes may take some time to propagate.
Unfortunately, TLS (STARTTLS on port 119) is no longer working, I get an “SSL negotiation failed” error from KNode. Only plaintext works.
It works for me (in Gnus).
Thanks for your quick answer. Indeed, I see that “openssl s_client -connect news.gmane.io:119 -starttls nntp” looks OK. I guess KNode/kdelibs4/Qt4 might be choking on TLS 1.3. I am going to continue debugging this and hopefully fix the Fedora KNode package.
So the issue is that news.gmane.io supports only TLS 1.2 and later, whereas KNode uses KIO 4 which hardcodes TLS 1.0.
I am going to try fixing our packaged KIO 4.
KNode users need this:
https://src.fedoraproject.org/rpms/kdelibs/blob/master/f/kdelibs-4.14.38-kio-tls1x.patch
kdelibs4 backport of this KF5 fix:
https://commits.kde.org/kio/8196a735bebc6fd5eaf9d293bd565c00ef98516b
In Fedora, this is fixed in kdelibs-4.14.38-17.fc* that I am building now. With that build, I can connect to news.gmane.io with STARTTLS.
Users of KNode on Fedora will want these updates:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-2bbec59830
https://bodhi.fedoraproject.org/updates/FEDORA-2020-eb5b2d1a60
I’m a long time gmane user, and would sorely miss it if it went away. Is there any way to donate a few dollars to help the cause?
Thanks, but that’s not necessary. 🙂
Hej Lars
I just want to thank you for securing this great service for us all!
Sorry, but I had no success to get access to news.gmane.io via Thunderbird on Windows 10 (all latest versions). I only get timeouts having no ideas what the reason might be.
Perhaps you’re trying to connect to the NNTPS (TLS) port? Try the NNTP port instead.
Thanks. That was the fault.
Many, many, many thanks for all your work!
Is there are still groups which is not resubscribed?
for example:
gmane.os.openbsd.tech
gmane.os.openbsd.misc
is working, but
gmane.os.openbsd.cvs
gmane.os.openbsd.ports.cvs
is not updated
I’ve now resubscribed those OpenBSD lists.
Hello Lars, Is NTTPS (NNTP+TLS) supported by news.gmane.io?
I tried ports 563 and 119, but both seem to time out when TLS is enabled.
(Port 119 with no encryption works for me.)
The server supports STARTTLS on port 119.
It seems Thunderbird supports STARTTLS (Opportunistic TLS) for IMAP/POP3/SMTP, but not for NNTP. I’ll just use the plaintext service.
Hello Lars, and congratulation for this gigantic tv-fueled effort.
I am on the admin side of the @lists.lyx.org lists and I see that some lists have xxx@gmane-mx.org as subscriber, while some others still use @m.gmane.org.
How can I fix it? For example, I see
gela-lyx-announce-2@m.gmane.org
as subscriber to the lyx-announce list. Shall I subscribe
gela-lyx-announce-2@gmane-mx.org
instead?
Thanks
All the lists should be at @m.gmane-mx.org now. Just changing the gmane.org bit to gmane-mx.org (and keeping the same pre-@ bits) should do the trick.
Thanks for the quick answer. Hopefully I got it right. I am not sure how to check that though (especially since I do not use gmane…).
I got two “Welcome to…” messages from lyx.org, so I think it worked.
Wow! After painful days of missing news.gmane.org updates and useless googling for “gmane dead again” that gives only those old hits from 2016, I’ve finally found this post! (it’s 2’nd hit when googling for “gmane 2020” 😉 )
Then it only took me to s/news.gmane.org/news.gmane.io/ in .gnus.el and .newsrc.eld to resurrect all the subscriptions! How nice!
THANKS A LOT, Lars, from yet another happy user!
Thanks again for your work on keeping this fantastic service running.
Are there any plans to re-enable to subscription form so that new mailing lists can be added?
It appears that gmane.science.linguistics.linguist-list has no new postings since the switch-over on January 15. Has it been resubscribed and if not, can it be? Thanks.
I tried signing it up a couple of times, but nothing happened.
Oh, well. Thanks for trying.
I don’t use the NNTP interface daily any more, otherwise I had come here earlier. I actually never knew who ran Gmane (that I’ve used on and off for 15 years or so) until I searched for a bit longer today.
Very nice to be able to read all this background, but sad at the same time that it causes you, Lars, so much grief. I’m anyway very thankful for the service you have provided to many of us for so long, and I hope that you get at least some satisfaction out of the knowledge that I’m not the only one for whom switching to gmane.io was not a major issue.
Gmane is important to me and I’m thankful for your work and for your dedication that kept it running all this time! Take care.
Thank you so much <3
Dear Lars,
There is “gmane.linux.network.connman” that seems to be stuck since 2016. I believe corresponding mailing list is currently . Could you please try to resurrected it?
I’ve put mailing list name for “gmane.linux.network.connman” into less-greater signs, and it vanished from the WEB form. Here it is again: “connman@lists.01.org”
I’ve now resubscribed the list.
Hi, thanks for this incredible service.
Can you check these 2 Lists:
gmane.comp.security.virus.clamav.announce
gmane.comp.security.virus.clamav.user
Thanks a lot, Lars! gmane.linux.network.connman now works for me!
Thanks for all the work over the years on this, Lars. I’ve been using gmane for a number of years, far preferring NNTP over mailing lists.
Is there a way yet for individual users to add subscriptions? I’m looking to add the opensuse-heroes ML (archive at https://lists.opensuse.org/heroes/). It uses mlmmj, like the other openSUSE mailing lists that are already part of the collection.
There isn’t currently any way to add new lists, but I’m writing an interface for that now. Should appear within a week or two.
Cool, thanks for all your hard work on this. I’ll keep my eyes open for it.
Hi,
thanks for your work with gmane.
I have a problem from the migration.
1. We have a mailing list which is configured to have reply-to to another mailing list.
2. When the mail go to gmane.io, the reply-to are translated to gmane.orf and not to gmane.io
Should it be possible to fix it ?
Is this due to address encryption? That is, the public.gmane.org thing? Hopefully that’ll be fixed at some point, but I don’t have an ETA now. The encryption can be switched off though; just request that through the “edit list” interface.
Hi Lars,
the rpmfusion mailing lists (https://lists.rpmfusion.org/archives/) appear to lo longer work, the last emails available on gmane.io are from 7th January. I have tried resubscribing via admin.gmane.io but nothing has changed. Would you mind having a look? Thanks!
It takes a while for me to get to the requests.
I use gmane.io through Thunderbird and it’s a great service. However, the NetBSD mailing lists reject all my posts because of “forged headers”. Is there a way to know how gmane actually processes messages before being posted onto a mailing list, so that I can send this information to the NetBSD mailing list admins?
If you have SPF enabled on your domain, you probably can’t post via Gmane, since many mailing lists will reject mail coming from Gmane with your From address.
Hi, I am trying to add a mailing list to gmane. It says “The list main@Fistscw.groups.io is already subscribed as gmane.recreation.radio.morse.fists” but it is not actually to be found in the archive. Can you give a look at this, Lars?
Use the “edit” form to submit a re-subscribe request.
Thanks Lars but I can’t see any “Edit” button. Screenshot:
https://i.imgur.com/JcQO7Wg.png
Search for a list, and then “request data update”.