My New Concert Blog

Multipletap at Cafe Oto was a lovely two-day festival featuring a butt-load of great Japanese peeps. Mostly noise stuff, but Ko Ishikawa played a lovely flute set, and Yumiko Tanaka did a very intense string-based thing.

Day one was good, and on day two they fixed everything that didn’t work on day one. Seating, videos, sound problems, etc. Iteration!

New Gmane SSDs

The Gmane news spool is 97% full, so I either had to delete some Gwene stuff, or buy more SSDs. Image

I bought more SSDs.  The current setup is 5x 512GB Samsungs in RAID5.  I bought 5x 1TB while in the US, so that gives us 2x the current size in RAID5, which should be enough for the next uhm five years? or so?

But the problem is how to do the switchover.  The last time it was pretty seamless.  I set up a new, spiffy machine with a spiffy hardware RAID controller. (See my secret diary for details.) Then synced all the articles over, and swapped some IP addresses at the end.

I really don’t want to buy another spiffy RAID controller this time.  But if I’m reusing the current hardware, there’s going to be downtime.

Here’s the plan:

1) Sync the spool over to an external SATA disk.

2) Take the server down, swap in the new SSDs, set up the RAID.

3) Rsync the articles from the external SATA disk to the RAID.

5) Profit!

2-5 will realistically take a day or so, with Gmane being totally dead while this is happening.  I think.

Hm…  or I could point the spool to the external disk while doing 3.  In read only mode.  Then there should only be a few hours downtime while I’m doing the RAID rebuild.  Hm.  Yes, I think that sounds doable…

So a few hours complete deadness, and a day or so in read-only mode.

But it’s a bit scary doing it this way.  If I were doing a completely separate new server, there would always be an easy way to roll back if things don’t work…


Emacs Cloud

I’m taking a vacation from my programming day job to fix Gnus/Emacs stuff.  As well as going to Utah.  But that’s finally over, and I’m Imageback on a couch in San Francisco.

I think I’ve covered the outstanding bugs now, so it’s time for new features.  Mah favourite.

First off: I’ve been meaning to implement “cloud” features for Gnus for yonks, but the right time never seemed to arrive.  But now it’s here!

Basically, my vision is that you’d open an Emacs on a totally fresh machine, say `M-x gnus’, answer a couple of questions, and then you have your entire setup on the new machine.

Using magic?  No.  Using Dropbox?  No.

Using IMAP.

Everybody has an IMAP account.  Somewhere.  Even if it’s only Gmail.

Basically, it should work like this:

There’s a special “*Emacs Cloud*” mailbox.  This contains a sequence of data chunks that will bring your Emacs/Gnus setup up-to-date.  There are base chunks, and there are incremental hunks.

The Emacs Cloud will cover a subset of the files needed: .gnus.el, News/*.SCORE, .authinfo, and .signature.  And perhaps a few more.  If you edit these files, they’ll be uploaded (in full) the next time you sync.  In addition, Gnus deltas are handled specially.  If you read a message in rec.arts.sf.written, a command to propagate that is added to a trace file that’s also uploaded.

The first time you ask Gnus to start clouding it up, it’ll upload all this data, in full, in a “base chunk”.  After this, only “incremental chunks” are uploaded.  When downloading, Gnus will download enough stuff to recreate the situation on the other side(s).  When the incremental chunks (cumulatively) become big enough that they’re bigger than a base chunk would be, a new base chunk is uploaded.

Before uploading anything, the entire chunk is compressed, encrypted (using symmetrical encryption), and then base64-encoded.

So, to return to the opening scenario: You’re on a fresh machine.  You say `M-x gnus’.  Gnus will then ask you “Do you wish to sync with the Emacs Cloud?”  You answer “yes”, and Gnus then asks you what the IMAP server is, what the IMAP user name and password are, and what the Emacs Cloud password is.  Gnus then connects to the IMAP server and downloads the data.  And there you are.

Later, during normal usage, Gnus will automatically detect updates done in other instances, and download/upload data as required.

So you get a kinda sorta virtual file system over IMAP.

Details to be decided: What kind of format to use for the chunks (something rnews-like?), and what encryption to use.  And…  er…  no, I think that’s it.

And then there’s only typing remaining and we’re done.  Unless I get a cold.  Then there’s procrastinating before starting to type.

Now, if somebody would just release an Android phone with a physical built-in keyboard, then I could just run Emacs natively on the phone, and read all mailing lists and blogs in Emacs.  And with eww, I’d never have to leave Emacs and actually use the horrible Android OS.