This week I have a gazillion things to share; I blame the fact that I took a proper vacation for the first time in ages.

After planning for a long time, we stripped out all generics from our Rust Willow implementation. This makes things, well, less generic, but also provides a tangible and honestly rather necessary reduction in complexity. The pull request ends up reducing our lines-of-code by about 15000. We have published the new code as willow25 0.7.0, and we have deprecated the old, generic willow_data_model and meadowcap crates.

On the WTP side of things, I realised that there is little reason to keep the protocol asymmetric. If both peers are allowed to request entries, that might even be another way of removing RequestPut messages from the protocol. But this decision is sufficiently fundamental that I have postponed working on the WTP until Sammy has enough time on her hands again to have her say as well.

So instead I turned back to my media file format designs. I have now specified exactly how to encode samples. All that is left now are the boring details of format-specific metadata. With the exception of settling on the sample predictor for audio, all the hard work is now done.

And as part of that work, I nerd-sniped myself into thinking about the role of magic numbers in file systems. So I wrote down a half-specification half-rant document that overthinks the topic quite a bit. Turns out I harbour strong feelings about magic numbers (who doesn't, right? Right?), to the point where Emmy and Miaourt had to make me tone down the polemics. Thank you folks =S

Speaking of overthinking things in a good way, I'd like to draw attention to this sneakerweb PR about presenting a sneakerweb collection in a random order. Somehow that premise escalated into a discussion of social dynamics, non-uniform sampling, and time complexity of various algorithm choices. In a good way.

There's been some other progress on the sneakerweb as well, with Emmy spearheading the modestly-named WIP: fix-all-the-things pull request. At the end of that PR will be the release of version 1.2.0, bringing some new features and making the sneakerweb work in Chrome and Edge. Thank you, Emmy (and sorry that I have somewhat dropped out of the loop regarding the state of the sneakerweb implementation)!

~Aljoscha

This week I’ve been with the kids. It’s summer holidays and a lot of their friends are away, so we’ve spent pretty much the whole week together. I didn’t even try to work, it wouldn’t have been any good.

The kids have a vague notion of what I do. It involves computers, they know that much. My older kid knows it involves ‘networks’. Mostly they know that I have a blog with fun music by Aljoscha, and comics which they like (their favourite is this one), and that my work means I bring home lots of stickers.

Several years ago I was a consultant working with commercial clients. After my son was born and started to become a little more conscious of the world, I started to get this sense of “I want my kid to think I’m cool” (I know, I’m doomed to failure, but I’m going to try). And I just couldn’t imagine my kids thinking it was cool that I made React apps for shoe stores (which back then was really pretty lucrative).

So I gave up doing the safe thing.

~sammy

The day before I headed to my vacation, I had an idea for a blog post that I started feverishly drafting. And now that I returned, the first thing I did was to complete it. It starts from a simple idea: what happens if you structure a programming language design around the broadcasting of information rather than calling methods on specific objects? This starting point has some interesting implications, from abandoning the notion of object identity, to spatial computing based on active broadcast rather than passive sensing. Read the full post on Broadcast-Based Programming here =).

I am slowly returning to more practical worm-blossom concerns now, in particular I want to start fleshing out the WTP into a full specification.

The WTP has three types of requests: RequestGet (analogous to HTTP GET), RequestGetMany (for requesting areas or 3d ranges), and RequestPut (analogoues to HTTP PUT). For a while, I was seriously considering to remove RequestPut, turning the WTP into a purely pull-based, read-only protocol. But while this has a certain elegance, it would force application developers to use a separate protocol (such as dripdrop) to write any (new) entries to a peer. In that world, you would either need to multiplex connections or to open multiple physical connections. And that would add complexity that outweighs the minimalism and orthogonality we would have achieved by trimming down the WTP.

With this decision made, here are the next steps toward a completed WTP:

• Rework the start-of-connection message flow. The current definitions assume that the very first thing every client has to do is to prove that they have the secret for the public key they will use in their read capabilities. But RequestPut messages do not involve read capabilities at all, and I can easily imagine write-only clients that never work with any read capabilities. And such clients shouldn’t be forced to do useless signature computations, nor should they have to wait for a server challenge before they can start issuing requests.
• Remove non-Bab usage. Partial verification is such a good feature that I do not want to encourage implementations that skip it. So no raw indexing into payloads by byte offset without Bab verification metadata.
• Define the responses to GetMany requests.
• Define encodings for everything. Surely this will be quick.
• Sketch Rust types for all request and response types. Use this to guide the way the spec defines things; currently the spec describes many optional fields together with rules for when they have to be present based on the values of other fields, and this would be much nicer to define in terms of enums.
• Pester Sammy to design a logotype.

And with that, we would have a completely specified and half-implemented WTP already. And we are already brainstorming applications we would like to implement that would be enabled by this.

~Aljoscha

Multiple people* have now confronted us about how confusing it is to navigate the family of worm-blossom websites. That there should be some kind of unified navigation between the worm-blossom, Willow, Bab, and sneakerweb sites. And being between projects, that seems like the perfect thing to do.

We'll see when it gets done. It's summer holidays and I'm with the kids next week. Many things simmering but nothing to report.

* Christine, and Aljoscha's dad

~sammy

I am still on vacation. But you know how zoologists determine whether an animal is kept in a way appropriate for its species by observing whether it retreats into passivity or whether it is active, exploratory, and playful? By that same metric, I am having a great vacation.

~Aljoscha

I'm not on vacation. But I'm not really on top of it yet. So this week, part eight of “Why is it called worm-blossom?”.

~sammy

Readers, this week Aljoscha and I are on holiday. So I've done what I can this week to evoke the feeling of my holiday, a summer break in a family home full of old paintings and photographs of times long past. Aljoscha's buggered off for this week's update, and good for him.

Last week I promised a truckload of impressions from Dweb camp, but to be honest most of those remaining reflections are about presentations I saw and the speakers I met, and I would like to share accompanying recordings of them, and I'm waiting for those to become available.

Instead I've been enjoying the intellectual slack you get from being on holiday. First there is just collapse, catching of breath, recuperation. But then after a few days you pick up the thread again, start laying out little patterns on the floor, experimenting with interesting knots. And before you know it, you're 'just doing a little bit of research'.

The next phase of Willow will be about exploring what sits just above the extreme latency of the Willow Drop Format and sneakerweb. So: networked communication. My plan is to use the sneakerweb itself to find the limits of comfort, the point where you say "this is fun, but I really don't want to do this with USB keys anymore". I think it will be really nice to use our own tools to do research in this way (I think this is what people call “drinking your own champagne”).

~sammy

Reader I am shattered. Knackered. Demolished. But home and with enough experiences for several weeks' worth of worm-blossom updates. So let’s start unloading the truck.

Sneakerweb was very much at the heart of worm-blossom’s presence at DWeb camp this year. We had a table at the ‘demo night market’, in which we had to beckon passers-by with nothing but our charms. The only problem: our neighbour was the inimitable Christine Lemmer-Webber of the Spritely Institute, who not only had prepared an incredible multi-screen, multimedia experience, but also has the overpowering voice of a lady who sells onions at the market. We did our humble best, I developed my patter (“you publish websites to your computer — no web hosts, no domain registrars, no credit cards!”) and spread the good word. And someone even came to our table with a .snk file.

We also held a sneakerweb workshop in which we installed the sneakerweb CLI, claimed some domains, exported some .snk files, and sent our participants off to the four corners of the earth with the seeds of the euro sneakerweb. I had that fantastic moment where you see how other people use something you made unprompted, and you go ohhhhh this is what this thing really is? We are learning a lot about the sneakerweb and folding it back in as quickly as we can. For starters: Aljoscha added a wonderful new getting started section to sneakerweb.org.

During the confererence Aljoscha would often walk by me with a USB key in hand, and say something like “and this one is going to New Zealand”. He was the star of coordinating the physical distribution of our corner of the sneakerweb, so thank you Aljoscha.

It feels like we’ve got a project that has a life of its own.

~sammy

Sneeeeeeeaaaaaaaakerwebbbbbbbb! At the very least, the sneakerwebsites we brought made it to Canada, the US, Argentina, New Zealand, Switzerland, the Netherlands, the UK, Denmark, and Germany.

Also, there have been quite a few contributors over in our repo and issue tracker. We are going to release a new version with a whole bunch of improvements fairly soon I hope. And if anyone out there has expertise in setting up cross-compilation for creating Windows and MacOS builds from a linux machine, please reach out!

DWeb Camp was a lot more than only sneakerweb distribution, of course. Meeting old and new friends, electro-acoustic jam sessions, stumbling into people who know what certificate transparency (my phd topic, I guess) is, interesting presentations and discussions, learning to set up a tent, overcoming crippling social anxiety, all that good stuff.

But, somewhat surprisingly to myself, the biggest effect camp had on me is a renewed sense of how political this space is. Not only the need for decentralised tools over centralised tools, but also the nuances within the decentralised world itself. I am interested in software that minimises hierarchy and power-over, and that is definitely not the case for all decentralised technology. Blockchains tend to reinforce the notion that some participants have more rights and value than others. AI tooling has its own baggage when it comes to autonomy and agency. Even something as innocent as requiring a working internet connection to function forces a dependency on users that could perhaps be avoided.

This has given me quite a bit of energy. Both to work on our projects, and to perhaps attempt to be more vocal about those values that — for me — make the differences between distributed systems, decentralised systems, and truly peer-to-peer systems. The introduction to p2p systems I started drafting a while ago is going to be far from a neutral technical document. But more on that when the time is right...

Finally, I want to thank Christine for her guest comic again, and also Dan and Miaourt for their respective editorials last week. A decade ago, I thought about how cool it was that so many webcomics ran guest updates when the authors took a break. And a decade later, I can finally confirm that being in a position to give the stage to friends is just as cool. Much love!

~Aljoscha

The calm of this week’s background track belies how hectic a week this has been — hence only a short update. Sammy has already travelled to DWeb camp and is busy setting things up. My preparations for DWeb camp have also entered the final phase. As have the exams I’m inflicting on my poor students. But basically all of my time not spent on teaching has been going into the sneakerweb.

Thanks to Dan’s tireless contributions, we have running software. I have been crafting sneakerwebsites. We have exchanged and ingested the first Willow Drop Format data. I am storing and reading a site that Sammy created. And it all runs on our own protocols and software. Willow feel so very real all of sudden!

And I can’t wait to share the neatness (neatidy? neatle?) with others at camp =)

~Aljoscha

Dear reader, it is hot. I write this editorial next to four strategically opened windows. This week’s heatwave prevented my ageing laptop from joining any video calls this week. What does our future look like? It’s going to be one where we use less energy, by choice or by force majeure.

Fortunately it seems things will cool down before I need to spend a week in a tent at DWeb camp. The schedule’s been announced, and worm-blossom has three events: The Willow Skill Tree, a Sneakerweb workshop, and Sneakerweb at the ‘demo night market’.

Sneakerweb has been the most prep. It needs some sites on it to give people an idea of what’s possible, so I’ve created sammy’s guide to the sneakerweb. My plan is to never publish this website on the world wide web. If you want to see it you’ll need to get it from me, or one of my friends, or one of my friend’s friends (etc), and import it into your sneakerweb collection. Aljoscha has also been preparing a few things for the sneakerweb, things which I have made drawings for that I will let Aljoscha choose whether to share or not (how firmly does he believe in the law of the sneakerweb?).

I’ve also been doing some grant proposal writing for the sneakerweb. Although this kind of writing is very annoying to do, at least it sort of forces you to express your stupid ideas as clearly as possible. Some of that writing I folded back into the upcoming sneakerweb site for the www, which is ready to go but for the downloads page. My funding ideas revolve around the question of whether we can harden the sneakerweb enough that it becomes useful in adversarial environments. I think there’s some obvious things we could try in this direction.

Anyway, this time next week I’ll be in Germany, writing an update from my tent. I need a palette cleanser.

~sammy

A couple things I want to report. First, the folks implementing Bab in Zig found a bug in our own implementation that resulted in incorrect WILLIAM3 digests. And then I found some more bugs. I've managed to fix everything I could identify, but our implementation still appears to diverge from BLAKE3 in ways that it shouldn’t. So no complete fixes published yet.

Second, Dan and I went on a fun spiritual journey to discover the true meaning of Ufotofu. Okay, maybe it was slightly less grand. But we did learn something about when to treat producers as being equal (or, more technically correct, as being equivalent in the finest sensible way).

The obvious approach is to consider two producers as equal if they exhibit the same observable behaviour in terms of yielded items, final values, and error values. Given that bulk production must be equivalent to item-by-item production and given that slurping is a transparent optimisation detail, this observable behaviour should be independent on which method calls are being used. Or so we thought. At the end of a long debugging session, we realised that actually slurping is not completely transparent. If it was, it would have to return the unit type after all, but it is allowed to report errors instead. And this can be relevant: we encountered a situation where it was absolutely sensible for slurping to yield a different error than the error that would be yielded when attempting to produce an item from the exact same state.

This left us with two choices: either allow observable side-effects when slurping (and, dually, flushing) and remove the check for error equality from ProducerExt::equals, or change the return type of slurp and flush to (). The latter is the cleaner solution mathematically, and would arguably be appropriate in a lazy language such as Haskell. And for slurping, infallibility is easy difficult to justify: producers already have to buffer any error they encounter if they encounter some items first. But the part that is difficult to justify is infallible flushing. A flush method that fails silently and delays reporting any encountered error until when you try to write the next item (or the final value, i.e., you close the consumer) feels, uhm, difficult to justify, to put it mildly. Flushing corresponds to fsync in the UNIX world after all, and a silently failing fsync is utterly absurd.

So we left the API unchanged and adjusted the mental model of producer equality instead. Because we are nothing if not pragmatic here at worm-blossom, especially when it comes to ufotofu.

And third and finally, I can reveal the secret side project I briefly mentioned a while ago: using the same tool as I use for the weekly background music, I made an album. For now, Commputer Has No Feelings will be distributed via sneakerweb only, and it features catchy titles such as Your deductions are only as good as your initial assumptions and Sometimes you need to wait a while before you can think about flowers again, but seeing flowers speeds up the process. Sammy drew the lovely cover artwork you can see to the right.

And around May, I accidentally finished a second album. Circumference has longer songs with shorter titles. And will also be available only via sneakernet for a while.

It is so enjoyable to go beyond short looping background music, and to devote more time to polishing the tracks. And despite the restrictive songwriting tool and the occasional goofy song title, making these albums was actually a form of emotional processing. So I hope that some of that music will resonate with others. I am excited to finally have the custom software that was 100% necessary to share these.

~Aljoscha