worm-blossom

worm-blossom

The endeavours of Aljoscha Meyer and Sammy Gwilym.

2026Week 713 Feb

“graceful_producer.slurp()”

A two panel comic.
        
        In the first panel, Sammy stands in front of a four panel comic of the same format as last week's. She says "So for this week's comic, let's invert last week's, and have you modelling t-shirts and I'm begging for your help?!"
        
        The second panel is introduced by the text: "Aljoscha's actual reply". Below, Aljoscha appears unmoved, and says "Sure, if you think that's funny". Sammy looks aghast in the corner.
A drawing of Sammy, one arm holding up her head with a pencil in it, the other resting on a piece of paper.

Shortly after last week’s update my body — sensing uncharacterstic relief — instantly succumbed to a flu that I’m still not completely rid of. So I spent a lot of time in bed reading The Rose Field and quietly feeling pleased with myself about my FOSDEM talk doing the rounds. Again, thank you for watching (and maybe even sharing) it, the response has meant a huge amount to me.

With no more slides to prepare, I returned to programming (shudder). It has been a while, and I feel clumsy and slow, especially in Rust. But I’ve started implementing the prerequisite encodings for Willow’s drop format. These use Willow’s private encodings, where items can only be successfully decoded with a bit of common knowledge. This might sound like overkill for a non-interactive method of exchange like the Drop format, but as Aljoscha reminded me: “turns out trying to omit as much data as possible results in short codes”. And what if that makes all the difference for someone to squeeze as much Willow data possible on their low-capacity USB key? That is what I tell myself as I port existing implementations to another, hopefully final incarnation of an Ufotofu-based encoding abstraction.

I’ve also been thinking about Discord, which will reportedly soon start requiring age verification to access all features. I belong to a few communities based on Discord which will sadly disperse if this ends up happening. And it doesn’t seem like there’s any persuasive answer for where to disperse to. Why is that? Is it because the pieces (protocols, people) aren’t there yet? Or is there a deeper problem in that even if we had all those pieces, fusing them into a monolithic whole requires an amount of coordination (and funding) we collectively can’t muster? I have included some links in this week's update to ponder.

~sammy

Shout-out!

Ufotofu now has combinators for chaining together two producers or two consumers, thanks to danmw. My favourite aspect of this contribution needs a bit of elaboration:

The BulkConsumerExt trait has a stage method, whose doc comment features the following paragraph:

This method only exists to preserve symmetry. We cannot even write a meaningful example for it. If you ever find a legitimate use, please let us know!

And as you probably already guessed: Dan's implementation of the chaining combinator for consumers uses stage! I added that method purely to preserve symmetry, anticipating that eventually some use for it would show up and that it would be really annoying if it was missing. Principled design, yay!

(The very fact that using this method is necessary in the first place showcases a problem with the current API design, but that’s a different story :>)

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

This week saw a fruitful discussion about Ufotofu, triggered by Dan’s implementation of chaining combinators. In a nutshell, the problem is that some methods irretrievably swallow up their arguments in case of an error.

There is an obvious fix, which is to have the error case return not only the actual error but also the original arguments. We should be able to apply this for bulk processors, but things are less clear for regular consumers: if consume returned the item it was supposed to consume in case of an error, then that would break the symmetry with the producer design.

For a terrible day-and-a-half or so, I was actually ready to break the symmetry here, in order to make the chaining combinator work without cloning (with a destructive consumer API, the only sensible workaround is to clone items before feeding them into the consumer, so that you still have the clone around after an error). But then I thankfully realised that everything works nicely for types that are Copy. So we can have “normal” APIs that require their items to implement Copy, and then do more expensive and clunky variants that require only Clone.

In any case, the discussion with Dan helped clarify a couple of muddy aspects around ufotofu, and has given me a new perspective on what makes Copy so special in the context of ufotofu items — Copy exempts types from linear typing, thus making them match my purely mathematical mental model of the design space.

~Aljoscha

2026Week 606 Feb

“blobjects”

A four panel comic, titled "SAMMY'S CONFERENCE OUTFITS!!!".
      
      In the first panel, Sammy is wearing a mesh top with high waisted jeans.
      
      In the second panel, Sammy is wearing a long sleeved striped shirt, turtleneck and midi skirt. A hand is thrust into the air behind her.
      
      In the third panel, Sammy is wearing a checkered skirt, dress shirt and scoop top. Someone says "help" in the background.
      
      In the final panel, Sammy is wearing a bolero jacket, scoop top, and houndstooth trousers. In the background, Aljoscha says "Sammy", "i'm dying".
A drawing of Sammy, one arm holding up her head with a pencil in it, the other resting on a piece of paper.

Back at last. And I owe the faithful readers of worm-blossom.org not one… but two editorials, as these last two weeks I was busy attending two very different events: p2p-basel and FOSDEM. The former is intimate and deep, the latter huge and sprawling. And I had a great time at both. Let’s start with Basel.

p2p-basel invites researchers and p2p practicioners to a weekend of lectures and talks at the University of Basel. This year there were only approximately 16 participants, and that creates a very intimate atmosphere for discussion. For example, the catering of the event is also peer-to-peer, with attendees participating in cooking and cleaning. Those moments where you stand shoulder to shoulder with someone while peeling carrots present wonderful opportunities for deep discussion.

A drawing of a conference room.

There are prepared presentations and a time dedicated to spontaneous unconference discussions, of which I moderated two this year.

The first was on ‘delay tolerant UIs', anticipating user confusion stemming from systems where authored data may only reach other users a week later. This might mean the arrival of a message divorced from its original context, or worse, placed into a new context which changes its meaning (e.g. a thumbs up emoji arriving during the discussion of something sad). Clearly we cannot simply lift the UIs of applications which presume near-instant data delivery and expect them to work.

a cropped photograph of a whiteboard with two columns: problems and mitigations. In the former column is a problem arising from delay tolerant systems in a UI, and the latter a suggested way to mitigate it.

The second was on ‘zero / low identity systems’, and I was really happy that so many others were also interested in this topic. Digital identity is often used to link different pieces of data together through a ‘digital double’ which can often be linked to a real person. The risk here speaks for itself, and I had a half-baked idea on how to reduce or eliminate identity in networks: random IDs for all data, with the option for the ‘owner’ of a random ID to derive a new one which everyone else can then verify is provably linked to the original ID. This forms a kind of ‘identity on demand’, or as someone else put it much more eloquently, ‘lazy identity’. It was pointed out that this is not conceptually unlike image board tripcodes (though I’ve since learnt these are apparently quite crackable). Maybe there’s something here, though? It would be interesting to plug such a system into Willow.

Huge thanks to p2p-basel’s organiser Erick, who has relentlessly improved the event year after year.

(CONTINUED BELOW)

~sammy

We're listening to...

Soichi Terada

While working late into the night I kept myself going by listening to the unreal jungle soundtrack of Ape Escape by Soichi Terada. No sooner had I mentioned this to our discord I was linked to this album by the same artist, which is just as jungle, just as fine, but a lot more chill and a lot less monkey. Thanks to rory k. for the recommendation!

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

Weekly updates and depressive episodes are not a good match.

I'm trying to take good care of myself, as much as I am able to. Even if that means taking a break in the exact week where FOSDEM has hopefully pulled a lot of fresh eyes toward us.

I often feel like Willow is coming into being too slowly. But even then, we are still doing meaningful and creative work, and putting it out for everyone to interact with. And that is something I can draw strength from, even when I am too off-balance to meaningfully act on that strength.

~Aljoscha

A drawing of Sammy, one arm holding up her head with a pencil in it, the other resting on a piece of paper.

A handful of days after returning from Basel, I departed again for FOSDEM in Brussels. This was my first time attending, and I was a little overwhelmed by the size of the event. So many buildings, people, and cat ears. And for some reason I chose to wear heels the whole weekend.

Luckily I was met by friends of worm-blossom (FoWB) Miaourt and danmw, and we spent Saturday discussing blobjects, fictional Willow companion hardware / tamagotchis, radio-friendly Willow, and Miaourt bearing a seemingly inexhaustible bag of waffles. We had a lovely worm-blossom.org dinner that evening, and although I did not say it at the time because I wanted to appear cool and not overly sappy, I thought to myself that if these were the kinds of people worm-blossom is interesting to, then surely we are doing something right.

I spent Sunday in the Local-first, CRDTs and Sync Engines devroom. My favourite talk was from Modal Collective, in which they detailed their motivation for building a new secure, iPhone-quality OS with p2p at its heart. It shared a lot of themes with my own presentation and I was excited that this line of thinking is turning into action. It’s always lovely to see adz of p2panda, and lovely to meet Tobias Bernard of GNOME too.

Then it was my turn to present. I’ve been working on my talk since the start of the year, and it had been through several incarnations before reaching its current one, leaving me a week and a half to produce approximately fifty drawings. Fortunately I embraced the scrappy spirit of worm-blossom.org, and put together possibly the only talk of FOSDEM with an opening cutscene (music provided by Aljoscha, naturally):

(There is also a live recording if you’d like to experience the live vibes and Q&A portion).

Readers, four days later I am still glowing from the reception to my talk. I’m so, so grateful for all the kind things people said to me afterwards, and the lovely conversations it afforded later that evening.

As I took the train home, I thought about how the p2p community has grown over the years, and how the connections between us have matured alongside the theory and implementations. Perhaps we can do something really wonderful together?

~sammy

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

The music for the intro of the FOSDEM presentation is a beepbox piece, of course:

~Aljoscha

2026Week 530 Jan

“we forgot to remember to express ourselves clearly”

A three panel comic, drawn with much less skill than usual.
        
        In the first panel, a bird says "Well, this is maybe a bit weird..."
        
        In the second panel, another bird continues "but Sammy couldn't do a comic this week..."
        
        The final panel consists of a badly drawn, grumpy owl concluding "and Aljoscha refused to draw anything but birds".
A bad drawing of Sammy, done in mspaint. It is bad because Aljoscha drew it.

Hello readers,

it is me, Samy Gwylim. Because I am focussing on preparing my presentation for fosdem this weekend, I cannot contribute to this week's update. I asked Aljoscha to handle things on his own. And for some reason I thought I wouldn't regret it.

This week, I was reflecting on Kirby Air Riding, the revolutionary home console game from the 90s that combined beat time with choppy jpegs of japanese cuisine. The game wonderfully captured the raw stuffs of human experience, and my assessment is not tinged by nostalgia at all. Maybe I should draw on this game for the fosdem presentation. No better way of standing out from the other presentations than with a healthy sprinkling of pixelated onigiri. Or, since I am low on time, salmon would be great? Genius.

~“samy”

Assorted Mottos

Each of our updates features a short motto in the header for that week. These are mostly snippets out of Sammy’s and Aljoscha’s conversations. Our buffer of collected snippets has grown a bit large by now, and it would be a shame to let good mottos go to waste. So here we go!

worm-blossom:

  • from quirky-but-bearable to completely insane
  • victory is all that matters
  • we are sorry
  • being-able-to-cross-a-moat-full-of-crocodiles-based access control
  • we didn’t have to do this, but we did it anyway
  • nobody had to do this, but we did it anyway
  • I feel like every time we try a crazy thing, we are rewarded for it
  • there's a certain amount of mania we just need to accept as part of your lifestyle
  • gardeners rather than architects
  • I’ve had another thought — oh no
  • look, we’re probably going to generate two more mottos over the course of this call
  • swedish-intention-coded dutch accent
  • why?
  • i *never* just get to state my beliefs as facts anymore, it’s so *unfair*
A bad drawing of Aljoscha, done in mspaint. It is bad because Aljoscha drew it.

So, P2P Basel happened. I had a nice and calm train ride there together with Andreas and Glyph from p2panda, where we avoided talking about p2p stuff for the most part, knowing the weekend would hold more than enough of that. I did get a bunch of music recommendations from Andreas.

The evening had a couple of workshop participants meet for lunch. I had a nice chat with Pierre from Pijul, who gave me some much needed reassurance that most everyone has moved on from their phd thesis topic by the time it comes down to writing, and also that most everyone hates publishing by that time. The key is to do it anyway, I guess. So for the first time in well over a year, I've actually been motivated to keep chipping away at this phd thingy.

Then Friday was the first actual day of the workshop. Except I didn't feel terribly well. I made it through the day, but ended up shambling back to the apartment with a fever. And then I spent the remaining weekend lying in bed, with Sammy bringing me the occasional meal from the venue.

I'm still recovering, so no implementation progress report this week. Though I am a bit concerned how long it has been since the last actual code release. I hope I'll get over the hump of releasing the first storage-backed-Bab version soon, so that things can get back to more regular, incremental feature releases.

~Aljoscha

We're listening to...

Kurt Vile

The first music recommendation Andreas gave to me on the train ride to Basel. I have a soft spot for this kind of repetition-heavy songwriting. Highlights for me are Hysteria and Mutinies.

2026Week 423 Jan

“all theory and no storage”

A three panel comic.
        
        In the first panel, Aljoscha and Sammy clamber towards a kind of monitor embedded in the rock. The monitor speaks: "Please"
        
        In the second panel, we close in on the screen of the monitor. It says: "You'll help me find mother..."
        
        The final panel is entirely taken up by the monitor's static and waves. In the centre of the noise, it says "won't you?".
A drawing of Sammy, one arm holding up her head with a pencil in it, the other resting on a piece of paper.

I’m writing this on the second leg of my seven hour train journey from the Hague to Basel. Like Aljoscha, I’m going to p2p-basel to listen rather than to speak. Tags: receptive, curious, critical. And frequently: confused. Plenty of leaning over to Aljoscha and whispering “what does that mean?”. I still remember him explaining monotonicity to me a few years ago with the patience of a saint (St. Numbers).

The FOSDEM talk’s shaping up rapidly. I’ve now updated its title and description to better reflect its contents: “Willow - protocols for an uncertain future”. This is going to be a very different take on Willow than we’ve presented before, so I hope it will be interesting even if you’ve been hanging around the project for a while.

This week, I want to start adding more random little one-off drawings to our weekly updates.

~sammy

We're listening to...

Sébastien Tellier

One of my favourite music listening moments is when you have the radio on, or are out and about somewhere, and you hear the beginning of something you can't quite put your finger on until it dawns on you that it's a remix or cover of a song you already love. Check out the Gilligan Ross remix.

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

I have been programming too much over the past weeks; it has been impoverishing my life. Case in point: tomorrow (I am writing this on Wednesday), I am leaving for p2p-basel, and I... have barely thought about that at all, being too busy doing computer things. I don't even have a presentation to give, whereas in the previous years my problem usually was having too many presentation ideas to choose from. Still, there is much to look forward to: I will be seeing old friends again, will meet new interesting folks (there'll be someone from the Pijul VCS, I might finally get closure on some question regarding this paper), there will be interesting discussions and fun evenings. And I'll be meeting Sammy in person for the first time in over a year!

Despite me programming too much, we will not get everything ready by FOSDEM that we had hoped to. Which makes me feel disappointed, despite the fact that I have actually been pretty productive: Bab slice verification is implemented, and I am close to having a simple storage backend that can store a single, contiguous slice. In terms of unlocking funding milestones, I am more productive than needed. Yet the self-imposed FOSDEM-deadline makes me feel bad about the current state of things.

Hence, I label the self-imposed deadline silly, and I will do my best to focus on everything I did implement instead of what is still missing. As usual, there is also the voice that wants Willow to finally become a real thing that people use. But if listening to that voice means wasting my time staring at a small rectangle all day, then I label that voice silly for now as well.

In less silly news, academia and macadamia have a Levenshtein distance of two. Hmmmm, macadamia... now that is something I can get behind!

~Aljoscha

2026Week 316 Jan

“it’s really just a hassle for *us*”

A four panel comic.
        
        In the first panel, Sammy look serious. The text above reads: "sammy thought about this. The mines were supposed to be metaphorical, and thus empty.". Aljoscha looks confused in the background and Sammy thinks to herself.
        
        In the second panel, we see a wider view of the mines. The text reads: "More importantly, it was meant to be free of the moral implications of its tacit extractivism.
        
        In the third panel, Sammy is set in pale green against a spiralling background. The text below her reads: "But don't the stories we tell say so much about the way we see the world? What we wish we could do without guilt? Sammy asked herself what this choice of setting said about her inner world!!!".
        
        In the final panel, Aljoscha reappears next to Sammy and says: "Hey. I took a look. I think there's a child trapped down here". Sammy winces.

We're listening to...

Sonic Youth

I (Aljoscha) rarely listen to music, but this song popped into my head multiple times this week. So I listened to myself (and the song) and tried to unwind a bit. I even had a cup of hot chocolate. This is a surprisingly good song for a cup of hot chocolate.

Burial

The star here is Archangel, composed to evoke a walk home through the city after a night out.

A drawing of Sammy, one arm holding up her head with a pencil in it, the other resting on a piece of paper.

This week my attention was once again fully occupied by the upcoming FOSDEM talk. After a run-through of a draft slide deck this morning, I felt dissatisfied, although everything was arguably there. Empty calories.

I am a person driven by dissatisfaction, and I work on things until I am happy with them. Aljoscha has the same malady. There is seemingly no cure. This makes the things we work on take a very long time. This website is a kind of compromise: we blog about our ongoing dissatisfaction on a weekly schedule.

So now I am restructuring the presentation for what is hopefully the final time. I mentioned last week that if you want to learn about what Willow is and how it works, we have a website. But why is Willow like it is? What kind of world made us feel such a thing was necessary? That is the kind of talk I think people will enjoy most.

It will be fun, maybe even insightful. If you have been reading worm-blossom.org for any amount of time, how this presentation turns out will not surprise you in the least.

~sammy

2026Week 209 Jan

“I had another thought while washing the dishes last night”

Another illustration of a underground scene. In the distant background, we see a jagged speech bubble coming from the rear of a cavern, crying "PLEASE". In the foreground, Aljoscha turns to Sammy (who is shuddering). The caption reads "I thought these mines were supposed to be purely metaphorical".
A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

My week was high-intensity but low-worm-blossom as I was single parenting. With the time I had between biking my kids to school through the snow, preparing meals, pleading with them to just open their mouth so I could brush their teeth... I mulled over my FOSDEM presentation.

I got stuck in a 'concept' phase. For me it would be a bit disappointing to just say what Willow is and explain it spec by spec. After all, we have a website for that. This presentation should act as a complement to it. But how? I sat around pondering. Thinking really hard. And guess what: it didn't help.

As soon as I switched to making — even with only a very vague idea — things began to progress. Being able to sketch, respond, tear down, iterate. Share an idea I'm excited about with Aljoscha, and he gets excited too. And now I can't wait to keep working on this thing.

And so the first thing we did was confirm with the devroom organisers that we would be able to play audio over the speakers.

~sammy

We're listening to...

Vega Trails

I heard Closer on the first night of the year, and it sort of perfectly summed up how 2025 had felt for me (sammy).

Michael Lückner

house full of time is just one of those tracks meant for cruising on your bike, gliding past people and lights and painted lines.

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

Rambly, reflective editorial incoming — creativity is fascinating (as is reading McGilchrist).

I was almost completely unable to make any progress coding this week, despite desperately wanting to. But whenever I opened the text editor, I would just be completely disinterested. I’d like to tell myself that I have the cognitive ability and trained skills to be able to work productively even when I’m not into it. But truthfully, the code I did force myself to write is so bad I will probably have to rework it at some point.

I usually struggle with creative output because I judge my own intuition too harshly and want to get things right. This website has been a good exercise in loosening up in that respect. But this week, the problem was of a different nature — I did not care enough to judge at all, leading to pretty bad code I normally would not let myself write. So where ideally there should be a healthy balance of generative creativity and quality control, this week had neither. And forcing things regardless just doesn’t work.

And then, out of nowhere, my brain delivered the solution for how to elegantly implement verification of incoming data streams in Bab. And suddenly, I felt like coding again, and have been happily coding since. This did not happen in time for finishing things by today, so I will detail this implementation approach next week.

Interestingly enough, while I was more or less unable to code, I did do some beepbox pieces I am not unhappy with after the fact — both of which I imagine working nicely in an RPG of some sorts, inspired by watching some Expedition 33 and Final Fantasy VII over the week:

And I did not dread the task of writing an editorial for this week either, despite having little idea what to write about before actually sitting down and starting to type. Not sure whether this says something about myself, or more about differences between writing, music, and coding.

~Aljoscha

2026Week 102 Jan

“agency over purity”

An illustration of a underground scene. In the distant background, we see two silhoutted figures, one looking towards the foreground. Close to us, we see a kind of monitor embedded in the rock, strange green waves on its screen. The caption below the illustration says "Did you hear something?"
A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

It was time to take a pause. In the last week I've spent time with family, ate, took my customary New Year's Eve bath, ate, and started a new journal for 2026. What little worm-blossom related stuff I did was thinking and sketching out my FOSDEM talk, specifically thinking about what you should say to an audience who have just seen twenty presentations on similar projects for the past several hours. Something high energy, controversial, more about vibes than details?

In honesty I am bracing myself for another intense year, perhaps harder than the last. So I am looking for ways to make it easier on myself. I found the practice of trying to make a funny comic each week a challenging one (with questionable results). I envy Aljoscha's free-form noodling in his little musical corner. So I'm going to have a little experiment with some more improvisational visual storytelling.

I hope you took some time to catch your breath over the last few weeks too, let's try our best!

~sammy

A teeny megaphone blaring out noise

No Announcement This Week

For the first time since launching the website, there is no announcement in the weekly update.

Wait a second...

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

Friday arrived so quickly, and I had not written an editorial yet. I have barely thought about the website update at all this week, actually. Does this mean I finally found a healthier attitude toward my work and this website, will I now lead a life of increased inner harmony and peace? Will my output dwindle as a consequence?

Nah, I simply spent most of the time learning and thinking about audio file formats and video file formats, in anticipation of the Cobweb and Rummager.

The number of file formats you would need to support to provide somewhat decent coverage for the media content on the web is intimidating, as is the staggering complexity of the individual file formats. So I have been pondering what a significantly more simple stack could look like. Only one file format per media type (images, sound, video), and simple formats at that.

The file formats would be pure delivery formats, no niceties for editing included. Further, the formats should support both lossy and lossless compression (which, sadly, rules out the Quite Okay Image format and the Quite Okay Audio format), down to extreme measures such as one-bit images or audio. And, ideally, the three formats would have identical or at least similar structure, and reuse the same underlying techniques. A nice challenge would be to fit all three formats (including their shared components) into 400 * 3 = 1200 lines of C code — the reference implementations of QOI and QOA take 400 lines of code each, after all.

A rough starting point for a unified approach to the three different file formats:

  • Regard each media type as a collection of channels, where each channel is a list of integer samples. For images, the four channels are three colour channels (YCbCr, specifically) and the alpha channel, and the one dimension is the row-wise content. For audio, each channel (one for mono, two for stereo, possibly more for more fancy stuff) is a stream of PCM data. For video, one channel encodes time-warying images, and the other channels are audio channels.
  • Channels are interleaved. Each channel has an independent frequency. For example, a video could encode images at 30 Hz but audio at 44.1 kHz. And an image could use lower frequencies for the colour channels (aka chroma subsampling).
  • Before encoding, quantise all signals to a channel-specific bitwidth. This is the (only) part where lossy compression enters the picture. The bitwidth of a channel can even be zero — say, for an alpha channel that has the same value everywhere. The simplemost quantisation scheme is simply to right-shift each sample by some amount, though for audio it is tempting to go with a non-linear quantisation scheme (because using fewer bits for high frequencies compared to low frequencies neatly exploits properties of human hearing).
  • On the quantised signals, use some form of Lempel-Ziv compression. When a value cannot be reconstructed from the dictionary, it is not encoded verbatim, however, but as a residual from a prediction (see next bullet point).
  • Each file format has a (different kind of) predictor, which tries to predict the next sample from the previous samples. We never encode samples verbatim, instead we encode the difference to the predicted value (the residual). With a good predictor, this means we will usually encode only small numbers, a property we can then exploit for more compact encodings. For images, the prediction is probably simply the previous sample. For sound, I’d tend toward reusing the LMS of the QOA format. And for video image frames, the pixel of each frame would be predicted as being the pixel at the same position of the previous frame, offset by a frame-specific x and y value (i.e., simple motion compensation, I'd also consider simple forms of Dolly and camera rotations if they admit simple integer-based implementations).
  • Finally, encode the residuals or the LZ compression stuff nicely and compactly. The easiest approach here would be to define a bit-based encoding and simply encode every sample separately, but it would also be nice to have a byte-based encoding — and grouping multiple samples together can be more efficient than encoding them individually. So there is a lot of design space here. Further, there are concerns around seeking: how can you efficiently jump to an arbitrary point of time in a time-varying signal? First, there probably needs to be a balance between absolutely encoded samples and relatively encoded samples. You'd use the absolute samples for seeking, and then do a linear scan across the remaining relative samples. The original anti-monotone binary linking scheme would be a really neat way for providing (only a bounded number of) forward-pointers between absolute samples. But still, with interleaved channels of different frequencies and bit-depths, and LZ compression, finding a good mechanism here is tricky.

There are further concerns such as container formats, and accessibility features like subtitles. But this would be the general idea for building an image format, an audio format, and a video format that all work in essentially the same way. It might not be unrealistic to use identical subsampling, quantisation, LZ compression and final encoding steps for all three formats. Aside from working out the specifics, the big remaining question is whether the lossless compression yields acceptable file size reductions compared to raw data, and whether the quantisation step (which is the only form of lossy compression in the whole approach) yields sufficient quality for the selected degree of compression. The two Quite Okay File formats make me fairly optimistic in that regard, however.

~Aljoscha

2025Week 5226 Dec

“potential”

Happy new year from worm-blossom.org!

An illustration of Aljoscha and Sam in mining gear, deep underground, surrounded by lava yet looking up towards the viewer with confidence.
A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

About two years ago, we published willowprotocol.org, and our server took a million requests on the first day. Soon we were getting emails from companies, universities, weird individuals, and it felt like we were really making something that made people sit up and take notice. We received our funding to build a rust implementation of Willow, and then… things stalled. It got quieter and quieter, until I imagine to the outside it seemed like nothing was happening at all. A lot did happen, of course, just not in a way we’d choose to have things happen again.

The end of 2025 brings the close to a weird era of willow, an era whose weirdness I’d like to reflect upon as I think I can recognise some of the ways it went differently than we’d hoped.

Our first mistake was maximalism. Willow Confidential Sync was the newest member of the Willow family, with the most novelty and flash. Range-based set reconciliation. Privately determining common interests between peers. Teeny tiny privacy-preserving message encodings.

As a poster child of privacy and efficiency, Confidential Sync radiated complexity onto everything around it. Willow storage needed to be able to efficiently fingerprint and partition ranges of local data for range-based set reconciliation. Storage needed special push-centric APIs so that new data could be broadcast to peers. Ufotofu needed a menagerie of encoding traits to be able to accommodate the private encodings which Confidential Sync uses to mitigate man-in-the-middle attacks.

We started to recognise the weight of this complexity early on, and specified the Willow Drop Format just a few months after the launch of willowprotocol.org. It’s a vastly simpler spec which asks far less of its dependencies. But at that point our funding goals and our destination were locked in: Confidential Sync was where we were going. And that path led us straight into the swamp.

At the same time, we were refactoring willowprotocol.org to use a new version of Macromania, the system we use to author the entire site (and this one!). Aljoscha made a big effort porting all the existing content to the new version, and it was so much nicer to author pages with that we couldn’t really go back to doing it the old way. And now we had a branch of the website with the new, not-quite-ready system and new content all mixed together. This interdependency congealed into a kind of seal which stopped us from making small updates to willowprotocol.orguntil everything was ready in one giant mega-update.

So willowprotocol.org was not getting updates. Which not only made it look like we were inactive to the outside world, but it also hurt our morale because we felt like we were digging into a bottomless pit with no recognition for our efforts. The longer the silence became, the worse our perfectionism got: we wanted to have something to show for ourselves after that long period of silence.

So we found ourselves working in frustrated silence amidst a myriad of interlocking dependencies. It was gruelling. In a struggle to keep the project on track, I ended up breaking personal connections which I treasured. There was also life outside of Willow which insisted on happening.

So what did we learn? Avoid complexity. Deliver frequently. Focus on doing as little possible to achieve your goal. Communicate often. How often have you heard these things? How many dozens of blog posts and presentations have you seen stressing these very points ad nauseam? I had certainly heard those hard-won lessons countless times — but found out I had to win them the hard way for myself, nevertheless.

I also have anti-takeaways: we were led into a swamp, but it was a very fruitful swamp. We used lots of time to make difficult, foundational design decisions, decisions which are already giving us increasingly stable foundations to work upon. It was long and tedious but it was also instructive, and we have the opportunity to actually use that experience.

worm-blossom.org is our antithesis to the era described in this editorial: frequent, ad lib, joyous. That’s what I want to bring more of to 2026!

~sammy

A teeny megaphone blaring out noise

Basic Store Trait

We have published version 0.6.1 of the willow-data-model crate. This update marks the beginning of our implementation of proper Willow storage, adding a basic trait for abstracting over stores, and a non-persistent MemoryStore implementing that trait.

Things are pretty basic right now; you cannot even implement synchronisation on top of the current trait, because it does not support receiving and verifying payloads incrementally. This will be the job of a subtrait. In general, the idea is to provide a hierarchy of traits, adding more advanced functionality such as Bab-related methods, the ability to summarise data for range-based set reconciliation, and so on.

We are also not building Willow’25-specific versions of this functionality yet, we want to first see how these new traits feel, and how they fit together with other pieces such as the Bab implementation. But still, this release is the first step toward the major missing building block before we (and everyone else) can start building actual applications. And now we have something to expand upon, to bring things into shape in time for FOSDEM!

Fuzzer Poetry

To the cargo-fuzz maintainers. We can look back to a year of productive testing, thanks to you.

I

There was a time when testing meant to me
Enumerating inputs and to pray
They'd cover everything there was to see
And missed no corner-cases on the way.

Then quickcheck promised instant peace of mind
Its elegant design had me enticed
Invariants and randomness combined
But sample sizes never quite sufficed.

A fuzzer proved to be the missing link
Ensuring every corner-case was checked
Allowing me to tentatively think
My code might actually be correct.

Now mind you writing tests is still a pain
But worth it for the confidence I gain.

II

I have been deeply in love
twice
no matter how long things go well
they can always shatter.

When I start the fuzzer I
wait
wait and wait
wait and wait and wait
pacing
wondering
if this will be the run that does not fail.

As the fuzzer hums and heats
reluctantly I let myself believe
but never know for certain.

How do I shield myself from disappointment
yet
allow myself
the hope that keeps me going
allow myself
to burn?

III

INFO: Running with entropic power schedule (0xFF,100).
INFO: Seed: 1658404568
INFO: Loaded 1 modules (74852 inline 8-bit counters): 74852 [0x562332a29c30, 0x562332a3c094),
INFO: Loaded 1 PC tables (74852 PCs): 74852 [0x562332a3c098,0x562332b606d8),
#2 INITED exec/s: 0 rss: 43Mb
NEW_FUNC[1/102]: 0x562332596b00 (BuildId: 4cf9597d4ab63dec5b3d24323cb8d9d0a41408a7)
NEW_FUNC[2/102]: 0x562332597000 (BuildId: 4cf9597d4ab63dec5b3d24323cb8d9d0a41408a7)
#5123 NEW cov: 726 ft: 727 corp: 2/51b lim: 53 exec/s: 0 rss: 46Mb L: 50/50 MS: 1 InsertRepeatedBytes-
#5124 NEW cov: 727 ft: 728 corp: 3/101b lim: 53 exec/s: 0 rss: 46Mb L: 50/50 MS: 1 CopyPart-
#5130 NEW cov: 727 ft: 748 corp: 4/151b lim: 53 exec/s: 0 rss: 46Mb L: 50/50 MS: 1 ChangeByte-
NEW_FUNC[1/11]: 0x5623325a6c60 (BuildId: 4cf9597d4ab63dec5b3d24323cb8d9d0a41408a7)
NEW_FUNC[2/11]: 0x5623325a8700 (BuildId: 4cf9597d4ab63dec5b3d24323cb8d9d0a41408a7)
Meanwhile though, our increasingly impersonal societies rend ever deeper wounds into our hearts.

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

What a year. Sammy looked back, and I will lay out some future plans — specifically, plans for Willow. By the end of January, we want to have a simple persistent storage implementation and a Willow Drop Format implementation. Obviously, we have many more plans. Here they are, roughly in the order in which we want to make them real.

With Meadowcap, persistent storage, and the Willow Drop Format implemented, it will be possible to build actual sneakernet-based applications. While it is tempting to add more powerful data synchronisation as the next step, we should probably revisit end-to-end encryption of entries and payloads first. The earlier we implement this, the fewer moving pieces we will need to change based on our learnings. The concrete scope of this will consist of specifying specific techniques and primitives as part of Willow’25, and then to augment our storage implementations with the ability to encrypt and decrypt data on the fly. Ideally, this work will introduce only small changes to our public APIs — but ensure that most everything going forward is encrypted by default. Realistically speaking, a whole bunch of data is going to end up on the servers of large cloud providers, after all.

Before I can describe the next endeavours we have planned, I need some slightly abstract groundwork: the distinction between pull-based and push-based APIs. In a pull-based API, the entity that wants data needs to actively ask for it. In a push-based API, a data provider can proactively send data to its subscribers as new data arrives. Push-based APIs have several advantages, but they are also significantly more complex to get right. To a large degree, I blame the subpar progress we have made in the past on prematurely trying to nail the perfect push-based APIs: the Willow Confidential Sync protocol for replicating data, as well as ambitious subscription-based APIs for interacting with local data stores. This is why we have adopted a pull-based-only approach for now; this decision has played a significant part in our frequency (and hopefully quality) of recent releases.

Keeping this push-vs-pull distinction in mind, the next logical step in making Willow more “real” is a synchronisation protocol over the internet. The Willow Confidential Sync protocol is deeply push-based, so it is not an option. Instead, we have concrete plans for a more HTTP-inspired protocol of relatively simple requests and responses. This is the Willow Transfer Protocol (WTP), and you can read a draft version here. Completing the WTP specification and implementing it will be the next larger chunk of work, and one I am looking forward to!

Once we have a WTP implementation, we will have a complete (pull-based) Willow stack. It will then be time to slowly move to more efficient push-based components. The first step there will probably be local APIs. More specifically, a Bab implementation that allows subscribing to new incoming data, and then a Willow storage implementation that allows subscribing to change events. This will be crucial for more advanced applications that want to dynamically maintain custom indexes for more efficient queries than simply interacting with a Willow storage directly. In the long term, I actually hope that Willow stores will be used for replication and as a foundational source-of-truth exclusively, whereas any data access would go through more appropriate special-purpose indexes. Subscribable stores are the key stepping stone for moving toward that world.

Another step we will need to take eventually is to provide a more efficient storage backend. Right now we are building simplistic, linear-time storage that simply scans through all data and filters out unwanted results. But in the mid-to-long-term, we will need a proper three-dimensional data structure to provide access in logarithmic time. I have thoughts already around a three-dimensional generalisation of zip-trees (and, possibly, G-trees). But these details are a topic for a different time.

Speaking of improved local data access: we currently provide only embedded Willow storage, but long-term we would like to move to an actual database server with which multiple applications can interact concurrently (using capabilities to scope access rights, of course). The basics of this are fairly straightforward, but the challenging part is making push-based APIs work. Ideally, we'd like applications to be able to maintain persistent subscriptions even across several sessions. This would require the database server to maintain an event log, and applications would supply an offset at which to resume their subscription upon connection. When an application falls too far behind, it would catch up via set reconciliation. This sounds nice in theory, but the actual specification and implementation process will probably be rather thorny.

And after all of that, there is Willow Confidential Sync. I assume that we will learn enough from the WTP to warrant reworking the confidential sync spec. And then we will need to implement it, of course. Which feels a bit daunting right now, yet also exciting — confidential sync is were so many of our design decisions converge into a package that will likely be greater than the sum of its parts. I want to live in the world where you can integrate that package into your application designs with miniscule effort.

Other Willow-related plans include benchmarking suites, transactional APIs for our local data storage, a peer-sampling service (possibly in concert with confidential sync), and implementing the Willow URIs spec. Then there are application ideas — Rummager, the Cobweb, perhaps libraries of common UI components. And beyond Willow, we still want to release Macromania, and we have too many ideas for other side-projects to even start enumerating them here.

This is already far more than we can hope to accomplish in 2026. And we’ll doubtlessly accumulate even more ideas along the way. 2026 will be a busy year, but also a fun one, and one brimming with potential.

~Aljoscha