worm-blossom

worm-blossom

The endeavours of Aljoscha Meyer and Sam Gwilym.

Joni Mitchell — I Had a King
The Mothers of Invention — Hungry Freaks, Daddy
Crosby, Stills, & Nash — Guinnevere
The Velvet Underground — Heroin
King Crimson — Lament
Patty Smith — Birdland
Gentle Giant — Aspirations
Kate Bush — The Man with the Child in His Eyes
Prince — I Would Die 4 U
Arthur Russel — Wonder Boy
David Bowie — Looking for Satellites
Marvin Pontiac — Bring Me Rocks
Sonic Youth — Expressway To Yr. Skull
Mr. Bungle — Quote Unquote
Neutral Milk Hotel — In the Aeroplane Over the Sea
Jeff Buckley — Mojo Pin
Sleepytime Gorilla Museum — Ambugaton
Hobson's Choice — We Dream, We Are
King Gizzard and the Lizard Wizard — Mr. Beat
tUnE-yArDs — Gangsta
Louis Cole — Your Moon
Buke and Gase — Sleep Gets Your Ghost
Mingjia & The Tortoise Orchestra — I Had a Mouth Once
Punch Brothers — Familiarity
The worm-blossom advent calendar is underway!

2025Week 4905 Dec

“surprise! spiritual growth!”

A comic with three panels.
        
In the first panel, Sammy is looking at a laptop, and asks: "What is it about 'Spotify Wrapped' that gets people so excited? Aljoscha, watering a plant in the background replies: "It's a fun way to summarise the year?"

In the second panel, Sammy asks: "But what about the people who don't use Spotify? How are we supposed to learn something meaningless about ourselves?". Aljoscha is in the foreground, his glasses glinting in the light.

Before the next panel, the word "LATER".

In the third panel, we see a print out in a monospace typeface, like a console. In ASCII art, it is titled "RUST UNWRAPPED". Below, it says:

- 14480 compiler errors raised
- 95483 warnings ignored
- 45 panics in production
- Ran cargo clean 98 times
- Target directory grew to 107 gigabytes
- You cried out of frustration only once this year
- The borrow checker cried out of frustration 
  only 37 times this year

AND BEST OF ALL…
- Time spent on garbage collection: 0 seconds
A teeny megaphone blaring out noise

Capabilities in willow25!

We've added a batteries-included version of the meadowcap capability system into willow25. The batteries in question are made of the secure and efficient ed25519 signature scheme.

use rand::rngs::OsRng;
use willow25::prelude::*;

// Randomly generate a new keypair.
let mut csprng = OsRng;
let (subspace_id, secret) =
  randomly_generate_subspace(&mut csprng);
let namespace_id =
  NamespaceId::from_bytes(&[17; 32]);

// Create a write capability.
let mut cap = WriteCapability::new_communal(
  namespace_id.clone(),
  subspace_id.clone(),
);

// We have a capability and the secret
// for its receiver. Time to write an
// authenticated entry.
let entry = Entry::builder()
  .namespace_id(namespace_id.clone())
  .subspace_id(subspace_id.clone())
  .path(path!("/ideas"))
  .timestamp(12345)
  .payload(b"chocolate with mustard")
  .build().unwrap();

let authed = entry
  .into_authorised_entry(&cap, &secret);

// It worked!
assert!(authed.is_some());

With access control in place, willow25 is ready for the next step of implementing persistent entry storage.

Shout-out!

Thank you to danmw, who has contributed new Path::create_slice and Path::create_suffix methods to willow_rs and willow25.

This could have been a very simple task, but the resulting code would not have been optimally efficient. So Dan went ahead and adjusted our Path struct, which required delving into the least-readable parts of our codebase. Didn’t seem to be a problem though, everything runs great — and in O(1) time instead of the O(n) solution we would have expected. Thank you for this effort!

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

Two weeks ago, @Soapdog asked me whether I could put the weekly songs on this website into a playlist. And thus, a slow descent into madness began. The tool in which I write these tracks has absolutely no support for that kind of thing — but it is open source and can be forked. And thus, I proudly present the official worm-blossom playlists page. Featuring not only a playlist of all weekly songs (automatically populated by Macromania, obviously), but also a bonus playlist of a bunch of tracks I have accumulated in a plaintext file over the years.

The original tool for creating this music is Beepbox, but I have been using a “mod” of it, which really just means a fork. Or rather, a fork of a fork [...] of a fork. There’s a family tree of how these mods have evolved, and I decided to fork pretty close to the leaves, at the 2024 mod AbyssBox (primarily because it has a ring modulation effect). The codebase is a succession of mostly independent, mostly undocumented spaghetti modifications to an undercommented core. So everything took longer than expected, but eventually I succeeded in adding my own underdocumented spaghetti modifications!

The codebase I started from had no notion of multi-song containers at all. The first thing I had to add was code that would allow the synthesizer to emit a notification once it had finished playing its song. Next, since many of the songs are infinite loops, I added support for slowly fading out a track after some number of repetitions (fun fact: you need a logarithmic decrease in volume, not a linear one). Finally, at the start and end of playback there would always be an audible click sound, so I added a miniscule exponential ramp-in and ramp-out. I cannot unhear that click in other beepbox mods now!

As a bonus, since we have full control over the audio player now instead of merely embedding it, I adjusted the colour scheme to be even more worm-blossomy — and I could finally remove some vertical spacing that had been bothering Sam! All of this was a fun side-quest, but now I am completely done with it and will almost certainly not add any additional features such as a playlist-to-zip-of-mp3s export, earwig compatibility, or even custom sound synthesis to it.

~Aljoscha

A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

For this week's comic, I composed an image to look like console output, printed it out, cut it out and glued it to the rest of the comic before scanning it back in and putting it on the internet. I hope somebody out there appreciates that. I make the original Rust Unwrapped image available here.

There's only one month left to do all the things I really wanted to do in 2025. Which means this week I got curtain bangs and started prototyping a video game. Will it be any good? If you never hear about it again, you will know the answer (the bangs look great though).

~sammy

A friendly computer waving his mouse, beckoning you to come contribute!!!

Contributions Welcome: Add Meadowcap default values (Rust)

https://codeberg.org/worm-blossom/willow_rs/issues/21

In Willow25, we specify a well-known keypair to use as a default, effectively enabling public access to any data “secured” by this keypair. But we have not added it to the Rust implementation yet. This issue is a nice opportunity for getting familiar with the new Meadowcap API.

It has been really nice to see pull requests come in for our contribution-welcome issues. For Willow, there are only a few left. We have had less luck with our ufotofu issues, but who knows, maybe this’ll be the week?

2025Week 4828 Nov

“some of our ducks are on a train in the wrong direction”

A five-panel comic.
        
In the first panel, we see Sammy sitting at a desk. She says "So I'll just keep working on this URI stuff. How about you?". A voice crackles from her monitor: "well...".

In the second panel, we see Aljocsha on the screen, his hair blown about. The background shows a blue sky with clouds. He says "these encodings are taking a bit longer than I'd hoped, but I think Mea-".

In the third panel, Aljoscha continues speaking: "Hang on. Plane." He looks up at a plane soaring above him in the sky.

In the fourth panel, we see Sammy sitting at her desk again. She asks: "Aljoscha, are you in the sky?". Her monitor replies: "Oh! Did I not tell you?".

In the final panel, we see Aljoscha riding an enormous bird-like thing, laptop in front of him. Aljoscha says "This week I've implemented a bird. It has a principled design and requires only O(1) space."

what's with the name?!

why is this website called ‘worm-blossom’? In this series we will explain why in a concise manner.

Part 3: The Sega Dreamcast

It is 1999. Isao Okawa, the chairman of Sega, is presiding over the release of what will be Sega’s final home console, the Dreamcast. Okawa has long advocated for Sega to give up on hardware and become a third-party publisher, but has supported the Dreamcast for one reason alone: it is the first home console to include a built-in modem.

Okawa believes so steadfastly in the power of the internet that he has personally paid for every Dreamcast in Japan to be bundled with a year of free internet access. He has instructed Yuji Naka, head of Sonic Team, to begin working on a flagship online game for the Dreamcast.

But Sonic Team has never built an online game before. They’ve spent the last few months working on ChuChu Rocket! just so they could learn more about designing networked games. But ChuChu Rocket is a 2D puzzle game, and what they have in mind is more ambitious: a 3D action roleplaying-game inspired by Diablo.

Beyond the technical complexities, connecting thousands of players from across the world brings its own design challenges: how do players speaking different languages communicate with each other when all they have is a gamepad? And how can they coordinate play when a party’s members are spread across the globe, across timezones?

Coincidentally, a deal between Sega and Swatch has been brewing. Okawa is one of the primary donors of MIT’s Junior Summits, and would have been present when Negroponte and Hayek Sr. announced Swatch Internet Time one year earlier. This is likely how he came to announce a new business alliance which will see Swatch Internet Time built into Sega’s software, including Sonic Team’s new online title.

One year later, Phantasy Star Online sees players from across the globe banding together to defeat enemies in real time, transcending language barriers using Symbol Chat, and coordinating play sessions using beats. It is introduced by a bombastic trailer which claims that the game allows players to do “whatever you want, 1000 beats a day, 365 days a year”. It is also a game where you play as a mercenary for a civilisation which has colonised a planet and in doing so inadvertently awakened a universally corrupting evil.

Phantasy Star Online is warmly received as a genre-defining release for home consoles, but Sega’s Dreamcast is discontinued just a few months after its launch, outflanked by competitors. Sega transitions from hardware manufacturer to third-party publisher, and Phantasy Star Online outlives the platform it was built to elevate.

But why is worm-blossom called that? Up to now, we've occupied ourselves with people who believed they could change the world with their systems. Next week we get to the part where I don’t have to do as much research to write these.

A teeny megaphone blaring out noise

meadowcap crate

We have released the meadowcap crate, a Rust implementation of our Meadowcap specification. Yay!

Meadowcap is the bespoke capability system we recommend for usage with Willow. You can use it to authorise the writing of new entries, and to determine whether a peer is allowed to request certain entries from you. Access can be scoped to arbitrary Areas, and anyone with access to some area can delegate that access to anyone else, optionally restricting the scope to an even smaller area.

This update also implements a spec change we did this week: our notions for denoting sets of potential entries based on their coordinate in the three-dimensional subspace_id-path-timestamp space must always denote nonempty groupings of entries now. Aljoscha’s editorial tells the story of how that change came about.

We did not manage to update the willow25 crate with a plug-and-play instantiation of Meadowcap this week — stay tuned for that update!

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

Equality is a notion we often use without paying close attention to it, and sometimes that can burn us. Take for example the notion of ranges in Willow. We implicitly assume a syntactic notion of equality: two ranges are equal when they have equal start and end points. An alternate definition might be a semantic one, where two ranges would be considered equal if they describe the same set of values.

If you allow ranges to be empty (i.e., ranges whose end value is less than or equal to their start value), you get a mismatch between the two definitions: the ranges [2..2) and [3..3) (and [7..4), for that matter) contain the same sets of values (none at all), but they differ syntactically.

Up until Thursday this week, we allowed empty ranges. After days of painful debugging during the reworking of our Meadowcap implementation, we do not anymore.

The trouble started innocently, while I happily ported our implementation of the EncodeAreaInArea encoding. This encoding works with an inner range of timestamps which must be included in some outer range of timestamps — i.e., the inner range must describe a subset of the timestamps described by the outer range. The encoding encodes the start time of the included range by adding some positive number to the start of the outer range. This works great for non-empty ranges, but not when trying to encode the empty range [3..3) inside the outer range [5..8). Oops.

A simple and efficient fix would be to normalise empty ranges when encoding: when encoding an empty range within the range [start..end), you could always encode it as [start..start). The problem: we have strict notions of what constitutes a valid scheme for encoding values in Willow. In particular, equal values must yield equal encodings, and nonequal values must yield nonequal encodings. Normalising empty ranges during encoding would violate those basic (and sensible) axioms. We have mixed syntactic and semantic notions of how to compare ranges in the same encoding, and the result was pain.

In our old implementation of ranges in Rust we forbade empty ranges, which is why this issue never materialised in actual test failures. Frustratingly enough, most of the time spent in updating our implementation of all entry groupings was in making sure that empty groupings would be handled correctly. And now we had to flat-out remove all empty groupings from the spec, which meant I had to refactor all that code again, mostly undoing the finicky work I did two weeks prior. Diagnosing what was going on and then refactoring all groupings took me several (unpleasant) days. And that is why the revisited Meadowcap took so much longer than I expected.

I hope I learned my lesson and will pay closer attention to (implicit) notions of equality in the future. I was partially aware of this issue already; we have always defined only nonempty intersections of areas, for example. But for ranges I somehow hadn’t spotted the issue. On the bright side, our principled definitions of the semantics of both groupings and of encoding functions meant that we did catch this issue after all — in fact, it resulted in actual failure of a property test. This is exactly the kind of specification problem that has the potential to cause a whole lot of pain years down the road. So as much as the delayed Meadowcap release annoys me, I am glad we now fixed the issue.

~Aljoscha

2025Week 4721 Nov

“it’s only five functions”

A piece of paper with various drafts of worm-blossom logos, including the final one that made it into our site header.
A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

This week I made good on my threat to build an Interactive Willow Data Model. It's a toy version of Willow which lets you view and add entries and 'sync' between three 'peers'.

Is having something to play with going to be the missing piece of our documentation story? I don't know. It's made me want to write documentation which intersperses theory with lots and lots of mini-toys, but who has the time?

Most of the work on this was sunk into two spots:

  • Making it so you could get an instant preview of what your changes would look like before committing them. This was something Aljoscha considered a must-have after looking at the first version, and unfortunately I had to agree with him.
  • Squeezing a lot of (variably sized) information into a snug three column layout. Lots of overflow to deal with.

The most unsettling part of working on this by far was how powerful I felt working with TypeScript and (P)react again. After a year plus of working in Rust, I thought I'd be wringing my hands at the sheer barbarity of it all, but instead I was completely at home in the filth.

~sammy

A teeny megaphone blaring out noise

compact_u64 crate

We have released compact_u64, a crate implementing the variable-length encoding of unsigned 64-bit integers we use throughout the Willow specs.

use ufotofu::codec_prelude::*;
use compact_u64::*;

let mut buf = [0; 3];
let mut con = (&mut buf).into_consumer();

cu64_encode_standalone(258, &mut con).await?;
assert_eq!(buf, [0xfd, 1, 2]);

let n = cu64_decode_standalone(
  &mut producer::clone_from_slice(&buf[..]),
).await?;
assert_eq!(n, 258);

This release is an important step toward our implementation of Meadowcap. We had hoped to release the full Meadowcap implementation this week already, but turns out porting all the encodings it relies on simply takes too much time. By next week we will hopefully be done.

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

On Variable-Length Integer Encodings

Most competent programmers are aware that linked lists are inefficient data structures, simply because they do things which modern hardware struggles with. Having indiviual elements point out to the next element (if any) is much less efficient than simply grouping elements next to each other in memory and prefixing them with the number of elements. The opening paragraph of the wikipedia article on linked lists points this out. The documentation for linked lists in the Rust standard library points it out in CAPS.

Here is a hot take: LEB128 — the most widely-used variable length integer encoding — is the linked list of varints. The basic idea is to represent an integer as a sequence of bytes. The first seven bits of each byte encode information about the integer, and the final bit indicates whether the end of the encoding has been reached or not.

For computers, this format is a pain to decode. Modern hardware wants to operate on multiple bytes simultaneously whenever possible (even without SIMD, processing still happens at words of multiple bytes), but an LEB int must be processed byte by byte. And for every single byte there is a branch which is highly difficult to predict.

Analogous to data structures for sequences, a much better design is to provide the length up front and to then have the actual data in a contiguous chunk. CBOR got this right, I humbly offer the VarU64 spec and the way we encode integers in Willow (which is essentially a generalisation of the CBOR design), other people have proposed similar designs. The exact choice between any of these barely matters. But they are all significantly more efficient than LEB128. And as a bonus feature, comparing their encodings lexicographically coincides with numerically comparing the encoded integers, which can be helpful when working with kv-stores — but is not the case with LEB128.

For the most part, this does not really matter. Very few systems will critically suffer from the performance penalty implied by LEB128. I just find it curious that so many competent engineers simply accept the very first design they stumble upon. And where is the warning in the first paragraph of the Wikipedia page on LEB128? Why does that page only provide links to “related encodings” that replicate the same design choice, instead of discussing alternatives?


Oops, looks like this week turned into a week of rants for me. Oh well, going forward I will at least be able to link to these pieces instead of repeating the same rants over and over in different discussions.

~Aljoscha

A Rant From the Ivory Tower

Kids these days. Keep on yapping about CRDTs, as if they found the best thing since sliced bread. Even though everyone knows the best thing since sliced bread is actually blockchain the internet of things artificial intelligence service-oriented architecture!

To recapitulate, an operation is called commutative if the order of its arguments does not influence the result. Quite obviously, if you only use commutative operations, then the order of your inputs does not matter. What else is new? Sure, “operation-based CRDT” sounds moderately impressive, but all you are doing is to apply a commutative (and associative) binary operator.

Technically, when you also add idempotence, you get a semilattice (operation-based CRDTs and state-based CRDTS are technically equivalent, after all). But let’s face it, most operation-based CRDTs in the wild are ad-hoc commutative operators that do not derive value from any underlying algebraic structure.

Did you know that you can turn any non-commutative operator into a commutative version by consistently permuting its arguments? For example, while subtraction is not commutative, an operator that takes two numbers and always subtracts the smaller number from the larger one is commutative. You can also make any operation idempotent if you require the arguments to carry a unique id, and keep track which ids you already processed.

So when a data structure expects all operations to arrive in sequence and has to store them all forever, so that they can be deduplicated and sorted, so that they can be made idempotent and commutative, so that the whole thing becomes an operation-based CRDT, forgive me if I am not particularly impressed.

Now, state-based CRDTS, on the other hand — or semi-lattices as they have been called for decades prior — are great. They provide sufficient algebraic structure to be actually useful, allowing for universally-applicable optimisations such as delta CRDTs (that paper also provides a definition of the concept of irredundant join decompositions, which is central to most interesting CRDT questions). Here are a couple more ideas of what could be explored:

If your application state is a join-semilattice, then you can represent it by building a tree whose leaves are the elements of the irredundant join decomposition of your state, and whose internal nodes store both the join over all children of the subtree and a hash of that join. You could use that tree to power a tree-based set-reconciliation algorithm (whether based on history-independent trees or not), giving you logarithmic-time state updates and the ability to synchronise them efficiently. More generally speaking, generalising set-reconciliation to join-decomposition-reconciliation seems quite promising (see, e.g., arxiv.org/pdf/2505.01144).

How do irredundant join decompositions relate to information-theoretic bounds on synchronisation traffic in general, and to compression of CRDT updates in particular? Lattices admit a clean representation theory (see Davey and Priestley chapters five and eleven) in terms of irredundant decompositions. Thus, it should be possible to define encodings purely in terms of these abstract representations, which could ideally lead to provably optimal encodings of updates irrespective of the details of the particular lattice.

The CALM theorem states that conflict-free updates must monotonically progress in some partial order. This does not mean, however, that the amount of information you need to track has to increase monotonically as well. Willow provides an example, where a single newer (i.e., greater-in-the-CALM-relevant-order) entry can overwrite many older entries, thus deleting information. What does this look like from a more general lattice-theoretic perspective? When some join-irredundant element is in the irredundant join decomposition of some state, then no other element less than it can be part of that decomposition. In other words, join-irredundant elements “high up” in the partial order of the lattice enable deletion. Can we design for the existence of such join-irredundant elements? Which algebraic properties hold in lattices where we have upper bounds on the length of maximal chains which do not involve join-irredundant elements? Can we leverage such upper bounds for efficient encodings of updates? Do bounds on the largest anti-chains factor into this as well? What are good formalisms to quantify the “deleted information” of join-irredundant elements (a good starting point might be to define a a non-lossy lattice which carries around all elements which have been join-irredundant at any point; there should be a natural lattice-homomorphism from any lattice into its non-lossy counterpart that would serve to compare the two at any point in the original order)?

In a network where multiple peers synchronise state via a CRDT, which optimisations can we enable if we get to assume that peers cannot lag too far behind? How do we properly define that in the first place — maximum distance between greatest and least state when interpreting the partial order as a directed graph? And once we have a suitable notion of such bounds, how can we automatically determine which information we can now drop? In other words: whereas join-irredundant decompositions are defined in terms of the full lattice, which meaningful (and practically useful) notions of decompositions can we enable by strategically cutting off elements from the bottom of the lattice (here, it can be really useful to work with semi-lattices instead of proper lattices, because in a join-semilattice you can remove minimal elements without jeopardising the semi-lattice axioms)? The braid folks have been exploring similar thoughts in their antimatter algorithm (and in general, they have been doing some nice abstract CRDT work, see, for example, their time-machine theory for unifying Operational Transformation and CRDTs).

How come barely anyone is talking about sublattices? A lattice within a lattice is exactly what you need in order to support fine-grained access control while still keeping things synchronisable. Same for setups where peers are not interest in all the data ever but merely in a subset of the data. If that subset forms a sublattice, then you know that you can synchronise that data with confidence. It is no coincidence that access control and partial sync work out really neatly in Willow: every Willow Area induces a sublattice. Since, arbitrary lattices can have exponentially many sublattices, formalisms that assume that arbitrary sublattices can be expressed are probably inapplicable in practice. But when restricting the set of sublattices that can be expressed (like Willow restricts access control to the granularity of Areas), what makes for a good set of sublattices? You probably want some closure properties, maybe something filter-like? I don’t know, I’m not even a mathematician, I just want more people to ask these questions! Because it feels scary to merely intuit that Areas are a good choice for Willow without knowing why.

Can we formalise a notion of intent-preserving lattices? When a text CRDT merges two overlapping edits in a deterministic but possibly unintuitive way, the intent of the author of either edit might be violated (and possibly even the intent of both authors). Is there a formal concept behind this human-centric notion? Staying with this example, instead of doing an arbitrary resolution, the join of the two edits could instead be a state which preserves both edits (“intents”?), at the cost of not mapping to a unique string of text. As long as such states can still be merged (accumulating ever more conflicting intents), you still have a CRDT, after all. That is basically what the Pijul vcs does. Again, can we formalise what it means that such a system preserves intents? Perhaps the existence of certain homomorphisms into the non-lossy lattice version sketched a few paragraphs earlier might be meaningful here?


All of these thoughts are merely starting points and semi-low-hanging fruits, but all of them have the potential to bring about actual engineering benefits for distributed systems. So I am all for CRDTs. But it feels to me like an increasing fraction of the usage of the CRDT terminology refers to ad-hoc variations of the brute-force approach of making arbitrary operations commute by applying updates in a consistent order (i.e., by sorting their arguments). Of course it is nice if a bunch of local-first applications speed up because somebody made a“JSON-CRDT” 10% faster. But that is not how we will make any qualitative progress on the road to ubiquitous peer-to-peer software. As uncomfortable as it may be, that will require actual math, not just repeating “CRDT” often enough to turn it into a buzzword.

2025Week 4614 Nov

“more than 80 bytes per week”

A four-panel comic.
      
      In the first panel, we are zoomed in on a hand holding a phone. On the screen there is a social media post reading "i've recently seen 3 nlnet funded projects all write their own static site generator for their own documentation. nerds with no oversight are just gonna nerd".
      
      In the second panel, we see Aljoscha and Sammy, both looking at the phone held in Sammy's hand.
      
      In the third panel, they look up at each other.
      
      In the fourth panel, Aljoscha say: "Technically our system can generate any kind of string". "Stop", say Sammy.

what's with the name?!

why is this website called ‘worm-blossom’? In this series we will explain why in a concise manner.

Part 2: Swatch Internet Time

It is 1998. Two centuries after the French revolutionaries tried (and failed) to metricise time, Nicolas Hayek, the CEO of Swatch, and Nicholas Negroponte, director of the MIT Media Lab share the stage at the MIT Junior Summit ‘98 to jointly announce “Swatch Internet Time”, which divides the day into 1000 beats.

Hayek is a businessman who has successfully reordered ailing Swiss watchmakers into the Swatch Group. It is safe to assume that for him, Internet Time is just business, a new marketing campaign. But Negroponte is a true believer: “Internet Time is absolute time for everybody. Internet Time is not geopolitical. It is global. In the future, for many people, real time will be Internet Time," says Negroponte, an unabashed techno-optimist who regularly asks readers of his column in Wired to “move bits, not atoms”.

He is not alone in his belief that the internet will bring about a liberal utopia. And perhaps that is why for a minute (or a beat) it looked like internet time might actually happen. Watches flashing the time in beats flew off the shelves in Japan. CNN displayed the time in beats in the corner of their news feed, 24/7. Ericsson phones displayed internet time. ICQ, a then popular instant messenger used it too.

But after the initial fervour, Swatch Internet fell into disuse, just as it had two centuries prior. But unlike France's decimal time where the fifth hour corresponded with noon, 500 beats could mean anything depending on where you were in the world. 500 beats could mean noon — if you were in Biel, where Swatch’s headquarters are situated — or it could mean morning, or midnight elsewhere. Swatch Internet Time asked us to abandon the real world as our frame of reference and replace it with the internet instead.

But why is worm-blossom called worm-blossom? If we turn the camera away from the stage at Junior Summit ‘98 and zoom into the audience, we can see Isao Okawa, the then chairman of Sega Enterprises. We’ll meet him again in our next instalment.

A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

worm-blossom: now on Fridays. We have moved the publishing day because having a Monday deadline put a lot of pressure on the weekend. On Thursdays I'd think I have so much time to write about the French revolution, handily forgetting that I never have the opportunity to use a computer on the weekends. Then Monday would turn into a very stressful day. Now Fridays can be a very stressful day instead, which is much better.

We sadly had our proposal for Rummager rejected this week. It's one of those ideas that is very hard to convey, and I'm starting to think we'll just need to build out a prototype of it in our spare time to be able to do that.

It's a bit similar to something else I noticed this week: no matter how much documentation we write — loaded with illustrations and references and sidenotes — you just can't build up the same kind of understanding as if you actually got your hands on it. I'm thinking of building out some interactive tools for the Willow Data Model page because of that.

In any case, we have no shortage of (funded) plans. Keep muddling through out there.

~sammy

We have opinions

...on 'fancy' CRDTs

Willow uses a last-write-wins strategy for resolving conflicts. With so many sophisticated alternatives out there, why would we do that?

Systems which automagically merge data tend to fall into two camps:

  • Good at merging self-contained units of data (e.g. merging multiple edits of text into a single coherent edit)
  • Good at merging collections of data (e.g. merging separate timelines of posts into a single timeline with everyone’s posts in it)

Intelligently merging self-contained data like text is undoubtedly an impressive technical (and scholarly) feat! But most of the connected applications we use day-to-day don’t need it: microblogging, chatrooms, forums, messaging, media sharing, issue trackers. All applications where people author data alone before contributing it to a common pool of data.

Fine-grained CRDTs have a hard limit on how much they can be optimised. And they often require lugging around a complete history of changes, forever.

Which is why Willow focuses on merging potentially huge collections of interleaved data. You get a system which can serve a broad spectrum of connected applications, and one which we know we can make efficient.

A teeny megaphone blaring out noise

Ufotofu-Based Encoding and Decoding

We have released a new minor version of ufotofu, adding traits and helpers for encoding and decoding values.

use ufotofu::codec_prelude::*;
use ufotofu::codec::endian::U32BE;

let mut buf = [99; 5];
let mut con = (&mut buf).into_consumer();

// Encode a u32 into big-endian bytes...
U32BE(258).encode(&mut con).await?;

assert_eq!(buf, [0, 0, 1, 2, 99]);

// ...and decode them back into a u32.
let mut pro = buf.into_producer();
assert_eq!(
  U32BE::decode_canonic(&mut pro).await?.0,
  258,
);

In addition to traits for conventional encoding and decoding, we also have a module for encoding and decoding relative to some context shared between encoder and decoder. These traits allow for techniques such as delta encodings. Many of the encodings of Willow rely on this to encode information significantly more compactly than if everything was encoded in isolation.

In general, the codec traits of ufotofu map directly to the notions of (relative) encoding relations we use throughout the Willow specs. This does not mean that ufotofu takes on an opinionated formalism applicable only to Willow. On the contrary, it simply means that the concepts on which we build encodings in Willow are useful general-purpose abstractions.

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

The codec traits we have added to ufotofu today have been part of older ufotofu versions for quite a while now. And they have been exceptionally useful throughout our Willow-in-Rust journey. In fact, they alone already justified ufotofu as a whole for us. What makes them so special?

Not just any value that can be turned into a bytestring gets to implement our Encodable trait. No, there are precise formal requirements that must be fulfilled by all possible encodings. Every possible value must have at least one encoding. No two nonequal values must have the same encoding. No valid encoding must be a prefix of any other encoding. That sort of thing (see the docs for the precise list). But still, how is that supposed to be incredibly productivity-boosting?

The answer lies in the humble codec::proptest module. It provides functions such as assert_codec. You call that function with a bunch of values, and the function panics if and only if the arguments are a counterexample to any of the formal criteria an encoding must fulfil.

We use this to let a coverage-guided fuzzer try to find counterexamples. And if it cannot, then we can be pretty darn confident that our implementation is correct. Which only few of our initial implementation attempts are. But we just press a button, and the computer gives us a set of failing inputs. No matter how many encodings we implement (and Willow alone has quite a few), we always get an almost ideal test setup and debugging aid for free.

And even better, this process has uncovered plenty of faulty definitions of encodings in the Willow specs as well. Some of our encodings are quite fiddly, and it is easy to lose track of some detail and specify an encoding which cannot be uniquely decoded, for example. But if the fuzzer cannot find a counterexample after 20 minutes, we can be highly confident that the encoding we specified is free of bugs.

Now that we have properly released ufotofu and its tooling around encodings, we will return to the Rust implementation of Willow proper. If everything goes according to plan, we will release a cleaned-up, well-documented, and ergonomic-to-use implementation of Meadowcap next week. And internally, it will be powered by ufos and tofu!

~Aljoscha

2025Week 4510 Nov

“Do geese see god?”

2025Week 4510 Nov

A teeny megaphone blaring out noise

ufotofu

We have released the ufotofu crate, a Rust library for working with streams and sinks, which will power our Rust implementations of Willow. Because ufotofu will be a quite foundational building block, we put a lot of effort into documentation, including a somewhat over-the-top website. Please check it out!

Rust already has a de-facto standard library for async sequence processing, so we do not necessarily expect wide adoption for ufotofu. But we believe that some of its designs are genuine improvements over many libraries out there across all programming languages. So we hope that awareness of our little crate will spread, and that it will influence the design of other libraries in this domain. The ufotofu website goes into all design choices in great detail, and is interesting not only to Rust programmers but to API designers in any programming language.

Special thanks go to Glyph, for suggesting to use a palindrome as the crate name, and for early implementation work.

Have you opened the ufotofu website yet?

A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

Last week I was chasing funds for the next generation of worm-blossom projects. One of these proposals is a general-purpose GUI application called Rummager. In our original application I breathlessly wrote about "organically expressing your own applications through the act of nesting free-form data". But what does that actually mean, you (and potential funders) ask? A picture's worth a thousand words, and I've combined several into a single image below which is a lot of economy.

We also put together the new ufotofu site. Ufotofu provides a fairly low-level abstraction, and I think we'd be forgiven for providing a link to a README and calling it a day. But to be earnest for a minute: right now we have funding which gives us a lot of freedom. We probably won't have that funding forever. When it's over, I want to look back at what we did with this opportunity and be satisfied we did our best.

That means we want to describe ufotofu really well. We want to make that description pleasant to read. And we want to add touches that make a reader want to stick around. Those final touches may seem frivolous (we made it so that when you scroll it looks like a UFO is beaming up stuff), but I think (hope) readers see frivolity and recognise it as care.

~sammy

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

I got into distributed systems through Secure Scuttlebutt (SSB). A lot of what happened during my first years in that community has stayed with me until today. In many cases, that is a good thing. In other cases, things are more debatable. My stance on libraries for streams and sinks might be one of those latter cases.

SSB was created by Dominic Tarr. He was well-known in the early node-js community, amongst other things for his pull-streams library. Node-js had its own set of stream abstractions, but Dominic judged he would be more productive if he built SSB on a custom library. And he was probably right. In the early years, people where pretty happy to code against a sane API, it was only years later that the non-standard streaming API became an obstacle for new contributors.

Then, there was André Staltz, who had created cycle.js and later also xstream and callbag. And when Cryptix and Keks implemented SSB in Go, the first thing they did was to build a custom streams library. Patchwork was built on a custom UI framework, built on custom observables.

So for some reason, I ended up absorbing knowledge from a bunch of people who were not afraid of rolling and re-rolling their own stream libraries. And over the years (starting, probably, around 2017), I began forming my own opinions on this matter. At the last few iterations of p2p-basel, I always ended up frantically scribbling on whiteboards to convey the isomorphism between a good stream abstraction and regular grammars. When we started Rust work for Willow, I somehow managed to convince Sam to use part of that funding for writing a new library of stream abstractions in Rust. We also managed to convince NLNet to fund this work. I submitted an essay about stream APIs to Onward! 2024 (which got rejected, but with some very helpful feedback), which has formed the basis of the ufotofu webpage.

I’m rambling. Ufotofu has been simmering for almost ten years now. Over that time I have changed as a person, as a computer scientist, as an engineer. The designs of ufotofu have also changed, but the core principles have always remained the same. I think what I am trying to say is that I am proud of this thing.

I’ve seen many people criticise Dominic for implementing SSB on top of a custom streaming abstraction. Calling it inaccessible, accusing him of not-invented-here syndrome. But I think Dominic made a pragmatic choice: he built his own tools, because he could make them better than the tools that were already around, and the task ahead of him was sufficiently large to justify the initial overhead. We are limited by the quality of our tools, after all. And I feel similarly about ufotofu. Which is why I am ignoring the lessons I perhaps should have learned from SSB. Willow in Rust is implemented on ufotofu, and as of now, I fully believe that to be the right choice.

~Aljoscha

2025Week 4403 Nov

“existence is inefficient”

what's with the name?!

why is this website called ‘worm-blossom’? In this series we will explain why in a concise manner.

Part 1: The French Revolution

France, 1792. The monarchy has been abolished and the French Republic established. Anything seems possible, and everything is up for reconsideration. After all, if you can overthrow a centuries-spanning monarchy, what can’t you do?

Revolutionary ideals, scientific vigour, and a bureaucratic need to govern a fledgling republic have brought about a metric revolution. Variable, localised, and unpredictable systems of weight and measure are being replaced by the gram and metre. These easily divisible units lubricate France’s scientific, military, political and economic progress.

Could the same by done for time? Jean-Charles de Borda, a military engineer and scientist thought so. By dividing the day into ten hours, the hour into 100 minutes, and each minute into 100 seconds, he argued that timekeeping itself could be revolutionised.

By 1793, the National Convention had formally adopted this system. Decimal time was used in all public records, and a number of decimal clocks were prominently displayed in a few French cities.

But mandatory use of decimal time was suspended just two years later. It didn’t seem to offer much of a benefit, nobody was going to enforce its use, and a lot of people felt that a decimal hour (equivalent to 2.4 regular hours) was just too long for an hour. Not to mention how annoying it was going to be to replace every watch and clock in France.

Decimal time would remain largely forgotten for another 203 years, which is where this series will be continued. See you next time!

A teeny megaphone blaring out noise

New willow25 Crate Version

We have released version 0.2.0 of the willow25 crate. This release adds the groupings module, which implements our groupings specification (who would have thought!)

There is a type for each kind of grouping: WillowRange for ranges, Range3d for 3d ranges, Area for areas, and AreaOfInterest for areas of interest.

As with the previous release, we tried to make working with these groupings as convenient as possible. For example, you can use the same method on entries to check for membership in any kind of grouping (a marvel powered by the Grouping trait):

use willow25::prelude::*;

let namespace_id = NamespaceId::from([0; 32]);
let subspace_id = SubspaceId::from([1; 32]);

let entry = Entry::builder()
    .namespace_id(namespace_id.clone())
    .subspace_id(subspace_id)
    .path(path!("/blog/ideas"))
    .timestamp(1793)
    .payload(b"Temps décimal! Très bien!")
    .build().unwrap();

// Range creation from arbitrary Rust ranges!
let time_range = TimeRange::from(..=2025.into());
assert!(entry.is_in(&time_range));

let area = Area::new(
    None, path!("/blog"), time_range,
);
assert!(entry.is_in(&area));

See the new groupings tutorial for more examples of what these APIs feel like, or read the API docs.

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

I accidentally published an incredibly nerdy but surprisingly useful Rust crate this week: order_theory.

The Willow specs often refer to minimal and maximal values (such as SubspaceIds or Paths). Sometimes we also need a notion of successors on these types. So I defined some traits capturing these concepts. And then I moved them into their own crate, because I needed them in several places. And since you cannot very well publish a trait claiming to provide order-theoretic concepts without traits for lattices, I added a corresponding hierarchy of traits as well.

Then I needed our NamespaceId and SubspaceId types to implement SuccessorExceptForGreatest. So I added a generic implementation of all order-related concepts of our new crate for arbitrary arrays, and simply delegated to those from willow25.

Next, I wanted to convert arbitrary Rust ranges (which feature both inclusive and exclusive bounds) to Willow ranges (which are less flexible). Turns out that you can do those conversions exactly if the type in question is totally ordered and has a notion of predecessors and successors. Good thing I had recently defined traits for exactly that!

All Willow groupings support a notion of intersections — the intersection of two groupings is the grouping which contains exactly the entries contained in both original groupings. Why, that sure sounds like a LowerSemilattice. How convenient I had a trait for that concept.

Finally, implementing intersections (and other order-related methods) is not always easy. So I added utilities for property testing to the order_theory crate. And these then found several errors in our previous Willow groupings implementations.

Math! Yay!

~Aljoscha

A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

Not much to report this week as I was travelling. The coming week is going to be busy as November is here, and 2026 is starting to rear its fuzzy little head. I think it's going to be a good year, but only if I write all the proposals and applications like I'm supposed to.

~sammy

2025Week 4327 Oct

“ideally, do the max”

A comic.
      
In the first panel, Anubis, Egyptian God of the Afterlife, addresses a man: "Imhotep! Do you bring anything with you into the afterlife?"

In the second panel Imhotep holds his arms out wide and declares: "Only my retinue of 50 servants, my lord". Said servants are in the background looking unhappy. Anubis replies: "Very well".

In the third panel, Anubis, Egyptian God of the Afterlife, addresses Aljoscha: "Aljoscha! Do you bring anything with you into the afterlife?" 

In the fourth panel Aljoscha holds his arms out wide and declares: "Only my 50 unreleased programming projects!". Behind him, unhappy spectres beg for release.

In the fifth panel, Anubis grimaces as Aljoscha further explains: "I only need to write the docs".
A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

The Big Giant Willow Website Update

willowprotocol.org, the online home of our peer-to-peer storage protocols, has been updated to reflect all of the work that we’ve done on Willow from April 2024 until now (work funded by our friends at NLnet! Hartelijk dank!). That’s eighteen months of work in a single serving! Argh!!! So what are the highlights?

Firstly, we’ve reworked Confidential Sync (neé W.G.P.S) to be man-in-the-middle attack resistant. In the old protocol, if two peers who did not know each other before were to sync over a compromised network (e.g. a network with a router modified to intercept traffic), an attacker would have been able to access all sorts of Willow-stored data! As Confidential Sync is expressly designed for secure syncing between untrusted peers, we've redesigned Private Interest Overlap to mitigate this kind of threat. You can learn more about the security properties and threat model of this design here.

The Drop Format (neé Sideloading protocol), which encodes a user-selected subset of Willow data into a single binary (ready to put on a USB key or send via Signal), now is able to add entries without their payload. We’ve also made the encoding more efficient so you can squeeze even more Willow data into a smaller number of bytes.

We’ve added a new page for Willow‘25, a recommended set of efficient, secure parameters for all the Willow specs. This way, nobody needs to witness the nightmare of generic types we’ve created for ourselves unless that’s what they really want.

There’s a new Rust section which has tutorials for the crates we’ve released so far. Not only does this section have a completely gratuitous illustration of Willowverse mascots Alfie and Betty riding a giant crab, but we’ve also integrated rustdoc support into willowprotocol.org’s referencing system. This means that our tutorials can reference things like structs and their methods, and link readers directly to up-to-date Rust API documentation. We will be regularly updating this section over the coming weeks and months!

There are new illustrations, diagrams, and comics pretty much everywhere. The Drop Format spec. The Private Interest Overlap Page.Encrypting Entries. The ‘More’ section. The About us page.

Another page which has lots of new illustrations is the front page, which we completely redesigned to make answering the question of “what is Willow?” a bit easier. It also has a big fun illustration for desktop users.

All of this work got stuck in a kind of mega branch comprised of 67,020 additions and 14,590 deletions over 330 commits with the sole description of ‘do not merge yet, wip’. No comment.

~sammy

Shout-out!

Many thanks to Miaourt for their contributions to Bab — which in turn made it into today’s willow25 crate release! Miaourt and I (Aljoscha speaking) did a pair coding session on Friday, which was quite fun. And a good reminder that I am merely an academic dabbling in coding, whereas there are people who have a lot more practice and are actually good at it. I was really impressed with how quickly they grasped everything, and they made several choices that resulted in much cleaner code than what I would have written.

A teeny megaphone blaring out noise

willow25 Crate Release

We have released the first version of our Rust implementation of Willow’25. This initial release implements the full Willow Data Model, with the parameter choices recommended in the Willow’25 spec.

Highlights include

  • an optimised immutable Path struct with zero-copy prefix creation,
  • a Timestamp struct allowing for convenient math such as some_timestamp - 3.seconds(), and
  • the Entry struct which will sit at the core of all Rust Willow applications.

We put a lot of care into making everything as ergonomic as possible. Look at the following snippet — isn’t it neat? We think it is neat!

use willow25::prelude::*;

// Create an entry for a specific payload,
// timestamped with the current time.
let entry = Entry::builder()
    .namespace_id([0; 32].into())
    .subspace_id([1; 32].into())
    .path(path!("/vacation/plan"))
    .now().unwrap()
    .payload(b"See many sights!")
    .build().unwrap();

// Create a new entry based on the previous
// one, with greater timestamp and a new payload.
let updated = Entry::prefilled_builder(&entry)
    .timestamp(entry.timestamp() + 5.minutes())
    .payload(b"See *all* the sights!")
    .build().unwrap();

// Assert that the new entry would overwrite
// the old entry if both were inserted in a store.
assert!(updated.prunes(&entry));

Accompanying this release is a brand-new Rust section on the Willow website with tutorials for our Willow crates. Check it out, even if only for the fun illustration!

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

There might exist gifted programmers who can write non-trivial code and get things right the first time. I am not among them. But still, there is no way around implementing everything for a first time.

Over the past year, we have implemented most everything we specify on the Willow website in Rust. For the first time. Technically, that code does what it is supposed to do. But a lot of it is cumbersome to use, and there are several structural problems I really don’t want to get into (here). Long story short: we are reworking that code, and re-releasing it over the coming weeks. And from what we can tell so far, the new code is pretty enjoyable to work with!

We started this week by releasing an implementation of the Willow data model. Here is the plan for what will follow:

  • Groupings (a requirement for any non-trivial Willow usage),
  • Meadowcap (at that point, you can start sketching out access-control for Willow-based applications),
  • traits for (persistent) storage for entries and payloads (at that point, you can write full application logic against the traits),
  • simple in-memory and persistent implementations of the store traits (at that point, you can write and run full offline-only applications), and
  • the Drop Format (at that point, you can write and run full sneakernet applications).

Once these are done, we will probably slow down a bit, and play around with our own application ideas, end-to-end encryption, network-based synchronisation, and anything else that might come up. In addition to working on Bab, Ufotofu, and Macromania, that is.

~Aljoscha

Shout-out!

Some weeks ago, Joakim, emailed us to point out that UNIX timestamps have less than great behaviour when it comes to leap seconds, and recommended International Atomic Time (TAI) as a better alternative. We have indeed updated the recommended interpretation of the timestamp in Willow to be a TAI timestamp.

And then over the past days, Joakim supplied us with many valuable details around the specifics of choosing a reference epoch (i.e., “which instant does the timestamp zero denote”), saving us from making a non-standard choice due to some confusing (and possibly buggy) documentation and code in the Rust hifitime crate. This might even help those crate developers to fix some independend bugs of their own. Quite a journey — thank you for your help!

2025Week 4220 Oct

“visceral ambivalence”

A comic with four panels. 
        
The first panel's text reads: "When it feels like you're getting nowhere, it can be tempting to try and progress through sheer force of will". Underneath in a drawing of a figure staring down at an empty plant pot, waves radiating from his head.
      
The second panel's text reads: "But most of the time the best thing to do is just back off and let the solution come to you". Below is a drawing of the same scene, except the figure is sunning on the ground in the background, and there is a tiny sprout in the plant pot.

The third panel reads: "Aljoscha often tells me he's going for a walk. When he gets back, he's figured out some novel data structure or mapped out the possibility space of some given premise." Above is drawn a kind of fish-eye lens drawing of Aljoscha walking through the park, the sun shining down on trees and flowers, abstract shapes radiating from his head.

The fourth panel reads: "I have ideas also". Below Sammy is shown showering with a speech bubble above her head. Inside, it shows the phrase "cloud storage" with the word cloud crossed out and replaced with 'crowd'. Sammy smiles with satisfaction and is muttering "genius...".

The Surfers’ Review

“fantastic site wow!!”
“Always excited to see what yourself and Aljoscha are up to, often outside my ability to understand but always filled with such an incredible feeling of hope”
“The humor and art work are also very pleasant.”
~ moid

We thank you for your warm reception to the first instalment of worm-blossom.org.

A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

This week I put a lot of time in non-willow projects. The biggest timesink was ufotofu, which is a library for working with lazy sequences (“sinks” and “streams”) in Rust. Specifically, I wrote tons of documentation, tests, and documentation tests in preparation for a proper release. This crate has served us well for our async needs while implementing Willow in Rust, and I am giddy about getting to share it more widely soon!

I also put some time into Macromania, our typescript-embedded templating language we use to generate willowprotocol.org in all its hyperlinked and tooltippy glory. We have accumulated a couple of breaking changes we wanted to do to the core of Macromania, and those are now all implemented. Sadly, this means that all the fancy macros powering tooltips etc do not work with that new version of Macromania.

As for the current state of Macromania: the core library is now stable and has pretty solid doc comments (the curious can look at the jsr release), but we still lack high-level documentation for onboarding folks into the madness. We will start working on a proper website featuring this high-level documentation soon-ish (for me, ufotofu has higher priority at the moment). That website might turn out fairly unimpressive initially, because we still lack all the really powerful macros — not only did we break their foundation, but we have also accumulated a bunch of breaking improvements we want to roll out for most of our core macros. Nevertheless, the new version of Macromania already builds this website, and does some fun stuff such as generating an RSS feed and highlighting code snippets. See the source and tinker around if you are feeling adventurous!

This was a highly productive week from my side for ufotofu and Macromania, yet I feel twinges of guilt. Both of these projects are fairly computer-centric, not human-centric. Well, ufotofu in particular; it tries to apply some mathematical principles to streamline a programming experience. Programming for programming’s sake, as if the world could seriously be improved by making computers and programming more efficient and thus even more entrenched... But still, I cannot deny that this kind of work is extremely fun and satisfying to me. In weeks like this, I carry with me some cognitive dissonance centered around escapism and indulgence, counteracted by telling myself that all of this keeps me motivated and will ultimately further Willow (not only by keeping me engaged, but also because our Willow implementations rely on ufotofu and our web presence relies on Macromania).

I want to end on a more light-hearted note, so, erm, have some borderline hyperactive sketches for two pianos (loosely inspired by listening to some Meredith Monk pieces):

Also, *how* is “instalment” correct spelling? ~_~

~Aljoscha

A teeny megaphone blaring out noise

Frugal Async Rust

A whole lot of the Rust code we write for Willow is async, and async Rust comes with some pitfalls. We have decided to make some simplifying assumptions to avoid the worst stumbling blocks. And those assumptions, we have now put on a webpage for others to reference. Frugal Async Rust describes how we forego certain features but in exchange avoid whole worlds of pain.

And we have released a frugal_async crate with some basic utilities adhering to the frugal async principles, for example, an awaitable Mutex:

use frugal_async::mutex::*;
use core::ops::{Deref, DerefMut};
use core::time::Duration;
use smol::{block_on, Timer};

let m = Mutex::new(0);
block_on(futures::future::join(async {
    let mut handle = m.write().await;
    // Simulate doing some work.
    Timer::after(Duration::from_millis(50)).await;
    assert_eq!(0, *handle.deref());
    *handle.deref_mut() = 1;
}, async {
    // This future is "faster", but has to wait
    // for the "work" performed by the first one.
    let mut handle = m.write().await;
    assert_eq!(1, *handle.deref());
}));

We encourage everyone who enjoys the details of async Rust to look at this issue around creating a macro for repeatedly calling several async methods in a loop and acting on the results as they become available.

A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

This week I have been mostly single parenting, so I did not get much programming done. Most of my creative energies went towards cooking meals.

For breakfast I often turn to the humble porridge. I like to put big chunks of apple in my porridge which cook just long enough to get soft but still keep their structure, kind of like in an apple pie. I like to cook my porridge low and slow, which really annoys everyone else.

You can't go far wrong with a curry. I break up a cauliflower into little florets and roast them in the oven until they start to caramelise at the edges. I could eat them just like that, but in a curry sauce they're even better. I make a curry sauce from scratch and it is completely inauthentic.

Salmon is great when you're low on time because you just smack it in the oven for a bit. In the colder months I like to do a roast vegetable mix alongside it: potato, beetroot, sweet potato, red onion, topped with some capers. The purple of the beetroot and onions look so good next to everything else.

And when it all became too much for me I just ordered pizza. hashtag winning

~sammy

And then, inspiration struck...

2025Week 4113 Oct

“we've done the first 99%, now we only need to do the other 99%”

Spindly tunnels wind down deep underground, branching off to aborted ends, and at the very bottom, a cavern with two twits inside. One asks the other: Why are we down here, anyway?
A drawing of Aljoscha, fingers interlaced, eager to deliver some interesting news.

A New Era Dawns

For the past year, we have been pretty busy here at worm-blossom. We have also been pretty quiet. But that is about to change. From now on, you can expect weekly updates about our progress on this very website. As you can probably tell already, we will restrict our reports to the minimal technical details, the bare necessities, the austere metrics of productivity. Nothing else.

Throughout our work on Willow, we increasingly had to take on technical debt in order to meet deadlines. Combined with our inclination of keeping work to ourselves until it is just perfect, we ended up accumulating a vast body of work we stayed fairly quiet about. Not only did this make us look stagnant, it also made us unhappy. But now, with our previous grant obligations fulfilled, and the next grant hopefully leaving us more breathing room, we decided to take action. Going forward, we will

  • identify minimally useful subsets of everything we do and publish these as early as possible,
  • apply this policy not only to future work, but also to our backlog of pretty cool accumulated things,
  • emphasise the exuberance and creative spark that drives us to do our work,
  • invite and mentor contributions to our projects, and
  • share our progress in the most joyful way we can on this website.

And what better way of holding us accountable to these lofty resolutions than to boldly and pinkly announce them on the world wide web?

~Aljoscha

A teeny megaphone blaring out noise

bab_rs Crate Release

We have released the first version of our Rust implementation of Bab. Bab is a family of hash functions inspired by BLAKE3, specifically designed for content-addressed storage systems. Read more about Bab here.

The bab_rs crate provides both a fully generic implementation (the Bab specification is parameterised over choices such as digest sizes) and a ready-to-use implementation of WILLIAM3 (our recommended default parameterisation).

While we have plans — and funding — for some sophisticated features around verified streaming, for now the crate provides the basic API you would expect for incremental hashing: you create a Hasher, repeatedly feed it with bytestrings, and eventually call finish to obtain a digest.

use bab::Hasher;

// Create a hasher.
let mut hasher = Hasher::new();

// Feed data into the hasher.
hasher.write(&[0, 1, 2]);
hasher.write(&[3, 4]);

// Obtain the WILLIAM3 digest.
let digest1 = hasher.finish();

assert_eq(digest1, [
    57, 194, 179, 8, 66, 14, 197, 192,
    5, 56, 128, 49, 249, 78, 60, 119,
    231, 114, 96, 94, 138, 132, 41, 172,
    54, 42, 99, 229, 59, 43, 122, 225,
]);
// Yay =)
A drawing of Sammy, arms crossed with a pencil tucked behind her ear.

This week we renamed two Willow specs: Willow General Purpose Sync and the Sideloading Protocol. I think it's important to sit up and notice when every time someone hears the name of your project, they say "like GPS??? Huh???". And at this point the term 'sideloading' has such a toxic association with locked down mobile platforms that it felt like a completely unforced error.

We renamed them to Confidential Sync and Drop Format, respectively. Not only are these nicer names, but it also meant I could redo the emblem for the WGPS.

A photograph of some sketches of the new Confidential Sync emblem

The WGPS was the only spec which didn't have a nature-inspired emblem, and this has now been remedied. I also remedied the total absence of pink from the lineup. It's a grapevine, because with confidential sync you "heard it through the grapevine". Isn't that precious?

~sammy