Building a secure key/value store

I’m hacking on Crochet again and this week I’m writing a small key/value store, so it seemed like a good opportunity to talk about some of the design approaches I use.

The key/value store is: global (every piece of code can see/modify it), but securely partitioned (no piece of code can see/modify keys outside of its partition), and has both pluggable storage backends and pluggable serialisation.

A few additional constraints: all of these features should work seamlessly (without users’ having to fiddle with configurations) across different runtimes (Crochet targets Node, Electron, and Web Browsers currently), and they must all be trivially testable without users having to care about complex dependency injection configurations. The system should also be debuggable without requiring significant effort from either the package developers or users.

A lot of these would be a significant undertaking in many modern languages. The initial pull request that implements it in Crochet is about ~800 lines of code, of which more than 50% is documentation (the one in this article is ~130 lines, including comments and whitespace).

So, what exactly goes into fulfilling all of these constraints?

OOP is great: Aliens

This is the second article in the “OOP is great” series, where we look at some of the features in OOPLs that I’ve adopted in Crochet, and what makes them great. Much of it might go against what you believe about OOP if you’re only familiar with languages like Java or C#.

Last time we looked into dispatch and multi-methods, what they’re useful for and why they were chosen for Crochet in particular.

This time we’ll look into a much less popular concept, but one that is widely applicable and has some very surprising connections to very recent advancements in type systems. Today we’ll talk about Aliens.

Aliens are a mechanism for Foreign Function Interfaces (FFIs) that, to the best of my knowledge, appeared in the Smalltalk community. But which also has interesting overlaps with transparent actors, membranes, interleaved interpreters, and linking types.

Incidentally, what all these jargon words have in common is really just one thing: interacting with foreign concepts—concepts that have, possibly, different semantics than the ones in your cozy programming language.

OOP is great: Multi-methods

This is the first article in the “OOP is great” series, where we look at some of the features in OOPLs that I’ve adopted in Crochet, and what makes them great. Much of it might go against what you believe about OOP if you’re only familiar with languages like Java or C#.

I have previously written a not so short article on some very unfortunate things about OOP, both as a programming philosophy and an umbrella description for features, that I’ve found during the years I spent researching programming languages.

This series will instead look into the positive things I’ve found during that time, and how these things shaped Crochet into what it is today. Features are neither “good” nor “bad” in isolation—I don’t think that kind of judgement is useful. So this series will look into these features in the context of Crochet, with a bit of more general reasoning here and there. Just keep that in mind.

This time we’ll look into Multi-methods: sets of functions that are selected based on more than one of its arguments. Though it hasn’t been adopted by many mainstream languages, you might have seen it in Julia and Clojure. They also appear in languages that are less popular, but feel “closer” to what most people think about when they hear “OOP”, like Slate, Dylan and Cecil.

Cecil’s implementation of this concept, in particular, is a huge inspiration for Crochet, and we’ll discuss more about it further down.