What is Tonk Core
Core represents the heart of our new architecture:
- A single WASM binary encapsulating a networked, CRDT-based virtual file system
- A new .tonk bundle format
- A web bootloader for executing Tonk bundles
Core WASM
The Core WASM package is built around the Automerge Rust library and new, experimental Rust-based Automerge repo called samod.
Core uses CRDTs to maintain a virtual file system where each file is backed by an Automerge document. This includes the metadata about the file system itself. It's all CRDTs all the way down.
Our WASM is initialized through a bundle format which carries all the necessary information to both initialize the network connections, the environment and the filesystem.
Moving forward, we intend to keep all the logic in the WASM binary to allow for maximum portability. We want Core to be able to run on everything from your phone to your browser to your raspberry pi.
Tonk Bundles
Tonk bundles are a zip format containing a) Automerge binary storage format, and b) a manifest file. That's it! For now, these bundles are given the .tonk extension.
The manifest file encodes which Automerge document to mount as the root and a list of network endpoints (with adapter types) that it will use to find syncing peers.
With these two bits of information we can parse out of the Tonk bundle all of the data, initialize it into an addressable file system and automatically attempt to connect across a variable number of network protocols (LAN, Wi-Fi Aware, P2P protocols, WebSockets, etc).
Tonk bundles may act as an entry point to a remote filesystem (by just loading the manifest file). They can also be a complete backup of a filesystem (by encoding all of the data into the bundle itself).
We believe this abstraction makes reasoning about the data layer simpler, because there is a material artifact that represents a mounted point and snapshot.
Host-Web Bootloader
Core is used by the Host-Web package. Host-Web is a lightweight website to load and play applications that live in your tonk.
Host-Web automatically looks for bundles under the /app path on your tonk virtual filesystem. It will display their names for you and you can select which one to load into the browser. Loading of the bundle is done via a service worker which intercepts fetch requests and serves those application files to the browser. All you have to do is bundle like you normally would for the web, store it on your VFS in the /app path, and it becomes playable. This is really cool stuff and we first saw this working in PVH's trail-runner.
This feature isn't fully supported across all browsers. We noticed some strange behavior in Arc browser, and it remains unsupported in Mozilla's Firefox, unfortunately. Last we checked, someone is actively working on adding Firefox support. Hopefully soon!
Design Goals for Tonk Core
The design goals for Core are simple:
- A tonk has instructions for how to connect to its peers.
- A tonk works offline.
- A tonk can theoretically run on any device and be served as any kind of application, even web apps.
- A tonk can be shared like a file.
- A tonk's data is encrypted and its entries can only be accessed by each entry's respective, permissioned group.
- A tonk will synchronize the state of its filesystem with any permissioned, connected peer.
- A tonk can be forked or remixed by changing its network or membership group and updating its state.
At the moment, there is work ongoing to bring a native encryption and permission system to Automerge. We believe that's a strong foundation and are hopeful to adopt it. You can read more about it here.
Forking and remixing also brings with it complicated behavior. Versioning is an obvious feature required to make forking and remixing feasible. Luckily, the current design of Automerge gives us a kind of versioning nearly for free.
Roadmap
We believe that Tonk in its current form already can provide a great deal of value to individuals and teams today. We are now in the process of running experiments on use cases.
These use cases require us to do a few things:
- We need an identity model
- We need to further build out our relay infrastructure to support peer connections and backups
- We need to build out tonks that are readily usable
These three things are our immediate priority because they allow us to deliver value today and provide the necessary friction and time to mature our technology.
As the technology matures we'll be able to expand out into the full design goals of Core.
The Destination
Tackling networked intelligence, collaboration and malleability is no easy task and will take time and resources to bake, but the end result will be supremely liberating.
It means you'll be able to work with software as quickly and easily as you work with documents today.
It means our digital lives will stay closer to home, but we won't lose the connections we value so much.
It means that software remains open and free and that you will never be the product for platform profit.
Find Out More
We're building Tonk in the open, together with you. Come build with us!
You can see the code on our GitHub complete with documentation.