logo icon
UpdatesDiscord Careers

Dialog integration

Product

Tonk architecture has been designed from the start for users to be in control of their digital lives. We believe this is an absolute must for anyone who wants to assemble and reshape software according to their needs and to the needs of their local environment. When people are given control over their data, new use cases are unlocked and when a network can fully connect without platform walls, new value is unlocked.

Our user testing has shown us that every group has its own expression. Groups like to collaborate in many different ways and are using a bag of tools already to accomplish tasks. Rather than changing over from the tools working for them already, they need to layer on bespoke tools to compliment their idiosyncratic workflows. These custom tools need to be easy to share and collaborate over. This is a pattern we see not just in groups but even across individuals.

As a result, we're looking to scope down the UI into a simple, pluggable interface that meets you where you work.

Finally, if you would like to be an early tester for Tonk, please come talk to us! We'd love to hear from you.

Engineering

The code base has gone through a large refactor, as we are in the process of integrating Dialog DB directly into Tonk. You're welcome to follow along on GitHub, but more information will be shared out here at a later time!

Tonk

New Features

  • Space crate - New crate for space functionality with WASM support via wasm-bindgen
  • Access service - Full-featured access control service including:
    • Service DID and keypair management
    • UCAN verification module for capability-based authorization
    • R2 pre-signed URL generation using dialog-s3-credentials
    • Router-based API pattern
    • Blake3 for addressing, SHA256 for integrity checks
  • Tonk UI - Minimum viable user interface

Infrastructure

  • Standardized on nixfmt for code formatting
  • Added Cachix for binary caching
  • Set up stable branch with publish workflow
  • CI improvements with Crane caching and access service publish workflow

Bug Fixes

  • Fixed GHA-specific temp directory handling

Dialog DB

New Features

  • TypeScript packages - TypeScript bindings for dialog-artifacts and port of experimental package
  • Storage backends - REST, S3, file system, and IDB storage backends with type-erased abstraction
  • Artifact system - Causality tracking, constrained selectors, storage overlay, and persisted revisions
  • Serialization - Switched to serde with Brotli compression and configurable compression levels
  • Query engine - Combined artifacts with query engine; ECS-style queries backported
  • Universal index - Fat keys with full datum storage for efficient indexing
  • Efficient columnar encoding - Optimized encoding for columnar data
  • CSV export/import - Bulk data import and export via CSV
  • React hook - React integration for dialog artifacts
  • Transactional memory - Support for transactional operations
  • Tree diffing/patching - Functionality for sparse replication with novelty detection
  • S3 authorization - UCAN-based capability system with async signing and self-authorization
  • Capability system - dialog-capability and dialog-effects crates for embedding dialog
  • Replica generics - Replica made generic over Authority for custom implementations (e.g., WebCryptoAuthority)
  • Diagnostics - dialog-diagnose crate and CLI tool

Improvements

  • Removed const generics from tree, hash, and distribution
  • Improved checksum representation
  • Removed leaky abstractions from capabilities
  • Switched S3 backend to virtual host URLs
  • Backported log! macro
  • Updated wasm-bindgen and wasm-bindgen-cli
  • Enhanced test infrastructure with dialog_common::test macro
  • Integration tests with actual R2/S3 buckets

Bug Fixes

  • Revision now correctly returns hash
  • Fixed reset behavior before commit
  • Corrected outdated divergence clock documentation
  • Demo and sync initialization fixes
  • Pinned s3s to 0.12.0-rc.3 to workaround regression
  • WASM compatibility fixes
  • Fixed race condition in upstream setup
  • Fixed S3 self-authorization
  • Fixed upstream capturing remote repo DID
  • Decommissioned leaky bytes abstraction

Other

  • Switched to MPL 2.0 license
  • Renamed from x- to dialog- prefix
  • Radicle CI integration
  • DBSP hypothesis and general architecture write-ups
  • Glossary of DialogDB concepts

Changelog

Tonk

All notable changes to this project will be documented in this file.

[unreleased]

Bug Fixes

  • Use GHA-specific temp directory (#381)

Features

  • Space crate (#357)
    • fix: clippy feedback
    • fix: import dialog query logic
    • capture hashes in nix
    • fix: clippy errors
    • fix: integrate wasm-bindgen into nix
    • chore: add tonk-space to nix
  • (access) Tonk access service (#365)
    • feat(access): TON-1748: project scaffolding
    • feat(access): TON-1753: service DID and keypair
    • feat(access): TON-1749: UCAN verification module
    • feat(access): TON-1750: R2 pre-signed URLs
    • feat(access): TON-1751: api design
    • feat(access): TON-1756: end-to-end testing
    • feat(access): TON-1752: test edge cases and failure modes
    • fix(access): linting
    • feat(access): TON-1767: space DID Invocation loop, stateless service
    • feat(access): TON-1768: router pattern instead of RPC
    • feat(access): TON-1770: replace presign with s3-presign crate
    • feat(access): TON-1769: blake3 for addressing, sha256 for integrity
    • chore(access): TON-1771: add access service to flake
    • chore: rename s3-presign -> dialog-s3-credentials
    • feat(ci): TON-1772: add access service publish workflow
    • feat(ci): use crane in flake
    • feat(ci): test workflow uses crane caching
  • Minimum viable Tonk UI (#361)

Miscellaneous Tasks

  • Set up stable branch
  • Run publish on stable branch
  • Standardize on nixfmt (#372)
  • Use cachix for binary caching (#378)

Dialog-DB

[unreleased]

Bug Fixes

  • Revision should return hash (#61)
  • Reset before commit (#65)
    • chore: create failing test
    • fix: reset behavior
    • fix: reset code
    • chore: refactor revision handling to remove redundancy
    • chore: simply null revision handling
  • Outdated and invalid statements (#94)
    • Fix invalid description of vector clocks
    • Describe since as counter as opposed to commit count.
    • Add note about convergence preference
  • Demo fixes (#108)
  • Make sync initialization more resilient
  • Pin s3s to 0.12.0-rc.3 to workaround regression in rc.5 (see https://github.com/s3s-project/s3s/issues/432)
  • Add empty tree support for tree_spec!
  • Versions
  • Switch S3 backend to virtual host URLs (#120)
  • Formatting, redundant mut & mut
  • Missing import async_stream import
  • Wasm compatibility (#143)
  • Race condition in upstream setup (#154)
  • (s3) Self-authorization (#158)
  • Upstream capturing remote repo did (#156)
  • Decomission bytes abstraction (#167)

Documentation

  • Scaffold new README.md (#72)

Features

  • TypeScript package for dialog-artifacts (#31)
    • feat: Transaction integrity or else rollback
    • feat: JavaScript bindings for web
    • chore: Add automation for web packages
    • fix: Fix nix on MacOS
    • feat: Add cache layer to storage
    • feat: Artifact selector improvements (#34)
    • chore: More Fact -> Artifact
    • feat: Enforce constrained selectors
  • Optimized bulk storage transfer (#35)
  • Artifact causality (#37)
    • feat: Add cause to Artifact
    • feat: Delete entries from trees
    • fix: JS bindings should throw
    • feat: Cause in JS artifacts
    • feat: Storage overlay
    • fix: Ensure that we use a unique DB in web tests
  • Switch to serde (#42)
    • feat: Use serde for (de)serialization
    • feat: Brotli compression for storage
    • chore: Type-erased storage backend (#43)
    • chore: Make it possible to erase the storage type
    • feat: Configurable compression level
  • Combine artifacts + query engine (#41)
  • Port "experimental" package to TypeScript (#48)
    • feat: ts rewrite
    • fix: what's included in the npm dist
    • fix: vscode config for yml
    • chore: revert job rename
    • chore: rename jobs
    • chore: Build dialog-experimental with Nix (#50)
  • CSV export and import (#52)
    • feat: CSV export and import
    • fix: Enable re-entrant selection of artifacts (#55)
  • Persisted artifacts revision (#58)
  • React hook (#49)
  • Setup Radicle CI integration (#69)
  • Universal index keys, URI entities and remove null (#89)
    • feat: Remove null
    • feat: "Universal" index construction — Indexes now use convertible fat keys and store the full datum (instead of partial information that would sometimes require cross-referencing data in different indexes)
  • Introduce dialog-diagnose crate and CLI tool (#91)
  • Universal index (#92)
  • Efficient encoding for columnar data (#95)
  • Rest storage backend (#45)
  • Add ability to iterate novelty from TreeDifference
  • Integrate tree novelty detection into push / pull system
  • Backport ecs style queries to tonk
  • S3 storage backend
  • Removes const generics from tree, hash and distribution (#116)
  • Dialog_common:test infrastructure (#114)
  • Setup integration test with actual r2/s3 buckets (#121)
  • Transactional memory (#119)
  • Tree diffing / patching functionality needed for sparse replication (#117)
  • (s3) TON-1770: extract presign logic into separate crate
  • (s3) Authorizer trait with UCAN support (#135)
  • UCAN-based capability system for S3 authorization (#146)
    • Adds capability system dialog-capability
    • Adds set of capabilities required for embedding dialog under dialog-effects
    • Updates s3 remote support by basing it on top of dialog-capability
    • Adds ucan mediated s3 remotes
  • (s3) TON-1813: async signing
  • (s3) TON-1822: support UCAN self-authorization
  • Backport log! macro (#155)
  • Remove leaky abstractions from capabilities (#157)
  • Improve checksum representation (#168)
  • (artifacts) Make Replica generic over Authority (#153)
    • This allows using custom Authority implementations (like WebCryptoAuthority) instead of the concrete Operator type. Key changes:
    • Replica<Backend, A> now generic with default A = Operator
    • Branches, Branch, Upstream, RemoteBranch, RemoteSite, RemoteRepository also generic
    • Added ConditionalSync bound for platform-specific Send/Sync requirements
    • Added Principal::from_did() to reconstruct principal from DID string
    • Remote S3 operations extract key bytes via Authority::secret_key_bytes()
    • chore(artifacts): address PR comments — OperatingAuthority trait alias, TryFrom<&Did> for Principal, use Did instead of Principal where possible
    • feat(artifacts): TON-1826: enum SigningAuthority

Miscellaneous Tasks

  • Rename from x- to dialog- (#28)
  • Make doc tests pass (#71)
  • DBSP hypothesis (#73)
    • chore: describe current hypothesis
    • chore: add llm notes for the dbsp hypothesis
  • Write up a general overview (#67)
  • Draft of how privacy & encryption can be managed (#68)
  • Describe divergence clock idea
  • Add Glossary of DialogDB Concepts (#97)
  • First draft of the tree node level diff
  • Integrate latest s3 backend
  • Update s3 tests to use dialog_common::test macro (#122)
  • Switch to MPL 2.0 license (#130)
  • Merge main into tonk branch
  • Cargo fmt --all
  • Undo unintended changes
  • Remove dialog dependency
  • (s3) Rename s3-presign -> dialog-s3-credentials
  • Update wasm-bindgen and wasm-bindgen-cli (#141)
  • Update flake for chromedriver 144
  • Satisfy clippy
  • (s3) TON-1825: update for async-only signing API

[last-commit-with-x-query] - 2025-04-11

Features

  • Add sync compatibility traits
  • Basic query crate (#1) — Implements basic pull queries (including non-recursive rules) over a triple store
  • Implement basic storage crate (#10)
  • File system and IDB storage backends (#11)
  • Port of ranked-prolly-tree crate (#13)
  • Basic facts crate (#17)

Miscellaneous Tasks

  • Setup adr (#2)