Architecture of the System
replikativ
consists of two parts, a core of CRDTs (convergent replicated
datatypes), especially a newly crafted one for
the git-like CDVCS datatype
(a bit more polished PaPoC 2016 paper)
in the replikativ.crdt.cdvcs
namespaces, and a generic replication protocol
for CRDTs in replikativ.core
and some middlewares. The fetch middleware deals
with pulling all referenced data and the hash middleware ensures cryptographic
integrity. The replication can be externally extended to any CRDT (as long as
all connected peers support it then). We will provide as many implementations as
possible by default for the open, global replication system. Together the CRDTs
and the replication provides conflict-free convergent replication. The datatypes
decouple resolution of application level state changes from replication over a
network.
The replication protocol partitions the global state space into user specific
places for CRDTs, [user-id crdt-id]
. All replication happens between these
distributed instances. All peers are supposed to automatically replicate CRDTs
of each user they subscribe to.
We make heavy use of
core.async to model peers
platform- and network-agnostic just as peers having a pair of
messaging channels from kabel
for edn
messages. We build on platform-neutral durable storage
through konserve. At the
core is a pub-sub
scheme between peers, but most functionality is
factored into middlewares
filtering and tweaking the in/out channel
pair of each peer’s pub-sub core. This allows decoupled extension of
the network protocol.
For a detailed documentation of the CDVCS implementation you can have a look at the introduction. Or to understand the pub-sub message protocol for replication.
The joint system has won a best poster award at EuroSys 2016.