siema

README.md

@@ -8,7 +8,7 @@
 ### The Raft Consensus Algorithm in Zig
 
 This repository houses `zaft` - a [Raft Consensus Algorithm](https://raft.github.io/) library implemented in Zig. It provides the building blocks
-for building distributed systems requiring consensus among replicated state machines, like databases.
+for creating distributed systems requiring consensus among replicated state machines, like databases.
 
 
 ![tang demo](tang.gif)
@@ -46,27 +46,21 @@ const zaft = @import("zaft");
 
 ## Usage
 
+This section will show you how to integrate `zaft` into your system step-by-step. If you want to take look at a fully working example,
+check out the [kv_store](./examples/kv_store) - in-memory, replicated key-value store based on `zaft`.
+
 > [!IMPORTANT]
 > This usage tutorial assumes some familiarity with the Raft Consensus Algorithm. If not, I highly advise you to at least skim through
-> the [Raft paper](https://raft.github.io/raft.pdf) and familiarize yourself with how the algorithm works. Don't worry, it's a very well
-> written paper!
+> the [Raft paper](https://raft.github.io/raft.pdf). Don't worry, it's a short and very well written paper!
 
-Firstly, initialise the `Raft` struct
+Firstly, initialise the `Raft` struct:
 
 ```zig
-// we'll get through to UserData and Entry later
+// we'll get to UserData and Entry in a second
 const Raft = @import("zaft").Raft(UserData, Entry);
 
-const config = Raft.Config{
-};
-const callbacks = Raft.Callbacks{
-    // ..
-};
-const initial_state = Raft.InitialState{
-    // ...
-};
-
-const raft = Raft.init(config, initial_state, callbacks)
+const raft = Raft.init(config, initial_state, callbacks);
+defer raft.deinit();
 ```
 
 `Raft.init` takes three arguments:
@@ -85,55 +79,54 @@ const config = Raft.Config{
 * `callbacks` - `Raft` will call this function to perform various actions:
 
 ```zig
+// makeRPC is used to send Raft messages to other nodes
+// this function should be non-blocking, that is, not wait for the response
 fn makeRPC(ud: *UserData, id: u32, rpc: Raft.RPC) !void {
-    // makeRPC is used to send Raft messages to other nodes
-    // this function should be non-blocking, that is, not wait for the response
     const address = ud.node_addresse[id];
     // it's your responsibility to serialize the message, consider using e.g. std.json
     const msg: []u8 = serialize(rpc);
     try ud.client.send(address, msg);
 }
 
+// Entry can be whatever you want
+// in this callback the entry should be applied to the state machine
+// applying an entry must be deterministic! That is, after applying the
+// same entries in the same order, the state machine must be in the same state
 fn applyEntry(ud: *UserData, entry: Entry) !void {
-    // Entry can be whatever you want
-    // in this callback the entry should be applied to the state machine
-    // applying an entry must be deterministic! That is, after applying the
-    // same entries in the same order, the state machine must be in the same state
-
     // let's assume that is some kind of key-value store
     switch(entry) {
         .add => |add| try ud.store.add(add.key, add.value),
         .remove => |remove| try ud.store.remove(remove.key),
     }
 }
 
+// this function needs to persist a new log entry
 fn logAppend(ud: *UserData, log_entry: Raft.LogEntry) !void {
-    // this function needs to persist a new log entry
     try ud.database.appendEntry(log_entry);
 }
 
+// this function needs to pop the last log entry from the persistent storage
 fn logPop(ud: *UserData) !Raft.LogEntry {
-    // this function needs to pop the last log entry from the persistent storage
     const log_entry = try ud.database.popEntry();
     return log_entry;
 }
 
+// this function needs to persist current_term
 fn persistCurrentTerm(ud: *UserData, current_term: u32) !void {
-    // this function needs to persist current_term
     try ud.database.persistCurrentTerm(current_term);
 }
 
-
+// this function needs to persist voted_for
 fn persistVotedFor(ud: *UserData, voted_for: ?u32) !void {
-    // this function needs to persist voted_for
     try ud.database.persistVotedFor(voted_for);
 }
 ```
 
 > [!WARNING]
-> Notice that all of the callbacks can return an error for the sake of convinience.
+> Notice that all of the callbacks can return an error (mostly for the sake of convinience).
+>
 > Error returned from `makeRPC` will be ignored, the RPC will be simply retried after
-> appropriate timeou. Error returned from other function, as of now, will result in panic.
+> appropriate timeout. Errors returned from other function, as of now, will result in panic.
 
 ```zig
 // pointer to user_data will be passed as a first argument to all of the callbacks
@@ -168,6 +161,43 @@ const initial_state = Raft.InitialState {
 };
 ```
 
+The `Raft` struct needs to be periodically ticked in order to trigger timeouts and other necessary actions. You can use a separate thread to do that, or
+built your app based on an event loop like [libexev](https://github.com/mitchellh/libxev) with its `xev.Timer`.
+
+```zig
+const tick_after = raft.tick();
+// tick_after is a number of milliseconds after which raft should be ticked again
+```
+
+For instance, [kv_store](./examples/kv_store/src/ticker.zig) uses a separate thread exclusively to tick the `Raft` struct.
+
+> [!WARNING]
+> The `Raft` struct is *not* thread-safe. Use appropriate synchronization means to makes sure it is not accessed simultaneously by many threads,
+> like `std.Thread.Mutex`.
+
+Next, messages from other Raft nodes need to be feed to local `Raft` struct by calling:
+
+```zig
+// you will need to receive and deserialize the messages from other peers
+const msg: Raft.RPC = try recv_msg();
+raft.handleRPC(msg);
+```
+
+Lastly, entries can be appended to `Raft`s log by calling:
+
+```zig
+const entry: Entry = ...
+const idx = try raft.appendEntry(entry);
+```
+
+It will return an index of the new entry. According to the Raft algorithm, you application should block on client request
+until the entry has been applied. You can use e.g. `std.Thread.Condition` and call its `notify` function in the `applyEntry` callback.
+You can check whether entry was applied by using `raft.checkIfApplied(idx)`. Take a look at how [kv_store](./examples/kv_store/src/main.zig) does this.
 
+`appendEntry` function will return error if the node is not a leader. In such case, you should redirect the client request to the leader node.
+You can chack which node is the leader by using `raft.getCurrentLeader()`. You can also check if the node is a leader proactively by calling
+`raft.checkifLeader()`.
 
+## Next steps
 
+1. Siema