From 69ec44f7b14a41021ed274a3fb5a0008c476090b Mon Sep 17 00:00:00 2001 From: Tim McMackin Date: Mon, 4 Dec 2023 13:46:06 -0500 Subject: [PATCH] Architecture overview (#161) * Rough draft of architecture and the components of Tezos * previous block in the chain * Reorg into smaller topics per discussion with Nic * layer 2 can only respond to messages from layer 1 * impractical on layer 1 * No need to separate these into separate pages * Clarifications * Revert changes to protocol topic because I'll handle it in another branch * Remove old protocol topic * These links are no longer accurate * Update based on work with Nic * Remove old info * It has control over which clients to accept calls from, which calls to accept, or whether to accept RPC calls at all. * Section for the Octez client * Review comments from Nic * updates are performed via an online governance process Co-authored-by: NicNomadic <148877430+NicNomadic@users.noreply.github.com> * capitalize Sapling * shields bakers from network attacks Co-authored-by: NicNomadic <148877430+NicNomadic@users.noreply.github.com> * commonly called Tezos networks Co-authored-by: NicNomadic <148877430+NicNomadic@users.noreply.github.com> * dedicated voting operations such as protocol proposals or protocol upvotes Co-authored-by: NicNomadic <148877430+NicNomadic@users.noreply.github.com> * Clarify operations based on diagram changes * This line could cause some confusion --------- Co-authored-by: NicNomadic <148877430+NicNomadic@users.noreply.github.com> --- docs/architecture.md | 24 ----- docs/architecture.mdx | 157 +++++++++++++++++++++++++++++ docs/architecture/protocol.md | 72 ------------- docs/architecture/smart-rollups.md | 4 - sidebars.js | 9 +- 5 files changed, 161 insertions(+), 105 deletions(-) delete mode 100644 docs/architecture.md create mode 100644 docs/architecture.mdx delete mode 100644 docs/architecture/protocol.md diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 4db8cb23c..000000000 --- a/docs/architecture.md +++ /dev/null @@ -1,24 +0,0 @@ -# Architecture - -- Who created it and who is in charge of it -- What entities participate in it -- How it is built (architecture, with diagrams) - -## Decentralization -## p2p network (introducting nodes, baking) -## smart contracts and dapps -## governance and roadmap for scaling - -## goals/inspiration, tiny bit of history -need for decentralization, smart contracts, then for Tezos, need for blockchains to evolve, be environmental friendly, secure -## p2p network, DOS protection, etc. -Idea: to make it more active, create an interactive experience where people simulate playing different roles, through the path of a transaction -## accounts, cryptocurrency, transactions -Here the animation would let people play with transactions ana d simulated wallet -## chain of blocks -idea that for people to agree on a sequence of transactions, they need to be grouped in block, and we create an append-only structure -## baking, consensus mechanism -"someone needs to propose a new block, so we have a system involving bakers, then a mechanism to pick the baker, confirm the block through attestations, and finality -you could have a small interface where you try to create the best block" -## recap: path of a transaction -wallet, simulation, propagation through the nodes, mempool, accepted by a baker, block is propagated and validated, then becomes final, shown as validated on the wallet, and visible by everyone \ No newline at end of file diff --git a/docs/architecture.mdx b/docs/architecture.mdx new file mode 100644 index 000000000..1aae651d7 --- /dev/null +++ b/docs/architecture.mdx @@ -0,0 +1,157 @@ +--- +title: Architecture +authors: Tim McMackin +last_update: + date: 28 November 2023 +--- + +import LucidDiagram from '@site/src/components/LucidDiagram'; + +The Tezos blockchain is composed of many Tezos nodes running around the world, complemented by other running daemons, such as bakers and accusers. +These processes collaborate with the overall goal of maintaining the blockchain data in a decentralized manner. + +The Tezos nodes are the most important piece in this architecture because they maintain the system and the blockchain data. +Users interact with nodes through many different clients, including command-line clients, wallets, and web applications. + +This diagram shows a high-level view of the Tezos system: + + + +## The blockchain data + +Although people often use the word "blockchain" to mean the entire system, strictly speaking, a Tezos blockchain is a series of blocks of data, each connected to the previous block in the chain, beginning with the genesis block. +The blockchain data is maintained by a network of Tezos nodes. +Nodes reach consensus on the next block before adding it to the chain. + +The primary Tezos network is referred to as Mainnet, and there are several other networks used for testing, referred to as testnets. +Anyone can create new test networks if needed. + +As shown in the diagram, the data inside a Tezos block includes the hash of the previous block in the chain and many operations, such as transactions that transfer tez or call smart contracts. +Blocks also include operations that are necessary for the management of the chain, including nodes' attestations that blocks are valid, called _consensus operations_, and votes on changes to the protocol, called _voting operations_. +For more information on the operations that can be included in blocks, see [Blocks and operations](https://tezos.gitlab.io/alpha/blocks_ops.html) in the Octez documentation. + +## Nodes + +A Tezos node has three main roles: + +- It validates blocks and operations +- It broadcasts blocks and operations to other nodes and receives them from other nodes over a peer-to-peer network +- It maintains a copy of the blockchain data and its associated state (also known as the ledger), which includes accounts and their balances, among other things + +Beside these technical roles, nodes must satisfy two other important requirements: + +- Support the governance of the blockchain +- Ensure the extensibility of the blockchain with new clients of different kinds. + +In order to meet these requirements, the software that nodes run is structured according to two major principles: + +- It is separated into a shell and a protocol, to support protocol evolution. +- It implements a client/server architecture, to allow composition with many other tools in a safe way. + +The Octez suite, which is an implementation of the Tezos node and other executables, instantiates these principles in the [Octez software architecture](https://tezos.gitlab.io/shell/the_big_picture.html). + +These two principles are detailed next. + +## Shell & Protocol + +The software that runs Tezos nodes is split into two main parts: + +- The shell, which handles low-level functions like data storage and peer-to-peer network communication +- The economic protocol that interprets the block operations (e.g., transactions) + +The relationship between the shell and the protocol is like the relationship between an operating system and an application. +The operating system stays stable while the application can update itself. +In this way, Tezos can update how it works (its protocol) without requiring nodes to accept major changes to the software that they run (the shell). +For example, nodes can update to a new protocol version without restarting the shell. + +### The Tezos self-amending protocol + +Unlike many other blockchains, Tezos is self-amending. +Its nodes can update the protocol that controls the possible operations and how they are processed; updates are performed via an online governance process. +These updates allow Tezos to adapt to new technologies and respond to user needs. +For example, protocol upgrades have added new features like Smart Rollups and have reduced the amount of time between blocks. + +This protocol is responsible for interpreting the operations in each block. +It also provides the logic that identifies erroneous blocks. + +Updates to the protocol can change this logic through a voting process, using dedicated voting operations such as protocol proposals or protocol upvotes. +For information about the voting process, see [Governance](./architecture/governance). + +## The shell + +The shell is responsible for the fundamental functions of a distributed software application, including: + +- Peer-to-peer communication that lets nodes exchange information +- Storage functionality that lets nodes store blocks, operations, and the current state of the chain +- A synchronization heuristic that starts nodes and keeps them in sync with the network +- A validator that checks that blocks are valid with help from the rules in the economic protocol + +In particular, the validator is responsible for resolving the available blocks into a single linear sequence of blocks. +It chooses between the various blocks that baking nodes create, uses the protocol to verify and score them, and selects the tree head with the highest score. +Then it uses that linear chain in all of its work with the protocol, so the protocol is never aware of multiple branches. + +## Tezos clients and servers + +In addition to the functions of the protocol and shell, a Tezos node also acts as a server to respond to queries and requests from clients. +A client can query the chain’s state and can inject blocks and operations into a node. + +Tezos uses this client-server architecture for these main reasons: + +- It improves security by separating the node, which is exposed to the internet, from the baker, which has access to the client keys. +The node and the baker can be on different computers, which allows the node to manage communication and shields bakers from network attacks. +- It allows bakers to have different implementations. +For example, different bakers may implement different transaction selection strategies. +- It allows clients and other tools to interact with the node and inspect its state. + +The node accepts calls from clients through its RPC interface. +It has control over which clients to accept calls from, which calls to accept, or whether to accept RPC calls at all. +For more information on the RPC interface, see [The RPC protocol](./architecture/rpc). + +### The baker daemon + +The baker daemon is an Octez program that is responsible for creating and proposing new blocks based on the operations proposed by different clients. +The baker daemon is associated with an account, which means that it has access to the account’s private key, which it uses to sign blocks and operations. + +### The accuser daemon + +The accuser daemon is an Octez program that monitors new blocks and looks for problems, such as when bakers try to add more than one block at a time. +When it finds a problem, it submits a denunciation to other nodes to refuse the new blocks and punish the offending node. + +### The Octez client + +The Octez client is a command-line tool that developers can use for many Tezos-related tasks, including: + +- Deploying, calling, testing, and interacting with contracts +- Deploying and interacting with Smart Rollups +- Working with accounts +- Calling RPC endpoints directly +- Running Sapling transactions +- Setting up baking operations for testing contracts + +For more information about the Octez client, see the [Octez documentation](https://tezos.gitlab.io/). + +### External clients + +Many external clients can add operations to the network of nodes or use nodes to inspect the state of the blockchain, including: + +- Web applications that use SDKs such as Taquito to send and receive information from Tezos +- Indexer websites that monitor the state of the network and allow users to search its history +- Wallet applications + +Nodes share operations with each other, so the node that includes an operation in a block may not be the node that the client originally sent the operation to. + +Anyone can run a node and select which clients to run and which requests to accept from clients. +Some typical use cases for nodes are: + +- A node running with no daemons, which maintains a copy of the blockchain data and enhances the distribution of the network without actively baking blocks. +Optionally, this node can open its RPC interface to serve different kinds of requests. +- A node along with a baker, an accuser, and a signer can be used to bake new blocks, activity which ensures that the blockchain progresses and yields rewards in tokens. + +## References + +For more information about the architecture of Tezos, see: + +- [Accounts](./architecture/accounts) +- [Tokens](./architecture/tokens) +- [Smart Optimistic Rollups](./architecture/smart-rollups) +- [Governance](./architecture/governance) diff --git a/docs/architecture/protocol.md b/docs/architecture/protocol.md deleted file mode 100644 index c00cab0b9..000000000 --- a/docs/architecture/protocol.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: Tezos Protocol and Shell -last_update: - date: 29 June 2023 ---- - -The primary characteristic that makes Tezos unique is its self-amending property. The part that amends itself is called the economic protocol. The rest of the Tezos node is what we call the shell. - -## Tezos' Self-Amending Protocol - -The protocol is responsible for interpreting the transactions and other administrative operations. It also has the responsibility to detect erroneous blocks. - -An important thing to notice is that the protocol always sees only one blockchain, that is, a linear sequence of blocks since the genesis. It does not know that it lives in an open network where nodes can propose alternative heads. - -Only the shell knows about the multiple heads. It is responsible for choosing between the various chain proposals that come from the bakers (the programs that create new blocks) of the network. The shell has the responsibility of selecting and downloading alternative chains, feeding them to the protocol, which in turn has the responsibility to check them for errors and give them an absolute score. The shell then simply selects the valid head of the highest absolute score. This part of the shell is called the validator. - -The rest of the shell includes the peer-to-peer layer, the disk storage of blocks, the operations to allow the node to transmit the chain data to new nodes, and the versioned state of the ledger. In-between the validator, the peer-to-peer layer, and the storage sits a component called the distributed database, that abstracts the fetching and replication of new chain data to the validator. - -Protocols are compiled using a tweaked OCaml compiler that does two things. First, it checks that the protocol’s main module has the right type. A good analogy is to see protocol as plug-ins, and in this case, it means that it respects the common plugin interface. Then, it restricts the typing environment of the protocol’s code so that it only calls authorized modules and functions. Seeing protocols as plug-ins, it means that the code only called primitives from the plug-in API. It is a form of statically enforced sandboxing. - -Note that the economic protocol on the chain is subject to an amendment procedure. On-chain operations can be used to switch from one protocol to another. - -Finally, the RPC layer is an important part of the node. It is how the client, third-party applications, and daemons can interact with the node and introspect its state. This component uses the mainstream JSON format and HTTP protocol. It uses the library resto. It is fully interoperable, and auto-descriptive, using JSON schema. - -A Tezos node has mainly three roles: it validates blocks and operations, it broadcasts them to (and retrieves them from) other nodes, and it maintains a main chain and its associated state (i.e. the ledger), which -includes accounts and their balances, among other things. Note that, as blocks only specify a predecessor block, exchanged blocks do not necessarily form a chain, but rather a tree. - -A Tezos node acts as a server, which responds to queries and requests from clients. Such queries and requests are implemented via RPC calls. A client can query the chain's state and can inject blocks and operations into a -node. One particular client is the `baker daemon`, which is associated to an account. In particular, the baker has access to the account's private key and thus can sign blocks and operations. - -The main reason for using such a client-server architecture is safety: to insulate the component that has access to the client keys, i.e. the baker, from the component which is exposed to the internet, i.e. the -node. Indeed, the node and the baker can sit on different computers and the baker does not need to be exposed to the internet. So nodes manage communication and shield bakers from network attacks, and bakers hold -secrets and bake blocks into the blockchain. - -Another advantage of this architecture is that bakers can more easily have different implementations, and this is important, for instance because different bakers may want to implement different transaction -selection strategies. - -As previously stated, Tezos is a self-amending blockchain. In large part of Tezos can be changed through a so-called amendment procedure. - -- the shell, which comprises the network and storage layer, and embeds -- the economic protocol component, which is the part that can be - changed through amendment. - -## The function of the economic protocol - -The economic protocol provides the rules for checking the validity of the blocks and operations, and for updating the blockchain state accordingly, by applying new valid blocks and operations on the current blockchain state. It defines under which conditions a block is a valid extension of the current blockchain, and defines an ordering on blocks to arbitrate between concurrent extensions. - -The rules it uses for recognizing and applying valid blocks can be changed through voting. Thus, the economic protocol represents the amendable part of Tezos. To learn about the voting and amendment process see [Tezos Agora's overview](https://www.tezosagora.org/learn) of Tezos Governance. - -Each Tezos protocol is named after a (usually ancient) city, with the first one named Athens. Each successive name follows the previous alphabetically, continuing to Babylon, Carthage, Delphi, and so on. At the time of writing this article, the network is on the Mumbai protocol, with Nairobi planned for June 2023. There is a protocol upgrade roughly every 3 months. - -## Shell-protocol interaction - -The economic protocol and the shell interact in order to ensure that the blocks being appended to the blockchain are valid. The shell does not accept a branch whose fork point is some number of cycles in the past--anything before is deemed immutable. Also, when receiving a new block the shell determines if the block is valid and its fitness is higher than the current head. - -The protocol provides support for validating blocks which can be modulated by different "validation modes". - - -## The Shell - -The shell contains everything that is not part of the economic protocol. It includes: - -- The Validator: the component responsible for checking that blocks coming from the network or a baker are valid, w.r.t. the rules defined by the economic protocol. -- The Prevalidator: responsible for determining which operations to propagate for this chain over the peer-to-peer network. -- The Storage Layer: responsible for aggregating blocks (along with their respective ledger state) and operations within blocks (along with their associated metadata). -- Synchronization heuristic: Process to bootstrap a node when first joining the network, and knowing when it's synced with the network. -- The peer-to-peer layer: this part is in charge of establishing and maintaining network connections with other nodes (via a gossip protocol). -- Protocol Environment: a restricted API that may be used by the Octez code to interact with the protocol, and vice versa. - - -:::note -In-depth documentation on the [shell](https://tezos.gitlab.io/shell/the_big_picture.html) and [protocol](https://tezos.gitlab.io/active/protocol.html) and the semantics of [operations and blocks](https://tezos.gitlab.io/active/blocks_ops.html) can be found in Tezos' [technical documentation](https://tezos.gitlab.io/index.html). diff --git a/docs/architecture/smart-rollups.md b/docs/architecture/smart-rollups.md index 2cd17447c..553da21fb 100644 --- a/docs/architecture/smart-rollups.md +++ b/docs/architecture/smart-rollups.md @@ -7,10 +7,6 @@ last_update: Rollups play a crucial part in providing next-generation scaling on Tezos. This page gives a technical introduction to Smart Rollups, their optimistic nature, and an intro to developing your own WASM kernel. -## Prerequisites - -This page covers an advanced topic at the bleeding edge of Tezos core development. If you are interested in more fundamental reading, a great place to start is [Tezos Protocol and Shell](./protocol) and [Smart Contracts](../smart-contracts/languages). - ## Examples For examples of Smart Rollups, see this repository: https://gitlab.com/tezos/kernel-gallery. diff --git a/sidebars.js b/sidebars.js index 3ded7a578..5c7820330 100644 --- a/sidebars.js +++ b/sidebars.js @@ -19,12 +19,11 @@ const sidebars = { { type: 'category', label: 'Architecture', - // link: { // TODO - // id: 'architecture', - // type: 'doc', - // }, + link: { + id: 'architecture', + type: 'doc', + }, items: [ - 'architecture/protocol', 'architecture/accounts', { type: 'category',