diff --git a/development-updates.md b/development-updates.md index fc7b45a9..3e7ff8db 100644 --- a/development-updates.md +++ b/development-updates.md @@ -86,7 +86,7 @@ With the gained insight into the protocol, the following weeks serve as a deep d | [DanGoron](https://github.com/gorondan) | | [Update 4](https://hackmd.io/@kboomro/HyeZXZgxO0) | [Update 5](https://hackmd.io/@kboomro/HkfNC7edR) | [enshrined Operator-Delegator Separation specification](projects/eods-specification.md) | | [Daniel Knopik](https://github.com/dknopik) | [Update 3](https://hackmd.io/@dknopik/epf-week3) | [Update 4](https://hackmd.io/@dknopik/epf-week4) | [Update 5](https://hackmd.io/@dknopik/epf-week5) | [Network Simulations with Shadow](projects/network-simulations-with-shadow.md) | | [Dsorken](https://github.com/Dsorken) | [Update 3](https://hackmd.io/@VgS_FqIfRay_4wp6pMBEgw/HJ9rQ0kD0) | [Update 4](https://hackmd.io/@VgS_FqIfRay_4wp6pMBEgw/r1VWP5_wA) | [Update 5](https://hackmd.io/@VgS_FqIfRay_4wp6pMBEgw/H19snvxuA) | [Besu Portal Client](projects/besu-portal-client.md) | -| [Ekaterina Riazantseva](https://github.com/KatyaRyazantseva) | [Update 3](https://hackmd.io/@katya-blockchain-dev/epf5-week-3) | [Update 4](https://hackmd.io/@katya-blockchain-dev/epf5-week-4) | [Update 5](https://hackmd.io/@katya-blockchain-dev/epf5-week-5) | [PeerDAS metrics](projects/peerdas-metrics.md) | +| [Ekaterina Riazantseva](https://github.com/KatyaRyazantseva) | [Update 3](https://hackmd.io/@katya-blockchain-dev/epf5-week-3) | [Update 4](https://hackmd.io/@katya-blockchain-dev/epf5-week-4) | [Update 5](https://hackmd.io/@katya-blockchain-dev/epf5-week-5) | [PeerDAS metrics specs](projects/peerdas-metrics-specs.md) | | [georgesheth](https://github.com/georgesheth) | [Update 3](https://hackmd.io/@georgesheth/rJxnQBrtC) | [Update 4](https://hackmd.io/@georgesheth/HJKkx3NSR) | [Update 5](https://hackmd.io/@georgesheth/Hk2r2BHFC) | [Push Based Custom Ceiling Partial Withdrawal for EIP-7251 (MaxEB)](projects/Push-Based-Custom-Ceiling-Partial-Withdraw-for-EIP7251-MaxEB.md) | | [ghili](https://github.com/ghiliweld) | [Update 3](https://hackmd.io/@ghili/Hy-NPDvI0) | [Update 4](https://hackmd.io/@ghili/ByCdpKVP0) | [Update 5](https://hackmd.io/@ghili/S1XOsWfdR) | [SSZ Benchmarking and Optimization](projects/ssz-benchmarking-and-optimization.md) | | [Glory Agatevure](https://github.com/gconnect) | [Update 3](https://hackmd.io/@gconnect/rJijgCI8C) | [Update 4](https://hackmd.io/@gconnect/r1TakB_wA) | [Update 5](https://hackmd.io/@gconnect/Skw5Z7f_A) | [Native Ephemery Testnet Client Pair Implementation](/projects/native-ephemery-client-pair-implementation.md) | diff --git a/projects/peerdas-metrics-specs.md b/projects/peerdas-metrics-specs.md new file mode 100644 index 00000000..b71603c6 --- /dev/null +++ b/projects/peerdas-metrics-specs.md @@ -0,0 +1,118 @@ +# PeerDAS Metrics Specifications + +Improving PeerDAS testing efficiency and research capabilities with unified metrics and an integrated dashboard. + +## Motivation + +PeerDAS is now in its testing phase, with CL clients actively participating. Identifying and utilizing specific metrics is crucial for evaluating efficiency, troubleshooting, and optimizing the protocol. Before PeerDAS is deployed on the mainnet, detailed testing is important to keep the Ethereum Protocol secure and efficient. This project aims to improve the testing and research process by providing core developers, DevOps, and research teams with easy access to PeerDAS metrics and a consolidated dashboard. + +## Project description + +PeerDAS Metrics Specifications focuses on identifying the most crucial PeerDAS metrics, standardizing them across consensus clients, and developing a unified Grafana dashboard to visualize these metrics. This setup provides teams with real-time insights, helping them quickly identify issues and optimize client performance. + +## Project components +- [PeerDAS metrics specifications](https://github.com/ethereum/beacon-metrics/pull/13) +- [DevOps Grafana dashboard](https://github.com/KatyaRyazantseva/PeerDAS-metrics/blob/main/dashboards/devops-peerdas-dashboard.json) +- [Kurtosis Grafana dashboard](https://github.com/ethpandaops/ethereum-package/blob/main/static_files/grafana-config/dashboards/peerdas-dashboard.json) + +## Specification + +The project is executed through these specific steps: + +1. Monitoring and Visualization + +Set up and visualize an initial set of Prometheus metrics using Kurtosis and Grafana. This allows for local testing and initial feedback before moving to devnets. + +2. Metrics Review and Expansion + +Continuously review the initial metrics, discussing with teams to identify additional metrics that may provide further value. + +3. Client Integration + +Integrate these standardized metrics into Teku, Grandine, Prysm, and Lighthouse to ensure consistency across the board. + +4. Standardization Discussions + +Participate in PeerDAS Breakout Rooms to reach consensus on these metrics, supporting collaboration and agreement among different client teams. + +5. Feedback and Iteration + +Discuss metrics results with mentors and teams, gather feedback and insights, and iteratively improve the metrics and dashboard based on that feedback. + +## Roadmap + +### **June** + +Explore the Ethereum Protocol, understand its current state and challenges, choose the project focus, and learn the tools like Kurtosis and Grafana. + +### **July** + +Create visualizations of the metrics in Grafana for Lighthouse. + +### **August** + +Implement the chosen metrics in Grandine. + +### **September** + +Implement the metrics in Teku. Create a consolidated Grafana dashboard. Analyze the metrics in comparison. + +### **October** + +Implement the metrics in Prysm. Analyze the metrics to determine their effectiveness, identify missing metrics, and discuss them with the teams. + +### **November** + +Wrap up the project's findings and present at DevCon. + +## Possible challenges + +The unpredictable nature of testing outcomes could lead to unexpected discoveries that may alter the project's direction, such as realizing that some metrics are not needed or that a particular metric is essential for the protocol's functionality. The implementation of the metrics on every client could also bring unexpected challenges. + +## Goal of the project +The project’s success will be measured by its ability to simplify PeerDAS testing and monitoring for DevOps and Consensus clients teams. It also intends to give researchers reliable insights into PeerDAS simulation and performance. + +## Collaborators + +### Fellows +- [Ekaterina Riazantseva]( https://github.com/KatyaRyazantseva) +- Collaboration with [Hangleang](https://github.com/hangleang) on PeerDAS metrics for Grandine. + +### Mentors +DevOps team: +- [Parithosh](https://github.com/parithosh) +- [Barnabas](https://github.com/barnabasbusa) + +CL clients: +- [Dmitrii Shmatko](https://github.com/zilm13) +- [Jimmy Chen](https://github.com/jimmygchen) + +## Resources +Ethereum Protocol +- [EPF Wiki](https://epf.wiki/) +- [Eth2 book: Upgrading Ethereum](https://eth2book.info/latest/book.pdf) +- [Ethereum Roadmap](https://ethroadmap.com/) +- [EPF Cohort 5](https://github.com/eth-protocol-fellows/cohort-five/tree/main) + +Consensus layer +- [Consensus specs](https://github.com/ethereum/consensus-specs) +- [Consensus metrics specs](https://github.com/ethereum/beacon-metrics/blob/master/metrics.md) + +PeerDAS +- [EIP-7594: PeerDAS - Peer Data Availability Sampling](https://eips.ethereum.org/EIPS/eip-7594) +- [Consensus specs: EIP-7594 PeerDAS](https://github.com/ethereum/consensus-specs/tree/master/specs/_features/eip7594) +- [PeerDAS by Dapplion, Youtube](https://www.youtube.com/watch?v=fCIPNxGXmmE&t=43s) +- [Danny Ryan: PeerDAS – a simpler DAS approach using battle-tested p2p components](https://ethresear.ch/t/peerdas-a-simpler-das-approach-using-battle-tested-p2p-components/16541) +- [Francesco: From 4844 to Danksharding: a path to scaling Ethereum DA](https://ethresear.ch/t/from-4844-to-danksharding-a-path-to-scaling-ethereum-da/18046) +- [Vitalik Buterin: An explanation of the sharding + DAS proposal](https://hackmd.io/@vbuterin/sharding_proposal) + +Kurtosis +- [Kurtosis Ethereum Package](https://github.com/ethpandaops/ethereum-package) +- [Kurtosis: A Deep Dive to Local Devnets](https://ethpandaops.io/posts/kurtosis-deep-dive/) +- [Kurtosis Docs](https://docs.kurtosis.com/) + +Grafana +- [Grafana webinars and videos](https://grafana.com/videos/?plcmt=learn-nav) +- [Grafana Docs: Prometheus](https://grafana.com/docs/grafana/latest/datasources/prometheus/) +- [Prometheus types of metrics](https://prometheus.io/docs/tutorials/understanding_metric_types/) +- [Histograms and summaries](https://prometheus.io/docs/practices/histograms/) diff --git a/projects/peerdas-metrics.md b/projects/peerdas-metrics.md deleted file mode 100644 index dd36a3bb..00000000 --- a/projects/peerdas-metrics.md +++ /dev/null @@ -1,78 +0,0 @@ -# PeerDAS Metrics - -Improving PeerDAS testing efficiency via metrics implementation and visualization. - -## Motivation - -PeerDAS is now entering its testing phase, with most CL clients actively participating. Identifying and utilizing specific metrics is crucial for evaluating efficiency, troubleshooting, and optimizing the protocol. Before PeerDAS is deployed on the mainnet, detailed testing is important to keep the Ethereum Protocol secure and efficient. The project aims to analyze and present these results in an accessible format for core developers and the research team. - -## Project description - -PeerDAS Metrics focuses on identifying the most crucial metrics, integrating them into CL clients (Teku, Grandine), and developing visual representations to help teams use these metrics effectively. Additionally, the project will present the analysis of these metrics, providing valuable insights for the research team. - -## Specification - -The project is executed through these specific steps: - -- Monitor and visualize the initial set of metrics using Kurtosis, Prometheus and Grafana. - -- Review the initial metrics, identify additional ones that could be beneficial. -- Integrate these metrics into Teku and potentially into Grandine. -- Discuss and potentially develop Beacon APIs based on the metrics. -- Visualize these APIs using Dora. -- Discuss the results of the metrics with mentors and teams to gather feedback and insights. - -## Roadmap - -### **June** - -Explore the Ethereum Protocol, understand its current state and challenges, choose the project focus, and learn the tools like Kurtosis, Dora, and Grafana. - -### **July** - -Create visualizations of the metrics in Grafana for all PeerDAS-implemented clients. - -### **August** - -Implement the chosen metrics in Teku and Grandine(?). - -### **September** - -Analyze the metrics to determine their effectiveness, identify missing metrics, and discuss potential additions and APIs with the team. - -### **October** - -Contribute to the development of APIs and create visualizations for the APIs on Dora. - -### **November** - -Wrap up the project's findings and present at DevCon. - -## Possible challenges - -The unpredictable nature of testing outcomes could lead to unexpected discoveries that may alter the project's direction, such as realizing that some metrics are not needed or that a particular metric is essential for the protocol's functionality, or even the realization that there is no need for an API. - -## Goal of the project -We define the project being successful as making the PeerDAS testing and monitoring process simpler for DevOps and client teams, and providing the research team with valuable insights into PeerDAS's practical application and performance. - -## Collaborators - -### Fellows -- [Ekaterina Riazantseva]( https://github.com/KatyaRyazantseva) -- Potential collaboration with [Hangleang](https://github.com/hangleang) and Ken Kaneki on the PeerDAS metrics of the Grandine. - -### Mentors -DevOps team: -- [Parithosh](https://github.com/parithosh) -- [Barnabas](https://github.com/barnabasbusa) -- [Savid](https://github.com/Savid) -- [pk910](https://github.com/pk910) - -CL clients: -- [Dmitrii Shmatko](https://github.com/zilm13) - -## Resources -- [PeerDAS – a simpler DAS approach using battle-tested p2p components](https://ethresear.ch/t/peerdas-a-simpler-das-approach-using-battle-tested-p2p-components/16541) -- [From 4844 to Danksharding: a path to scaling Ethereum DA](https://ethresear.ch/t/from-4844-to-danksharding-a-path-to-scaling-ethereum-da/18046) -- [Kurtosis](https://docs.kurtosis.com/) -- [Dora](https://github.com/ethpandaops/dora)