Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Katya: Week-21 updates and update project proposal #539

Merged
merged 2 commits into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion development-updates.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ With the gained insight into the protocol, the following weeks serve as a deep d
| [DanGoron](https://github.com/gorondan) | | [Update 4](https://hackmd.io/@kboomro/HyeZXZgxO0) | [Update 5](https://hackmd.io/@kboomro/HkfNC7edR) | [enshrined Operator-Delegator Separation specification](projects/eods-specification.md) |
| [Daniel Knopik](https://github.com/dknopik) | [Update 3](https://hackmd.io/@dknopik/epf-week3) | [Update 4](https://hackmd.io/@dknopik/epf-week4) | [Update 5](https://hackmd.io/@dknopik/epf-week5) | [Network Simulations with Shadow](projects/network-simulations-with-shadow.md) |
| [Dsorken](https://github.com/Dsorken) | [Update 3](https://hackmd.io/@VgS_FqIfRay_4wp6pMBEgw/HJ9rQ0kD0) | [Update 4](https://hackmd.io/@VgS_FqIfRay_4wp6pMBEgw/r1VWP5_wA) | [Update 5](https://hackmd.io/@VgS_FqIfRay_4wp6pMBEgw/H19snvxuA) | [Besu Portal Client](projects/besu-portal-client.md) |
| [Ekaterina Riazantseva](https://github.com/KatyaRyazantseva) | [Update 3](https://hackmd.io/@katya-blockchain-dev/epf5-week-3) | [Update 4](https://hackmd.io/@katya-blockchain-dev/epf5-week-4) | [Update 5](https://hackmd.io/@katya-blockchain-dev/epf5-week-5) | [PeerDAS metrics](projects/peerdas-metrics.md) |
| [Ekaterina Riazantseva](https://github.com/KatyaRyazantseva) | [Update 3](https://hackmd.io/@katya-blockchain-dev/epf5-week-3) | [Update 4](https://hackmd.io/@katya-blockchain-dev/epf5-week-4) | [Update 5](https://hackmd.io/@katya-blockchain-dev/epf5-week-5) | [PeerDAS metrics specs](projects/peerdas-metrics-specs.md) |
| [georgesheth](https://github.com/georgesheth) | [Update 3](https://hackmd.io/@georgesheth/rJxnQBrtC) | [Update 4](https://hackmd.io/@georgesheth/HJKkx3NSR) | [Update 5](https://hackmd.io/@georgesheth/Hk2r2BHFC) | [Push Based Custom Ceiling Partial Withdrawal for EIP-7251 (MaxEB)](projects/Push-Based-Custom-Ceiling-Partial-Withdraw-for-EIP7251-MaxEB.md) |
| [ghili](https://github.com/ghiliweld) | [Update 3](https://hackmd.io/@ghili/Hy-NPDvI0) | [Update 4](https://hackmd.io/@ghili/ByCdpKVP0) | [Update 5](https://hackmd.io/@ghili/S1XOsWfdR) | [SSZ Benchmarking and Optimization](projects/ssz-benchmarking-and-optimization.md) |
| [Glory Agatevure](https://github.com/gconnect) | [Update 3](https://hackmd.io/@gconnect/rJijgCI8C) | [Update 4](https://hackmd.io/@gconnect/r1TakB_wA) | [Update 5](https://hackmd.io/@gconnect/Skw5Z7f_A) | [Native Ephemery Testnet Client Pair Implementation](/projects/native-ephemery-client-pair-implementation.md) |
Expand Down
118 changes: 118 additions & 0 deletions projects/peerdas-metrics-specs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# PeerDAS Metrics Specifications

Improving PeerDAS testing efficiency and research capabilities with unified metrics and an integrated dashboard.

## Motivation

PeerDAS is now in its testing phase, with CL clients actively participating. Identifying and utilizing specific metrics is crucial for evaluating efficiency, troubleshooting, and optimizing the protocol. Before PeerDAS is deployed on the mainnet, detailed testing is important to keep the Ethereum Protocol secure and efficient. This project aims to improve the testing and research process by providing core developers, DevOps, and research teams with easy access to PeerDAS metrics and a consolidated dashboard.

## Project description

PeerDAS Metrics Specifications focuses on identifying the most crucial PeerDAS metrics, standardizing them across consensus clients, and developing a unified Grafana dashboard to visualize these metrics. This setup provides teams with real-time insights, helping them quickly identify issues and optimize client performance.

## Project components
- [PeerDAS metrics specifications](https://github.com/ethereum/beacon-metrics/pull/13)
- [DevOps Grafana dashboard](https://github.com/KatyaRyazantseva/PeerDAS-metrics/blob/main/dashboards/devops-peerdas-dashboard.json)
- [Kurtosis Grafana dashboard](https://github.com/ethpandaops/ethereum-package/blob/main/static_files/grafana-config/dashboards/peerdas-dashboard.json)

## Specification

The project is executed through these specific steps:

1. Monitoring and Visualization

Set up and visualize an initial set of Prometheus metrics using Kurtosis and Grafana. This allows for local testing and initial feedback before moving to devnets.

2. Metrics Review and Expansion

Continuously review the initial metrics, discussing with teams to identify additional metrics that may provide further value.

3. Client Integration

Integrate these standardized metrics into Teku, Grandine, Prysm, and Lighthouse to ensure consistency across the board.

4. Standardization Discussions

Participate in PeerDAS Breakout Rooms to reach consensus on these metrics, supporting collaboration and agreement among different client teams.

5. Feedback and Iteration

Discuss metrics results with mentors and teams, gather feedback and insights, and iteratively improve the metrics and dashboard based on that feedback.

## Roadmap

### **June**

Explore the Ethereum Protocol, understand its current state and challenges, choose the project focus, and learn the tools like Kurtosis and Grafana.

### **July**

Create visualizations of the metrics in Grafana for Lighthouse.

### **August**

Implement the chosen metrics in Grandine.

### **September**

Implement the metrics in Teku. Create a consolidated Grafana dashboard. Analyze the metrics in comparison.

### **October**

Implement the metrics in Prysm. Analyze the metrics to determine their effectiveness, identify missing metrics, and discuss them with the teams.

### **November**

Wrap up the project's findings and present at DevCon.

## Possible challenges

The unpredictable nature of testing outcomes could lead to unexpected discoveries that may alter the project's direction, such as realizing that some metrics are not needed or that a particular metric is essential for the protocol's functionality. The implementation of the metrics on every client could also bring unexpected challenges.

## Goal of the project
The project’s success will be measured by its ability to simplify PeerDAS testing and monitoring for DevOps and Consensus clients teams. It also intends to give researchers reliable insights into PeerDAS simulation and performance.

## Collaborators

### Fellows
- [Ekaterina Riazantseva]( https://github.com/KatyaRyazantseva)
- Collaboration with [Hangleang](https://github.com/hangleang) on PeerDAS metrics for Grandine.

### Mentors
DevOps team:
- [Parithosh](https://github.com/parithosh)
- [Barnabas](https://github.com/barnabasbusa)

CL clients:
- [Dmitrii Shmatko](https://github.com/zilm13)
- [Jimmy Chen](https://github.com/jimmygchen)

## Resources
Ethereum Protocol
- [EPF Wiki](https://epf.wiki/)
- [Eth2 book: Upgrading Ethereum](https://eth2book.info/latest/book.pdf)
- [Ethereum Roadmap](https://ethroadmap.com/)
- [EPF Cohort 5](https://github.com/eth-protocol-fellows/cohort-five/tree/main)

Consensus layer
- [Consensus specs](https://github.com/ethereum/consensus-specs)
- [Consensus metrics specs](https://github.com/ethereum/beacon-metrics/blob/master/metrics.md)

PeerDAS
- [EIP-7594: PeerDAS - Peer Data Availability Sampling](https://eips.ethereum.org/EIPS/eip-7594)
- [Consensus specs: EIP-7594 PeerDAS](https://github.com/ethereum/consensus-specs/tree/master/specs/_features/eip7594)
- [PeerDAS by Dapplion, Youtube](https://www.youtube.com/watch?v=fCIPNxGXmmE&t=43s)
- [Danny Ryan: PeerDAS – a simpler DAS approach using battle-tested p2p components](https://ethresear.ch/t/peerdas-a-simpler-das-approach-using-battle-tested-p2p-components/16541)
- [Francesco: From 4844 to Danksharding: a path to scaling Ethereum DA](https://ethresear.ch/t/from-4844-to-danksharding-a-path-to-scaling-ethereum-da/18046)
- [Vitalik Buterin: An explanation of the sharding + DAS proposal](https://hackmd.io/@vbuterin/sharding_proposal)

Kurtosis
- [Kurtosis Ethereum Package](https://github.com/ethpandaops/ethereum-package)
- [Kurtosis: A Deep Dive to Local Devnets](https://ethpandaops.io/posts/kurtosis-deep-dive/)
- [Kurtosis Docs](https://docs.kurtosis.com/)

Grafana
- [Grafana webinars and videos](https://grafana.com/videos/?plcmt=learn-nav)
- [Grafana Docs: Prometheus](https://grafana.com/docs/grafana/latest/datasources/prometheus/)
- [Prometheus types of metrics](https://prometheus.io/docs/tutorials/understanding_metric_types/)
- [Histograms and summaries](https://prometheus.io/docs/practices/histograms/)
78 changes: 0 additions & 78 deletions projects/peerdas-metrics.md

This file was deleted.