MemLens is a comprehensive CPU and memory profiler for ResilientDB which aims to provide granular performance metrics by integrating profiling tools like eBPF, node exporter, process exporter, LevelDB internal statistics and custom prometheus hooks. We aim to provide a platform which provides database specific metrics that provides detailed information on what happens under the hood and help optimize system performance. This is an ambitious idea which we are improving on constantly but are proud to share a packaged version of MemLens currently.
MemLens provides CPU, memory and build statistics aiming to provide detailed information on what happens under the hood and help optimize system performance.
- Integrated terminal like playground to Get and Set values to/from
ResilientDB. - Real-time flame graph generation for memory and CPU profiling and detailed call stack analysis for processes.
- Custom CPP hooks integrated to provide metrics not exposed by LevelDB like cache hit ratio, storage engine metrics like number of SST tables, number of files at each level etc.
- Web-based frontend for interactive performance insights.
- Aggregating different profiling tools and packaging into a single dashboard.
- Lightweight and optimized for Linux-based systems.
- Granular Profiling: Analyze memory and CPU usage down to individual function calls.
- Flame Graph Visualization: Generate real-time flame graphs to identify bottlenecks.
- Call Stack Recording: Track and analyze call stacks during runtime.
- User-Friendly Frontend: Visualize results via a web-based interface.
- Dependency Analyzer: Check dependency graphs generated by bazel and visualize them in a easy to view way.
- Output Options: Export results in JSON, PNG, and other formats and analyze them using external tools.
Description: Flamegraph showing the call stack when a Get method is called on ResilientDB's key value store. This graph shows the function caller which is invoked by the ResilientDB-GraphQL service and is serviced at the end by the storage engine which is LevelDB. The storage engine calls the in memory LRU cache to check for cache hits and when not found calls its internal functions to retrieve the key value pair.
Todo: Add more examples
-
Operating System: Linux (e.g., Ubuntu 20.04+).
-
Announcement: We know that these are a lot of tools to install and setup. Our team is working on automating the process of installation and setup.
-
Service Dependencies:
- Install our fork ResilientDB from here https://github.com/harish876/incubator-resilientDB
- Install ResilientDB's Graphql service to expose HTTP endpoints to use the Get and Set values to the Key value store from here https://github.com/apache/incubator-resilientdb-graphql.
- Install MemLen's middleware service to aggregate data from our profiling services and expose HTTP endpoints to allow the MemLens Dashboard to access this data. https://github.com/harish876/MemLens-middleware
-
System Dependencies: These services are essential for hooking into various aspects of ResilientDB and monitoring its performance.
- Prometheus: Expose metrics collected by services like
node_exporterandprocess-exporterto the frontend. - NodeExporter: Expose system-level metrics.
- ProcessExporter: Expose process-level metrics.
- Pyroscope (v0.37.0): Provide fine-grained profiling metrics at a process level using eBPF.
- Prometheus: Expose metrics collected by services like
-
Offline Mode - Learn, Explore and familiarize with the dashboard without setting up the profiling environment.
- Clone this repository. This is the frontend Dashboard to access the profiling data collected. Running the dashboard without an accompanying backend will display data profiled and collected by the team during the time of demo.
git clone https://github.com/Bismanpal-Singh/MemLens
-
Online/Live Mode - Gauge metrics, optimise performance by looking at live metrics from the profiling environment.
- Clone this repository. This is the frontend Dashboard to access the profiling data collected.
git clone https://github.com/Bismanpal-Singh/MemLens
- Clone the ResilientDB repository. This is the service which will be profiled by MemLens. Follow the installation instructions to install ResilientDB and set it up.
git clone https://github.com/harish876/incubator-resilientDB
- Clone the ResilientDB Graphql repository. This is the service which will be used to Get and Set data from ResilientDB's key value store. Follow the installation instructions to install and setup this service.
git clone https://github.com/apache/incubator-resilientdb-graphql.
- Clone the MemLens middleware service. This is the service which will be used to aggregate data from different profiling services.
git clone https://github.com/harish876/MemLens-middleware
- Follow this step by step installation manual to install, setup, resolve errors, and configure the profiling environment. TODO ADD LINK
-
Harish Gokul (http://github.com/harish876): Lead Developer for MemLens responsible for
- Ideation, Execution and Architecture Design for the project.
- Development of CPP Hooks, integration of prometheus hooks, and third party profiling libraries.
- Development of profiling environment
- Development of MemLens middleware service
- Contributed heavily to the frontend dashboard design.
- Documentation and blog author.
-
Bismanpal Anand (http://github.com/bismanpal-singh): Designer & Developer for MemLens responsible for
- Structured and Designed the MemLens website.
- Experimented with various approaches to enhance usability while maintaining simplicity and elegance.
- Contributed to the design of the frontend dashboard, ensuring intuitive visualization of metrics and data.
- Contributed in implementing the MemLens middleware service to bridge data collection and presentation.
- Contributed heavily to the frontend dashboard design.
- Gathered memory metrics and profiling data by working with APIs & tools and integrated them into the dashboard
- We would like to acknowledge the good folks of ResView for the inspiration and a reference point to start our project.