polish(pu): polish readme using colorful icons

CHANGELOG.md

@@ -1,3 +1,12 @@
+2024.07.12 (v0.1.0)
+- env: SumToThree env from pooltool(#227)
+- algo: UniZero (#232)
+- algo: ReZero (#238)
+- feature: add logs and config documentations (#220)
+- polish: polish atari_env_action_space_map, fix test_muzero_game_buffer
+- polish: polish release.yml
+- style: update discord link and add a new badge in readme (#221)
+
 2024.04.12 (v0.0.5)
 - env: MemoryEnv (#197)
 - env: mountain_car (#181)

README.md

@@ -28,14 +28,15 @@
 [![GitHub license](https://img.shields.io/github/license/opendilab/LightZero)](https://github.com/opendilab/LightZero/blob/master/LICENSE)
 [![discord badge](https://dcbadge.vercel.app/api/server/dkZS2JF56X?style=flat)](https://discord.gg/dkZS2JF56X)
 
-Updated on 2024.06.19 LightZero-v0.0.5
+Updated on 2024.07.12 LightZero-v0.1.0
+
+English | [简体中文(Simplified Chinese)](https://github.com/opendilab/LightZero/blob/main/README.zh.md) | [LightZero Paper](https://arxiv.org/abs/2310.08348) | [🔥UniZero Paper](https://arxiv.org/abs/2406.10667) | [🔥ReZero Paper](https://arxiv.org/abs/2404.16364)
 
 > LightZero is a lightweight, efficient, and easy-to-understand open-source algorithm toolkit that combines Monte Carlo Tree Search (MCTS) and Deep Reinforcement Learning (RL).
 > For any questions about LightZero, you can consult the RAG-based Q&A assistant: [ZeroPal](https://huggingface.co/spaces/OpenDILabCommunity/ZeroPal).
 
-English | [简体中文(Simplified Chinese)](https://github.com/opendilab/LightZero/blob/main/README.zh.md) | [LightZero Paper](https://arxiv.org/abs/2310.08348) | [🔥UniZero Paper](https://arxiv.org/abs/2406.10667) | [🔥ReZero Paper](https://arxiv.org/abs/2404.16364)
 
-## Background
+## 🔍 Background
 
 The integration of Monte Carlo Tree Search and Deep Reinforcement Learning,
 exemplified by AlphaZero and MuZero,
@@ -44,9 +45,9 @@ This advanced methodology has also made significant strides in scientific domain
 The following is an overview of the historical evolution of the Monte Carlo Tree Search algorithm series:
 ![pipeline](assets/mcts_rl_evolution_overview.png)
 
-## Overview
+## 🎨 Overview
 
-**LightZero** is an open-source algorithm toolkit that combines MCTS and RL for PyTorch. It provides support for a range of MCTS-based RL algorithms and applications with the following advantages:
+**LightZero** is an open-source algorithm toolkit that combines Monte Carlo Tree Search (MCTS) and Reinforcement Learning (RL) for PyTorch. It supports a range of MCTS-based RL algorithms and applications, offering several key advantages:
 - Lightweight.
 - Efficient.
 - Easy-to-understand.
@@ -64,6 +65,7 @@ For further details, please refer to [Features](#features), [Framework Structure
   - [Integrated Algorithms](#integrated-algorithms)
 - [Installation](#installation)
 - [Quick Start](#quick-start)
+- [Documentation](#documentation)
 - [Benchmark](#benchmark)
 - [Awesome-MCTS Notes](#awesome-mcts-notes)
   - [Paper Notes](#paper-notes)
@@ -76,15 +78,15 @@ For further details, please refer to [Features](#features), [Framework Structure
 - [Acknowledgments](#acknowledgments)
 - [License](#license)
 
-### Features
+### 💥 Features
 
 **Lightweight**: LightZero integrates multiple MCTS algorithm families and can solve decision-making problems with various attributes in a lightweight framework. The algorithms and environments LightZero implemented can be found [here](#integrated-algorithms).
 
 **Efficient**: LightZero uses mixed heterogeneous computing programming to improve computational efficiency for the most time-consuming part of MCTS algorithms.
 
 **Easy-to-understand**: LightZero provides detailed documentation and algorithm framework diagrams for all integrated algorithms to help users understand the algorithm's core and compare the differences and similarities between algorithms under the same paradigm. LightZero also provides function call graphs and network structure diagrams for algorithm code implementation, making it easier for users to locate critical code. All the documentation can be found [here](#paper-notes).
 
-### Framework Structure
+### 🧩 Framework Structure
 
 [comment]: <> (<p align="center">)
 
@@ -111,7 +113,7 @@ The above picture is the framework pipeline of LightZero. We briefly introduce t
 
 For the file structure of LightZero, please refer to [lightzero_file_structure](https://github.com/opendilab/LightZero/blob/main/assets/lightzero_file_structure.svg).
 
-### Integrated Algorithms
+### 🎁 Integrated Algorithms
 LightZero is a library with a [PyTorch](https://pytorch.org/) implementation of MCTS algorithms (sometimes combined with cython and cpp), including:
 - [AlphaZero](https://www.science.org/doi/10.1126/science.aar6404)
 - [MuZero](https://arxiv.org/abs/1911.08265)
@@ -150,7 +152,7 @@ The environments and algorithms currently supported by LightZero are shown in th
 <sup>(3): "---" means that this algorithm doesn't support this environment.</sup>
 
 
-## Installation
+## ⚙️ Installation
 
 You can install the latest LightZero in development from the GitHub source codes with the following command:
 
@@ -164,7 +166,7 @@ Kindly note that LightZero currently supports compilation only on `Linux` and `m
 We are actively working towards extending this support to the `Windows` platform. 
 Your patience during this transition is greatly appreciated.
 
-## Installation with Docker
+### Installation with Docker
 
 We also provide a Dockerfile that sets up an environment with all dependencies needed to run the LightZero library. This Docker image is based on Ubuntu 20.04 and installs Python 3.8, along with other necessary tools and libraries.
 Here's how to use our Dockerfile to build a Docker image, run a container from this image, and execute LightZero code inside the container.
@@ -190,7 +192,7 @@ Here's how to use our Dockerfile to build a Docker image, run a container from t
 
 [comment]: <> (- [AlphaGo Zero]&#40;https://www.nature.com/articles/nature24270&#41; )
 
-## Quick Start
+## 🚀 Quick Start
 
 Train a MuZero agent to play [CartPole](https://gymnasium.farama.org/environments/classic_control/cart_pole/):
 
@@ -213,7 +215,7 @@ cd LightZero
 python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
 ```
 
-## Customization Documentation
+## 📚 Documentation
 
 For those interested in customizing environments and algorithms, we provide relevant guides:
 
@@ -224,9 +226,9 @@ For those interested in customizing environments and algorithms, we provide rele
 
 Should you have any questions, feel free to contact us for support.
 
-## Benchmark
+## 📊 Benchmark
 
-<details open><summary>Click to collapse</summary>
+<details><summary>Click to expand</summary>
 
 - Below are the benchmark results of [AlphaZero](https://github.com/opendilab/LightZero/blob/main/lzero/policy/alphazero.py) and [MuZero](https://github.com/opendilab/LightZero/blob/main/lzero/policy/muzero.py) on three board games: [TicTacToe](https://github.com/opendilab/LightZero/blob/main/zoo/board_games/tictactoe/envs/tictactoe_env.py), [Connect4](https://github.com/opendilab/LightZero/blob/main/zoo/board_games/connect4/envs/connect4_env.py), [Gomoku](https://github.com/opendilab/LightZero/blob/main/zoo/board_games/gomoku/envs/gomoku_env.py).
 <p align="center">
@@ -281,7 +283,7 @@ and two MuJoCo continuous action space games: [Hopper-v3](https://github.com/ope
 </details>
 
 
-## Awesome-MCTS Notes
+## 📝 Awesome-MCTS Notes
 
 ### Paper Notes
 The following are the detailed paper notes (in Chinese) of the above algorithms:
@@ -523,7 +525,7 @@ Here is a collection of research papers about **Monte Carlo Tree Search**.
 </details>
 
 
-## Feedback and Contribution
+## 💬 Feedback and Contribution
 
 - [File an issue](https://github.com/opendilab/LightZero/issues/new/choose) on Github
 - Open or participate in our [discussion forum](https://github.com/opendilab/LightZero/discussions)
@@ -537,7 +539,7 @@ Here is a collection of research papers about **Monte Carlo Tree Search**.
 [comment]: <> (And `CONTRIBUTING.md` offers some necessary information.)
 
 
-## Citation
+## 🌏 Citation
 ```latex
 @article{niu2024lightzero,
   title={LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios},
@@ -562,7 +564,7 @@ Here is a collection of research papers about **Monte Carlo Tree Search**.
 }
 ```
 
-## Acknowledgments
+## 💓 Acknowledgments
 
 This project has been developed partially based on the following pioneering works on GitHub repositories.
 We express our profound gratitude for these foundational resources:
@@ -580,7 +582,7 @@ Thanks to all who contributed to this project:
 </a>
 
 
-## License
+## 🏷️ License
 All code within this repository is under [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
 
 <p align="right">(<a href="#top">Back to top</a>)</p>

README.zh.md

@@ -27,20 +27,21 @@
 [![Contributors](https://img.shields.io/github/contributors/opendilab/LightZero)](https://github.com/opendilab/LightZero/graphs/contributors)
 [![GitHub license](https://img.shields.io/github/license/opendilab/LightZero)](https://github.com/opendilab/LightZero/blob/master/LICENSE)
 
-最近更新于 2024.06.19 LightZero-v0.0.5
+最近更新于 2024.07.12 LightZero-v0.1.0
+
+[English](https://github.com/opendilab/LightZero/blob/main/README.md) | 简体中文 | [LightZero 论文](https://arxiv.org/abs/2310.08348) | [🔥UniZero 论文](https://arxiv.org/abs/2406.10667) | [🔥ReZero 论文](https://arxiv.org/abs/2404.16364)
 
 > LightZero 是一个轻量、高效、易懂的 MCTS+RL 开源算法库。
 > 有关 LightZero 的任何疑问，您都可以咨询基于 RAG 技术的问答助手：[ZeroPal](https://huggingface.co/spaces/OpenDILabCommunity/ZeroPal)。
 
-[English](https://github.com/opendilab/LightZero/blob/main/README.md) | 简体中文 | [LightZero 论文](https://arxiv.org/abs/2310.08348) | [🔥UniZero 论文](https://arxiv.org/abs/2406.10667) | [🔥ReZero 论文](https://arxiv.org/abs/2404.16364)
 
 
-## 背景
+## 🔍 背景
 
 以 AlphaZero, MuZero 为代表的结合蒙特卡洛树搜索 (Monte Carlo Tree Search, MCTS) 和深度强化学习 (Deep Reinforcemeent Learning, DRL) 的方法，在诸如围棋，Atari 等各种游戏上取得了超人的水平，也在诸如蛋白质结构预测，矩阵乘法算法寻找等科学领域取得了可喜的进展。下图为蒙特卡洛树搜索（MCTS）算法族的发展历史：
 ![pipeline](assets/mcts_rl_evolution_overview.png)
 
-## 概览
+## 🎨 概览
 
 **LightZero** 是一个结合了蒙特卡洛树搜索和强化学习的开源算法工具包。 它支持一系列基于 MCTS 的 RL 算法，具有以下优点：
 - 轻量。
@@ -59,6 +60,7 @@
     - [集成算法](#集成算法)
 - [安装方法](#安装方法)
 - [快速开始](#快速开始)
+- [文档](#文档)
 - [基线算法比较](#基线算法比较)
 - [MCTS相关笔记](#MCTS-相关笔记)
     - [论文笔记](#论文笔记)
@@ -71,14 +73,14 @@
 - [致谢](#致谢)
 - [许可证](#许可证)
 
-### 特点
+### 💥 特点
 **轻量**：LightZero 中集成了多种 MCTS 族算法，能够在同一框架下轻量化地解决多种属性的决策问题。
 
 **高效**：LightZero 针对 MCTS 族算法中耗时最长的环节，采用混合异构计算编程提高计算效率。
 
 **易懂**：LightZero 为所有集成的算法提供了详细文档和算法框架图，帮助用户理解算法内核，在同一范式下比较算法之间的异同。同时，LightZero 也为算法的代码实现提供了函数调用图和网络结构图，便于用户定位关键代码。
 
-### 框架结构
+### 🧩 框架结构
 
 <p align="center">
   <img src="assets/lightzero_pipeline.svg" alt="Image Description 2" width="50%" height="auto" style="margin: 0 1%;">
@@ -98,7 +100,7 @@
 
 关于 LightZero 的文件结构，请参考 [lightzero_file_structure](https://github.com/opendilab/LightZero/blob/main/assets/lightzero_file_structure.svg)。
 
-### 集成算法
+### 🎁 集成算法
 LightZero 是基于 [PyTorch](https://pytorch.org/) 实现的 MCTS 算法库，在 MCTS 的实现中也用到了 cython 和 cpp。同时，LightZero 的框架主要基于 [DI-engine](https://github.com/opendilab/DI-engine) 实现。目前 LightZero 中集成的算法包括：
 - [AlphaZero](https://www.science.org/doi/10.1126/science.aar6404)
 - [MuZero](https://arxiv.org/abs/1911.08265)
@@ -135,7 +137,7 @@ LightZero 目前支持的环境及算法如下表所示：
 
 <sup>(3): "---" 表示该算法不支持此环境。</sup>
 
-## 安装方法
+## ⚙️ 安装方法
 
 可以用以下命令从 Github 的源码中安装最新版的 LightZero：
 
@@ -174,7 +176,7 @@ pip3 install -e .
     python ./LightZero/zoo/classic_control/cartpole/config/cartpole_muzero_config.py
     ```
 
-## 快速开始
+## 🚀 快速开始
 使用如下代码在 [CartPole](https://gymnasium.farama.org/environments/classic_control/cart_pole/) 环境上快速训练一个 MuZero 智能体:
 
 ```bash
@@ -195,7 +197,7 @@ python3 -u zoo/atari/config/atari_muzero_config.py
 cd LightZero
 python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
 ```
-## 文档
+## 📚 文档
 
 为希望定制环境和算法的用户，我们提供了相应的指南：
 
@@ -204,11 +206,11 @@ python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
 - [如何设置配置文件？](https://github.com/opendilab/LightZero/blob/main/docs/source/tutorials/config/config_zh.md)
 - [日志系统](https://github.com/opendilab/LightZero/blob/main/docs/source/tutorials/logs/logs_zh.md)
 
-如有任何疑问，欢迎随时联系我们寻求帮助。
+如有任何疑问，欢迎随时联系我们。
 
-## 基线算法比较
+## 📊 基线算法比较
 
-<details open><summary>点击折叠</summary>
+<details><summary>点击查看</summary>
 
 - [AlphaZero](https://github.com/opendilab/LightZero/blob/main/lzero/policy/alphazero.py) 和 [MuZero](https://github.com/opendilab/LightZero/blob/main/lzero/policy/muzero.py) 在3个棋类游戏（[TicTacToe (井字棋)](https://github.com/opendilab/LightZero/blob/main/zoo/board_games/tictactoe/envs/tictactoe_env.py)，[Connect4](https://github.com/opendilab/LightZero/blob/main/zoo/board_games/connect4/envs/connect4_env.py) 和 [Gomoku (五子棋)](https://github.com/opendilab/LightZero/blob/main/zoo/board_games/gomoku/envs/gomoku_env.py)）上的基线结果：
 <p align="center">
@@ -261,7 +263,7 @@ python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
 
 </details>
 
-## MCTS 相关笔记
+## 📝 MCTS 相关笔记
 
 ### 论文笔记
 
@@ -292,7 +294,7 @@ python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
 以下是 LightZero 中集成算法的框架概览图：
 
 <details closed>
-<summary>(点击查看更多)</summary>
+<summary>(点击查看)</summary>
 
 [MCTS](https://github.com/opendilab/LightZero/blob/main/assets/algo_overview/mcts_overview.pdf)
 
@@ -315,7 +317,7 @@ python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
 ### 重要论文
 
 <details closed>
-<summary>(点击查看更多)</summary>
+<summary>(点击查看)</summary>
 
 #### LightZero Implemented series
 
@@ -359,7 +361,7 @@ python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
 ### 其他论文
 
 <details closed>
-<summary>(点击查看更多)</summary>
+<summary>(点击查看)</summary>
 
 #### ICML
 - [Scalable Safe Policy Improvement via Monte Carlo Tree Search](https://openreview.net/pdf?id=tevbBSzSfK) 2023
@@ -519,7 +521,7 @@ and internal state transition dynamics,
 - [Sample-Efficient Neural Architecture Search by Learning Actions for Monte Carlo Tree Search](https://arxiv.org/pdf/1906.06832) IEEE Transactions on Pattern Analysis and Machine Intelligence 2022.
 </details>
 
-## 反馈意见和贡献
+## 💬 反馈意见和贡献
 - 有任何疑问或意见都可以在 github 上直接 [提出 issue](https://github.com/opendilab/LightZero/issues/new/choose)
 - 开启或参加 [GitHub 论坛](https://github.com/opendilab/LightZero/discussions)
 - 在 LightZero [discord server](https://discord.gg/qZTQTycu) 上进行讨论
@@ -528,7 +530,7 @@ and internal state transition dynamics,
 - 感谢所有的反馈意见，包括对算法和系统设计。这些反馈意见和建议都会让 LightZero 变得更好。 
 
 
-## 引用
+## 🌏 引用
 
 ```latex
 @article{niu2024lightzero,
@@ -554,7 +556,7 @@ and internal state transition dynamics,
 }
 ```
 
-## 致谢
+## 💓 致谢
 此算法库的实现部分基于以下 GitHub 仓库，非常感谢这些开创性工作：
 - https://github.com/opendilab/DI-engine
 - https://github.com/deepmind/mctx
@@ -569,7 +571,7 @@ and internal state transition dynamics,
 <img src="https://contrib.rocks/image?repo=opendilab/LightZero" />
 </a>
 
-## 许可证
+## 🏷️ 许可证
 
 本仓库中的所有代码都符合 [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)。