Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/main'
Browse files Browse the repository at this point in the history
  • Loading branch information
boyugou committed Dec 18, 2024
2 parents 78620c2 + 3ef5399 commit 5068c17
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 18 deletions.
39 changes: 21 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Awesome GUI Agent Paper List
# Awesome GUI Agent Paper List: Comprehensive Paper Repository on GUI Agents.

This paper list covers a variety of papers related to GUI Agents, including but not limited to:
This paper list covers a variety of papers related to GUI Agents, such as:
- GUI Understanding
- Datasets
- Benchmarks
- New frameworks
- New models
- Vision, language, multimodal foundation models (with explicit support for GUI)
- Works in general Domains extensively used by GUI Agents (e.g., SoM prompting)
- Works in general domains extensively used by GUI Agents (e.g., SoM prompting)


<div style="display: flex; justify-content: space-around;">
Expand All @@ -17,10 +17,11 @@ This paper list covers a variety of papers related to GUI Agents, including but



## Papers Grouped by Envs
## Papers Grouped by Environments
| [Web](paper_by_env/paper_web.md) | [Mobile](paper_by_env/paper_mobile.md) | [Desktop](paper_by_env/paper_desktop.md) | [GUI](paper_by_env/paper_gui.md) | [Misc](paper_by_env/paper_misc.md) |
|--------------------------------|---------------------------------------|------------------------------------------|----------------------------------|------------------------------------|

(Misc: Papers that do not specifically study for GUIs.)

## Papers Grouped by Keywords
[Model](paper_by_key/paper_model.md) | [Framework](paper_by_key/paper_framework.md) | [Dataset](paper_by_key/paper_dataset.md) | [Benchmark](paper_by_key/paper_benchmark.md) | [Safety](paper_by_key/paper_safety.md) | [Survey](paper_by_key/paper_survey.md) | \n[UI understanding](paper_by_key/paper_UI%20understanding.md) | [Evaluation](paper_by_key/paper_evaluation.md) | [Fine-tuning](paper_by_key/paper_fine-tuning.md) | [Foundation model](paper_by_key/paper_foundation%20model.md) | [Grounding](paper_by_key/paper_grounding.md) | [Learning](paper_by_key/paper_learning.md) | [Planning](paper_by_key/paper_planning.md) | [Programming-by-demonstration](paper_by_key/paper_programming-by-demonstration.md) | [Reasoning](paper_by_key/paper_reasoning.md) | [Reinforcement learning](paper_by_key/paper_reinforcement%20learning.md) | [Synthetic data](paper_by_key/paper_synthetic%20data.md) | [Vision-language model](paper_by_key/paper_vision-language%20model.md) | [Visual grounding](paper_by_key/paper_visual%20grounding.md) | [Web-based tasks](paper_by_key/paper_web-based%20tasks.md)
Expand Down Expand Up @@ -1563,13 +1564,16 @@ This paper list covers a variety of papers related to GUI Agents, including but



## How to Add a Paper

<details>
<summary>format example</summary>

## How to Add a Paper or Update the README

Please fork and update:
- [paper list](update_template_or_data/update_paper_list.md)
- [README template](update_template_or_data/update_readme_template.md)
- [automatic workflow](.github/workflows/main.yml)

You can use [this GPTs](https://chatgpt.com/g/g-VqW9ONrgL-gui-paper-list) to quickly search and get a formatted paper entry automatically.
<details>
<summary>format example and explanation</summary>

```
- [title](paper link)
Expand All @@ -1583,18 +1587,17 @@ This paper list covers a variety of papers related to GUI Agents, including but
```

Regarding the 🔑 Key:
- model: Indicates a newly trained model.
- framework: If the paper proposes a new framework.
- dataset: If a new dataset is created and published.
- benchmark: If a new benchmark is established (add "dataset" if there's a new training set).
- primary Innovation: List the main focus or innovation in the study.
- Common Abbreviations: Include commonly used abbreviations associated with the paper. (model names, framework names, etc.)

| Key | Definition |
|-----------------|---------------------------------------------------------------------------------------|
| model | Indicates a newly trained model. |
| framework | If the paper proposes a new agent framework. |
| dataset | If a new (training) dataset is created and published. |
| benchmark | If a new benchmark is established (also add "dataset" if there's a new training set). |
| primary studies | List the main focus or innovation in the study. |
| Abbreviations | Include commonly used abbreviations associated with the paper (model names, framework names, etc.). |
For missing information, use "Unknown."

You can contribute by providing either the paper title or a fully formatted entry in [Paper Collection](https://github.com/boyugou/GUI-Agents-Paper-List/issues/1). You’re also welcome to open a new PR with your submission.

For ease of use, you can use [this GPTs](https://chatgpt.com/g/g-VqW9ONrgL-gui-paper-list) to help search your paper and format the entry automatically. (prompts in auto_prompt.txt)


</details>
Expand Down
Binary file modified update_template_or_data/statistics/keyword_wordcloud.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 5068c17

Please sign in to comment.