Update readme

README.md

@@ -1,15 +1,18 @@
 # Market creator
 
-Market creator (market maker) is an autonomous service created with the [Open Autonomy framework](https://docs.autonolas.network/open-autonomy/) that processes worldwide news using an LLM and opens prediction markets on the Gnosis chain. The service roughly works as follows:
+Market Creator (or Market Maker) is an autonomous service that interacts with news providers and LLMs to **create prediction markets** on the [Omen](https://aiomen.eth.limo/) platform (Gnosis chain). The workflow of the service is as follows:
 
 1. Gather headlines and summaries of recent news through a third-party provider.
 2. Interact with an LLM (using the gathered information in the previous step) to obtain a collection of suitable questions to open prediction markets associated to future events.
-3. Propose questions to a market approval service endpoint. Users manually approve suitable markets using that endpoint.
-4. Collect user-approved markets from the market approval service.
-5. Send the necessary transactions to the Gnosis chain to open and fund the chosen prediction market.
-6. Repeat steps 1-5. When `NUM_MARKETS` (configurable) have been created, the service will cycle in a waiting state.
+3. Propose the generated markets to a market approval server (a maximum of `NUM_MARKETS` are proposed). Users interact with this server, review and approve/reject the proposed markets.
+4. Collect an user-approved market from the server.
+5. Send the necessary transactions to the Gnosis chain to open and fund the market on [Omen](https://aiomen.eth.limo/).
+6. Remove liquidity of markets whose closing date is <= 1 day.
+7. Repeat.
 
-## Developers
+The Market Creator service is an [agent service](https://docs.autonolas.network/open-autonomy/get_started/what_is_an_agent_service/) (or autonomous service) based on the [Open Autonomy framework](https://docs.autonolas.network/open-autonomy/). Below we show you how to prepare your environment, how to prepare the agent keys, and how to configure and run the service.
+
+## Prepare the environment
 
 - System requirements:
 
@@ -19,117 +22,163 @@ Market creator (market maker) is an autonomous service created with the [Open Au
   - [Docker Engine](https://docs.docker.com/engine/install/)
   - [Docker Compose](https://docs.docker.com/compose/install/)
 
-- Clone the repository:
+- Clone this repository:
 
       git clone https://github.com/valory-xyz/market-creator.git
 
 - Create development environment:
 
       poetry install && poetry shell
 
-- Configure command line:
+- Configure the Open Autonomy framework:
 
       autonomy init --reset --author valory --remote --ipfs --ipfs-node "/dns/registry.autonolas.tech/tcp/443/https"
 
-- Pull packages:
+- Pull packages required to run the service:
 
       autonomy packages sync --update-packages
 
-## Market maker runtime parameters
+## Prepare the keys and the Safe
 
-Market maker has several configurable parameters for creating markets. These parameters can be configured at agent level (i.e., when running the code as a single agent) in the file [`packages/valory/agents/market_maker/aea-config.yaml`](https://github.com/valory-xyz/market-creator/blob/main/packages/valory/agents/market_maker/aea-config.yaml) ); and at service level (i.e., when running the code as a service composed of agent(s)) in the file [`packages/valory/services/market_maker/service.yaml`](https://github.com/valory-xyz/market-creator/blob/main/packages/valory/services/market_maker/service.yaml)).
+You need a **Gnosis keypair** and a **[Safe](https://safe.global/) address** to run the service.
 
-- `num_markets`: Number of markets an agent to allowed to create before, default is 1
-- `market_fee`: Fees for creating a market, default is 2 unit (Eth, xDAI, etc...)
-- `initial_funds`: Initial funds for the market, default is 1 unit (Eth, xDAI, WxDAI, etc...)
-- `market_timeout`: Maximum time allowed for finalizing answer on `realitio`, default is 1 day
-- `minimum_market_time`: Minimum time for which the market should be active after opening, default is 7 days
-- `realitio_contract`: Address of the realitio oracle contract, [default](https://gnosisscan.io/address/0x79e32aE03fb27B07C89c0c568F80287C01ca2E57/)
-- `realitio_oracle_proxy_contract`: Address of the realitio oracle proxy contract, [default](https://gnosisscan.io/address/0x2bf1BFb0eB6276a4F4B60044068Cb8CdEB89f79B/)
-- `conditional_tokens_contract`: Address of the conditional tokens that are going to be used, [default](https://gnosisscan.io/address/0xCeAfDD6bc0bEF976fdCd1112955828E00543c0Ce/)
-- `fpmm_deterministic_factory_contract`: Address of the fixed product marker maker contract, [default](https://gnosisscan.io/address/0x9083A2B699c0a4AD06F63580BDE2635d26a3eeF0)
-- `collateral_tokens_contract`: Address of the collateral token to be used for market, default is [WxDAI](https://gnosisscan.io/address/0xe91d153e0b41518a2ce8dd3d7944fa863463a97d)
-- `arbitrator_contract`: Address of the arbitration provider contract, default is [kleros](https://gnosisscan.io/address/0xe40DD83a262da3f56976038F1554Fe541Fa75ecd)
+First, prepare the `keys.json` file with the Gnosis keypair of your agent. (Replace the uppercase placeholders below):
 
-The market maker agent is configured to work with the Gnosis chain by default, if you want to use the agent with other chains you can figure out what contracts to use from [here](https://github.com/protofire/omen-exchange/blob/a98fff28a71fa53b43e7ae069924564dd597d9ba/README.md)
+    cat > keys.json << EOF
+    [
+    {
+        "address": "YOUR_AGENT_ADDRESS",
+        "private_key": "YOUR_AGENT_PRIVATE_KEY"
+    }
+    ]
+    EOF
 
-## Testing a single agent locally
+Next, prepare the [Safe](https://safe.global/). The trader agent runs as part of a **trader service**, 
+which is an [autonomous service](https://docs.autonolas.network/open-autonomy/get_started/what_is_an_agent_service/) 
+represented on-chain in the [Autonolas Protocol](https://docs.autonolas.network/protocol/) by a [Safe](https://safe.global/) multisig. Follow the next steps to obtain a **Safe address** corresponding to your agent address:
 
-Run a Tendermint node using
+1. Visit https://registry.olas.network/services/mint and connect to the Gnosis network. We recommend connecting using a wallet with a Gnosis EOA account that you own.
+2. Fill in the following fields:
+    - *"Owner address"*: a Gnosis address for which you will be able to sign later using a supported wallet. If you want to use the address you are connected to, click on *"Prefill Address"*.
+    - Click on *"Generate Hash & File"* and enter the value corresponding to the `service/valory/trader/0.1.0` key in [`packages.json`](https://github.com/valory-xyz/trader/blob/main/packages/packages.json)
+    - *"Canonical agent Ids"*: enter the number `12`
+    - *"No. of slots to canonical agent Ids"*: enter the number `1`
+    - *"Cost of agent instance bond (wei)"*: enter the number `10000000000000000`
+    - *"Threshold"*: enter the number `1`
+3. Press the *"Submit"* button. Your wallet will ask you to approve the transaction. Once the transaction is settled, you should see a message indicating that the service NFT has been minted successfully. You should also see that the service is in *Pre-Registration* state.
+4. Next, you can navigate to https://registry.olas.network/services#my-services, select your service and go through the steps:
+    1. Activate registration
+    2. Register agents: **here, you must use your agent address**.
+    3. This is the last step. A transaction for the Safe deployment is already prepared and needs to be executed.
+5. After completing the process you should see that your service is **Deployed**, and you will be able to retrieve your **Safe contract address** as shown in the image below:
 
-```bash
-bash run_tm.sh
-```
+<img src="/img/safe_address_screenshot.png" alt="Safe address field" width="500"/>
 
-and in a separate terminal, run
+**You need to provide some funds both to your agent address (xDAI) and to the Safe address (Wrapped xDAI) in order to open prediction markets in Omen.**
 
-```bash
-bash run_agent.sh
-```
+## Configure the service
 
-Now running an agent this way, will not create any market since the agent depends on gnosis multisig contract to execute the final transactions which actually creates the agent. So if you want to run an agent end 2 end you will require a gnosis multisig safe with 1 registered member. To do this run following in your terminal
+Set up the following environment variables, which will modify the performance of the trading agent. **Please read their description below**. We provide some defaults, but feel free to experiment with different values. Note that you need to provide `YOUR_AGENT_ADDRESS` and `YOUR_SAFE_ADDRESS` from the section above.
 
 ```bash
-aea generate-key ethereum
+export ETHEREUM_LEDGER_RPC=INSERT_YOUR_RPC
+export ETHEREUM_LEDGER_CHAIN_ID=100
+
+export ALL_PARTICIPANTS='["YOUR_AGENT_ADDRESS"]'
+export SAFE_CONTRACT_ADDRESS="YOUR_SAFE_ADDRESS"
+
+export NEWSAPI_ENDPOINT=https://newsapi.org/v2/everything
+export NEWSAPI_API_KEY=YOUR_NEWSAPI_API_KEY
+export OPENAI_API_KEY=YOUR_OPENAI_API_KEY
+export ENGINE="gpt-4"
+export MARKET_APPROVAL_SERVER_URL=YOUR_MARKET_APPROVAL_SERVER_URL
+export MARKET_APPROVAL_SERVER_API_KEY=YOUR_MARKET_APPROVAL_SERVER_API_KEY
+
+export NUM_MARKETS=10
+export TOPICS='["business","science","technology","politics","arts","weather"]'
+export MARKET_FEE=2
+export INITIAL_FUNDS=1
+export MINIMUM_MARKET_TIME=1
+export MARKET_TIMEOUT=1
+export MARKET_IDENTIFICATION_PROMPT=$(sed -e ':a' -e 'N' -e '$!ba' \
+  -e 's/"/\\"/g' \
+  -e "s/'/\\\'/g" \
+  -e 's/:/;/g' \
+  -e 's/\n/\\n/g' \
+  market_identification_prompt.txt)
 ```
 
-This will generate a ethereum private in your working directory in a file named `ethereum_private_key.txt`. You can add some funds to this key and create a gnosis multisig using their [app](https://app.safe.global/welcome). Once you create a multisig, update the [safe_contract_address](https://github.com/valory-xyz/market-creator/blob/0bab9ff6b41c2f024cc1f0d2aa149347fd0f47a9/packages/valory/agents/market_maker/aea-config.yaml#L149) parameter on the `aea-config.yaml` and use the `ethereum_private_key.txt` to run the agent using the script mentioned above. 
+These are the descriptions of the variables used by the Market Creator service. If you do not define them, they will take their default values:
+
+- `ETHEREUM_LEDGER_RPC`: RPC endpoint for the agent (you can get an RPC endpoint, e.g. [here](https://getblock.io/)).
+- `ETHEREUM_LEDGER_CHAIN_ID`: identifier of the chain on which the service is running (Gnosis=100).
+- `ALL_PARTICIPANTS`: list of all the agent addresses participating in the service. In this example we only are using a single agent.
+- `SAFE_CONTRACT_ADDRESS`: address of the agents multisig wallet created [in the previous section](#prepare-the-keys-and-the-safe).
+- `NEWSAPI_ENDPOINT`: [Newsapi](https://newsapi.org/) endpoint to retrieve recent news headlines and summaries.
+- `NEWSAPI_API_KEY`: Your [Newsapi](https://newsapi.org/) API key. You can get one for testing purposes for free.
+- `OPENAI_API_KEY`: Your [OpenAI](https://openai.com/) API key. You can get one for testing purposes for free.
+- `ENGINE`: [OpenAI](https://openai.com/) engine. Default (and recommended) is `gpt-4`.
+- `MARKET_APPROVAL_SERVER_URL`: Your market approval server URL. It must be publicly accessible. [See below](#launch-the-market-approval-server).
+- `MARKET_APPROVAL_SERVER_API_KEY`: Your market approval server API key. [See below](#launch-the-market-approval-server)
+- `NUM_MARKETS`: Number of markets that the service will propose to the market approval server. Default is 1.
+- `TOPICS`: Topics to create the markets.
+- `MARKET_FEE`: % liquidity provider fees on the market. Default is 2%.
+- `INITIAL_FUNDS`: Initial liquidity funds for the market, in cents. Default is 1 cent (0.01 Eth, xDAI, WxDAI, etc...).
+- `MINIMUM_MARKET_TIME`: Minimum time for which the proposed markets should be active after opening. Default is 7 days.
+- `MARKET_TIMEOUT`: How long people will have to correct incorrect responses after they are posted on [reality.eth](https://realityeth.github.io/). Default is 1 day.
+- `MARKET_IDENTIFICATION_PROMPT`: Prompt to create the market proposals on the market approval server. As defined above, it reads the contents of the file `market_identification_prompt.txt`, which you can modify and experiment with different prompts You must include the placeholders `{event_date}`, `{input_news}` and `{topics}` in the prompt.
+
+## Launch the market approval server
+
+The service requires to interact with a *market approval server*, which must be publicly accessible from the Internet. The workflow is as follows:
+
+1. The Market Creator service proposes markets to the market approval server.
+2. Users approve/reject the proposed markets (using `curl` commands).
+3. The Market Creator service picks a random approved market from the server and creates the market in Omen.
+
+To launch your owns instance of the market approval server:
+
+- Define an arbitrary API key, and compute its SHA-256 hash as a hex string.
+- Edit the file [market_approval_server.py](https://github.com/valory-xyz/market-creator/blob/main/market_approval_server/market_approval_server.py) and add the computed hash to the dictionary `DEFAULT_API_KEYS`.
+- Run the file [market_approval_server.py](https://github.com/valory-xyz/market-creator/blob/main/market_approval_server/market_approval_server.py) in a server of your choice. Ensure it is publicly accessible from the Internet.
 
-Also make sure your multisig safe account holds some amount of the tokens which you're planning on using as collateral. By default the agent uses `WxDAI` as collateral.
+You can access the root address http://server_ip:5000 and examine the different commands you can interact with the server. Please, note that it runs on an unsecured http connection by default.
 
-## Testing the service against Gnosis Mainnet
+## Run the service
 
-- Prepare the agent keys:
+- Fetch the service:
 
     ```bash
-    cat > keys.json << EOF
-    [
-    {
-        "address": "<your_agent_address>",
-        "private_key": "<your_agent_private_key>"
-    }
-    ]
-    EOF
+    autonomy fetch --local --service valory/market_maker && cd market_maker
     ```
 
-
-- Prepare an .env file with the following environment variables.
-    Note that if you do not specify an environment variable, it will take its default value from the service configuration file [`packages/valory/services/market_maker/service.yaml`](https://github.com/valory-xyz/market-creator/blob/main/packages/valory/services/market_maker/service.yaml).
+- Build the Docker image:
 
     ```bash
-    NUM_MARKETS=1
-    NEWSAPI_ENDPOINT=https://newsapi.org/v2/everything
-    NEWSAPI_API_KEY=<your_news_api_key>
-    MARKET_FEE=1
-    INITIAL_FUNDS=1
-    MARKET_TIMEOUT=1
-    REALITIO_CONTRACT=0x79e32aE03fb27B07C89c0c568F80287C01ca2E57
-    REALITIO_ORACLE_PROXY_CONTRACT=0xab16d643ba051c11962da645f74632d3130c81e2
-    CONDITIONAL_TOKENS_CONTRACT=0xCeAfDD6bc0bEF976fdCd1112955828E00543c0Ce
-    FPMM_DETERMINISTIC_FACTORY_CONTRACT=0x9083A2B699c0a4AD06F63580BDE2635d26a3eeF0
-    COLLATERAL_TOKENS_CONTRACT=0xe91d153e0b41518a2ce8dd3d7944fa863463a97d
-    ARBITRATOR_CONTRACT=0xe40dd83a262da3f56976038f1554fe541fa75ecd
-    MARKET_APPROVAL_SERVER_URL=<market_approval_server_url>
-    MARKET_APPROVAL_SERVER_API_KEY=<market_approval_server_api_key>
-    MULTISEND_ADDRESS=<multisend_address>
-    OPENAI_API_KEY=<your_openai_api_key>
-    ETHEREUM_LEDGER_RPC=<chain_rpc_endpoint>
-    ETHEREUM_LEDGER_CHAIN_ID=<chain_id>
-    ALL_PARTICIPANTS='["<your_agent_address>"]'
-    RESET_PAUSE_DURATION=10
-
-    # The directory specified in KEY_DIR must contain a keys.json file
-    # with the addresses and private keys of the agents.
-    KEY_DIR=<path_to_the_folder_containing_your_keys>
-
-    # Topics to create questions 
-    TOPICS='["business","science","technology","politics","arts","weather"]'
+    autonomy build-image
     ```
 
-- Edit the file `market_identification_prompt.txt`. You must include the placeholders `{input_news}` and `{topics}` in the prompt.
+- Copy your `keys.json` file prepared [in the previous section](#prepare-the-keys-and-the-safe) in the same directory:
 
-- Build and run the service:
+    ```bash
+    cp path/to/keys.json .
+    ```
+
+- Build the deployment with a single agent and run:
 
     ```bash
-    bash run_service.sh
+    autonomy deploy build --n 1 -ltm
+    autonomy deploy run --build-dir abci_build/
     ```
+
+For convenience, we provide a template script [run_service.sh](https://github.com/valory-xyz/market-creator/blob/main/run_service.sh) that you can modify and experiment with different values.
+
+## For advanced users
+
+The market maker agent is configured to work with the Gnosis chain by default, if you want to use the agent with other chains you can figure out what contracts to use from [here](https://github.com/protofire/omen-exchange/blob/a98fff28a71fa53b43e7ae069924564dd597d9ba/README.md)
+
+You can explore the [`service.yaml`](https://github.com/valory-xyz/market-creator/blob/main/packages/valory/services/market_maker/service.yaml)) file, which contains all the possible configuration variables for the service.
+
+The Safe of the service holds the collateral token used to provide the initial liquidity to the markets created. By default the service uses `WxDAI` as collateral. This is configured through the environment variable `COLLATERAL_TOKEN_CONTRACT`, which points to the address of the collateral token to be used for market. The default is [WxDAI](https://gnosisscan.io/address/0xe91d153e0b41518a2ce8dd3d7944fa863463a97d).
+
+Finally, if you are experienced with the [Open Autonomy](https://docs.autonolas.network/) framework, you can also modify the internal business logic of the service yourself.

run_service.sh

@@ -1,28 +1,48 @@
 #!/usr/bin/env bash
 
-# Load env vars
-export $(grep -v '^#' .env | xargs)
-
+# Define environment variables
+export ETHEREUM_LEDGER_RPC=INSERT_YOUR_RPC
+export ETHEREUM_LEDGER_CHAIN_ID=100
+
+export ALL_PARTICIPANTS='["YOUR_AGENT_ADDRESS"]'
+export SAFE_CONTRACT_ADDRESS="YOUR_SAFE_ADDRESS"
+
+export NEWSAPI_ENDPOINT=https://newsapi.org/v2/everything
+export NEWSAPI_API_KEY=YOUR_NEWSAPI_API_KEY
+export OPENAI_API_KEY=YOUR_OPENAI_API_KEY
+export ENGINE="gpt-4"
+export MARKET_APPROVAL_SERVER_URL=YOUR_MARKET_APPROVAL_SERVER_URL
+export MARKET_APPROVAL_SERVER_API_KEY=YOUR_MARKET_APPROVAL_SERVER_API_KEY
+
+export NUM_MARKETS=10
+export TOPICS='["business","science","technology","politics","arts","weather"]'
+export MARKET_FEE=2
+export INITIAL_FUNDS=1
+export MINIMUM_MARKET_TIME=1
+export MARKET_TIMEOUT=1
 export MARKET_IDENTIFICATION_PROMPT=$(sed -e ':a' -e 'N' -e '$!ba' \
   -e 's/"/\\"/g' \
   -e "s/'/\\\'/g" \
   -e 's/:/;/g' \
   -e 's/\n/\\n/g' \
   market_identification_prompt.txt)
 
-make clean
 
+make clean
 autonomy packages lock
 autonomy push-all
 
-autonomy fetch --local --service valory/market_maker --alias fetched_service && cd fetched_service
+# Fetch the service
+autonomy fetch --local --service valory/market_maker && cd market_maker
 
 # Build the image
 autonomy build-image
 
 # Copy keys and build the deployment
-cp $KEY_DIR/keys.json ./keys.json
+# MODIFY THIS PATH IF REQUIRED
+cp ../keys.json ./keys.json
 
+# Build the deployment
 autonomy deploy build -ltm
 
 # Run the deployment