Skip to content

Commit

Permalink
adde dcoker build instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
NiftyliuS committed Feb 3, 2024
1 parent 0b3b152 commit bca0d37
Showing 1 changed file with 25 additions and 0 deletions.
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Simple python code that can run inference on LLM models with rest api interface.
- [Run with huggingface model](#run-with-huggingface-model)
- [Run with local model](#run-with-local-model)
- [Run with custom env-file](#run-with-custom-env-file)
- [Docker build](#docker-build)
- [Configuration values and parameters](#env-values-and-parameters)
- [Rest API server config](#rest-api-server-config)
- [General config](#general-config)
Expand Down Expand Up @@ -148,6 +149,30 @@ https://raw.githubusercontent.com/alexandriaproject-io/llm-inference/main/.env.e
docker run --gpus all --env-file .env.example -e MODEL_PATH="[huggingface model url]" -p 6060:6060 niftylius/llm-inference:latest
```

## Docker build

Clone git project with: \
`git clone [email protected]:alexandriaproject-io/llm-inference.git`

Navigate in to the project folder and run:

```shell
# build auto docker image
docker build -f .\docker\Dockerfile.auto -t [Your Image tag name] .

# build cuda 12.1 docker image
docker build -f .\docker\Dockerfile.cuda12 -t [Your Image tag name] .

# build cuda 11.8 docker image
docker build -f .\docker\Dockerfile.cuda11 -t [Your Image tag name] .

# build cpu docker image
docker build -f .\docker\Dockerfile.cpu -t [Your Image tag name] .

# build llama-cpp
docker build -f .\docker\Dockerfile.llama-cpp.cuda -t [Your Image tag name] .
```

## .env values and parameters

You can find the example file here [.env.example](.env.example)
Expand Down

0 comments on commit bca0d37

Please sign in to comment.