diff --git a/.runpod/README.md b/.runpod/README.md index 4fed299..303ab47 100644 --- a/.runpod/README.md +++ b/.runpod/README.md @@ -32,6 +32,9 @@ All behaviour is controlled through environment variables: For complete configuration options, see the [full configuration documentation](https://github.com/runpod-workers/worker-vllm/blob/main/docs/configuration.md). +### Specify Transformers Version +To change the version of the [Transformers library](https://github.com/huggingface/transformers) use the `TRANSFORMERS_VERSION` environment variable to specify the version you want to use. Note this might break the handler, so use for development purposes. + ## API Usage This worker supports two API formats: **RunPod native** and **OpenAI-compatible**. diff --git a/README.md b/README.md index 7488c8a..56e156c 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,10 @@ Any env var whose name matches a valid `AsyncEngineArgs` field (uppercased) is a For the complete list of all available environment variables, examples, and detailed descriptions: **[Configuration](docs/configuration.md)** +### Specify Transformers Version +To change the version of the [Transformers library](https://github.com/huggingface/transformers) use the `TRANSFORMERS_VERSION` environment variable to specify the version you want to use. Note this might break the handler, so use for development purposes. + + ## Option 2: Build Docker Image with Model Inside To build an image with the model baked in, you must specify the following docker arguments when building the image.