owen/worker-vllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Jorg Doku
2023-08-03 10:48:40 -05:00
committed by GitHub
parent 04baa8903d
commit 9e836cc6d1
1 changed files with 14 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -14
View File
@@ -28,20 +28,20 @@ Ensure that you have Docker installed and properly set up before running the doc
## Model Inputs
```
| Argument | Type | Description |
|--------------------|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| n | int | Number of output sequences to return for the given prompt. |
| best_of | Optional[int] | Number of output sequences that are generated from the prompt. From these `best_of` sequences, the top `n` sequences are returned. `best_of` must be greater than or equal to `n`. This is treated as the beam width when `use_beam_search` is True. By default, `best_of` is set to `n`. |
| presence_penalty | float | Float that penalizes new tokens based on whether they appear in the generated text so far. Values > 0 encourage the model to use new tokens, while values < 0 encourage the model to repeat tokens. |
| frequency_penalty | float | Float that penalizes new tokens based on their frequency in the generated text so far. Values > 0 encourage the model to use new tokens, while values < 0 encourage the model to repeat tokens. |
| temperature | float | Float that controls the randomness of the sampling. Lower values make the model more deterministic, while higher values make the model more random. Zero means greedy sampling. |
| top_p | float | Float that controls the cumulative probability of the top tokens to consider. Must be in (0, 1]. Set to 1 to consider all tokens. |
| top_k | int | Integer that controls the number of top tokens to consider. Set to -1 to consider all tokens. |
| use_beam_search | bool | Whether to use beam search instead of sampling. |
| stop | Union[None, str, List[str]] | List of strings that stop the generation when they are generated. The returned output will not contain the stop strings. |
| ignore_eos | bool | Whether to ignore the EOS token and continue generating tokens after the EOS token is generated. |
| max_tokens | int | Maximum number of tokens to generate per output sequence. |
| logprobs | Optional[int] | Number of log probabilities to return per output token. |
| Argument | Type | Default | Description |
|--------------------|-----------------|-----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| n | int | 1 | Number of output sequences to return for the given prompt. |
| best_of | Optional[int] | None | Number of output sequences that are generated from the prompt. From these `best_of` sequences, the top `n` sequences are returned. `best_of` must be greater than or equal to `n`. This is treated as the beam width when `use_beam_search` is True. By default, `best_of` is set to `n`. |
| presence_penalty | float | 0.0 | Float that penalizes new tokens based on whether they appear in the generated text so far. Values > 0 encourage the model to use new tokens, while values < 0 encourage the model to repeat tokens. |
| frequency_penalty | float | 0.0 | Float that penalizes new tokens based on their frequency in the generated text so far. Values > 0 encourage the model to use new tokens, while values < 0 encourage the model to repeat tokens. |
| temperature | float | 1.0 | Float that controls the randomness of the sampling. Lower values make the model more deterministic, while higher values make the model more random. Zero means greedy sampling. |
| top_p | float | 1.0 | Float that controls the cumulative probability of the top tokens to consider. Must be in (0, 1]. Set to 1 to consider all tokens. |
| top_k | int | -1 | Integer that controls the number of top tokens to consider. Set to -1 to consider all tokens. |
| use_beam_search | bool | False | Whether to use beam search instead of sampling. |
| stop | Union[None, str, List[str]] | None | List of strings that stop the generation when they are generated. The returned output will not contain the stop strings. |
| ignore_eos | bool | False | Whether to ignore the EOS token and continue generating tokens after the EOS token is generated. |
| max_tokens | int | 256 | Maximum number of tokens to generate per output sequence. |
| logprobs | Optional[int] | None | Number of log probabilities to return per output token. |
```
## Test Inputs