Improve CUDA setup and chat request handling

README.md

@@ -10,6 +10,70 @@ See [`SPEC.md`](SPEC.md).
 pip install -e .
 ```
 
+### Nvidia CUDA install
+
+If a node has an Nvidia GPU, it must install a CUDA-enabled PyTorch build. If you see:
+
+```text
+Torch not compiled with CUDA enabled
+```
+
+then the node installed the CPU-only PyTorch package.
+
+Recommended fix on a macOS/Linux Nvidia node:
+
+```bash
+source .venv/bin/activate
+pip uninstall -y torch torchvision torchaudio
+pip install --index-url https://download.pytorch.org/whl/cu121 torch torchvision torchaudio
+pip install -e .
+```
+
+Recommended fix on a Windows Nvidia node using PowerShell:
+
+```powershell
+.\.venv\Scripts\Activate.ps1
+python -m pip uninstall -y torch torchvision torchaudio
+python -m pip install --index-url https://download.pytorch.org/whl/cu121 torch torchvision torchaudio
+python -m pip install -e .
+```
+
+If PowerShell blocks venv activation, run this once in the same PowerShell window:
+
+```powershell
+Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
+.\.venv\Scripts\Activate.ps1
+```
+
+Windows Command Prompt alternative:
+
+```bat
+.venv\Scripts\activate.bat
+python -m pip uninstall -y torch torchvision torchaudio
+python -m pip install --index-url https://download.pytorch.org/whl/cu121 torch torchvision torchaudio
+python -m pip install -e .
+```
+
+For newer CUDA builds, PyTorch may also provide `cu124` or `cu126` wheels. Check https://pytorch.org/get-started/locally/ if `cu121` is not appropriate.
+
+Verify CUDA support:
+
+```bash
+python - <<'PY'
+import torch
+print('torch:', torch.__version__)
+print('cuda available:', torch.cuda.is_available())
+print('cuda version:', torch.version.cuda)
+print('gpu:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else None)
+PY
+```
+
+Then run the node with:
+
+```bash
+truecluster node --cluster-host YOUR_CLUSTER_IP --cluster-port 7001 --device cuda:0
+```
+
 ## Run a cluster
 
 ```bash