Inital commit

2026-06-04 18:11:26 -05:00
commit 9bd7f5838e
24 changed files with 2454 additions and 0 deletions
@@ -0,0 +1,31 @@
+# TrueCluster
+
+Prototype heterogeneous pipeline-parallel LLM inference cluster.
+
+See [`SPEC.md`](SPEC.md).
+
+## Install
+
+```bash
+pip install -e .
+```
+
+## Run a cluster
+
+```bash
+truecluster cluster --model Qwen/Qwen2.5-0.5B-Instruct --max-nodes 1 --quant fp16
+```
+
+## Run a node
+
+```bash
+truecluster node --cluster-host 127.0.0.1 --cluster-port 7001 --device auto
+```
+
+## Generate
+
+```bash
+curl http://127.0.0.1:8000/v1/completions \
+  -H 'content-type: application/json' \
+  -d '{"model":"Qwen/Qwen2.5-0.5B-Instruct","prompt":"Hello","max_tokens":32}'
+```