Instructions to use noahsta/lean27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use noahsta/lean27b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: llama cli -hf noahsta/lean27b:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: llama cli -hf noahsta/lean27b:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf noahsta/lean27b:Q8_0
Use Docker
docker model run hf.co/noahsta/lean27b:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use noahsta/lean27b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "noahsta/lean27b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "noahsta/lean27b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/noahsta/lean27b:Q8_0
- Ollama
How to use noahsta/lean27b with Ollama:
ollama run hf.co/noahsta/lean27b:Q8_0
- Unsloth Desktop
- Pi
How to use noahsta/lean27b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "noahsta/lean27b:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use noahsta/lean27b with Docker Model Runner:
docker model run hf.co/noahsta/lean27b:Q8_0
- Lemonade
How to use noahsta/lean27b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull noahsta/lean27b:Q8_0
Run and chat with the model
lemonade run user.lean27b-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use noahsta/lean27b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default noahsta/lean27b:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use noahsta/lean27b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "noahsta/lean27b:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
lean27b
Qwen/Qwen3.6-27B LoRA-finetuned on Lean 4 / Mathlib4 source, merged and released as a single Q8_0 GGUF.
Weight-decay sweep at rank 32, all layers. The released model is wd = 0.4 (darkest line). Final full-validation loss: 0.652. The tool-call sanity check passed 3/3 in every epoch.
Data
- Every
.leanfile underMathlib/in mathlib4 at tagv4.33.0. - 80/20 train/val split by file (seed 42).
- Files shorter than 1024 tokens are kept whole. Longer files are split only at blank lines (
\n\n) and packed greedily into chunks of at most 1024 tokens. - Plain causal-LM loss on all tokens. No chat template, no instruction data.
Training
| Parameter | Value |
|---|---|
| Method | LoRA (peft), bf16 base, gradient checkpointing |
| Rank / alpha / dropout | 32 / 32 / 0.05 |
| Target modules | q_proj k_proj v_proj o_proj in_proj_qkv out_proj gate_proj up_proj down_proj |
| Layers | all 64 |
| Optimizer | AdamW, lr 1e-5 (constant, no warmup), weight decay 0.4 |
| Grad clip | 1.0 |
| Batch size | 8 (no accumulation) |
| Max sequence length | 1024 |
| Epochs | 5 |
| Seed | 42 (data order reshuffled with seed + epoch) |
| Hardware | 1ร 80 GB GPU |
| Libraries | torch 2.13.0, transformers 5.15.1, peft 0.20.0, accelerate 1.14.0 |
After each epoch, a check confirms the model still emits valid tool calls (grep / web_search / ls). Training stops early if 2 of the 3 calls fail in two consecutive epochs. This never triggered.
Export. The LoRA was merged into the bf16 base, and the vision tower was dropped. The result was converted with llama.cpp convert_hf_to_gguf.py --outtype q8_0. The GGUF metadata was then patched from block_count=65, nextn_predict_layers=1 to 64 / 0, because the MTP head is not included. No tensor data was changed.
Usage
The file is lean27b-r32_wd40_full-Q8_0.gguf (28.6 GB). It needs a llama.cpp build that supports qwen35.
llama-server -m lean27b-r32_wd40_full-Q8_0.gguf -ngl 99 -c 262144 --jinja
- Text only: no mmproj, no MTP head.
- VRAM: about 26.6 GiB for the weights, about 41.6 GiB at the full 262k context.
- Sampling defaults embedded in the GGUF: temperature 1.0, top_p 0.95, top_k 20.
- Downloads last month
- 39
8-bit
Model tree for noahsta/lean27b
Base model
Qwen/Qwen3.6-27B