Native C++ and PyTorch inference. No cloud. No API keys. Run frontier models on your own hardware.
git clone https://github.com/LMGNU/llm.cpp
Choose your preferred environment to get started.
# 1. Create a virtual environment python -m venv .venv .venv/Scripts/python.exe -m pip install --upgrade pip # 2. Install dependencies cd backend ../.venv/Scripts/python.exe -m pip install -r requirements.txt # 3. Interactive chat (PyTorch inference) python engine/inference.py --checkpoint "engine/best_model.pt" # 4. Single generation python engine/inference.py --checkpoint "engine/best_model.pt" --prompt "Hello" --max-new-tokens 100 --temperature 1.0
# 1. Build g++ -std=c++17 -O2 -I. -Iinclude -o llm.exe main.cpp # 2. Train from scratch ./llm.exe data/input.txt # 3. Chat mode ./llm.exe data/input.txt --chat # 4. Generation mode ./llm.exe data/input.txt --generate
Optimized for every major platform — from laptops to data centers.