Massed Compute

Launch GPU VM

# Launch via Massed Compute dashboard or API
# Product: gpu_1x_l40
# Image: 184 (Ubuntu Server 24.04 w/ Drivers)
# SSH Key: attach your public key
# Instance name: unsloth-finetune

Wait until the VM is running and copy the SSH details. This run landed in us-central-3 (Des Moines).

Verify GPU Access

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP \
  'nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader'

This run printed NVIDIA L40, 46068 MiB, 580.126.16.

Install Unsloth

Pin whatever pip actually installs. This VM printed Unsloth 2026.8.20, torch 2.11.0+cu130, TRL 0.24.0. Import unsloth before trl / transformers / peft or Unsloth warns that patches may be skipped.

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euxo pipefail
sudo apt-get update -qq
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y python3.12-venv
python3 -m venv ~/unsloth_env
~/unsloth_env/bin/pip install -U pip
~/unsloth_env/bin/pip install unsloth
~/unsloth_env/bin/python - <<'PY'
import unsloth, torch, trl, peft, transformers, bitsandbytes
print("unsloth", unsloth.__version__)
print("torch", torch.__version__, "cuda", torch.version.cuda)
print("trl", trl.__version__)
print("peft", peft.__version__)
print("transformers", transformers.__version__)
print("bnb", bitsandbytes.__version__)
print("gpu", torch.cuda.get_device_name(0))
PY
EOF

Unsloth’s current install page is Install Unsloth via pip. The older docs.unsloth.ai/get-started/installing-+-updating URL 404s.

Dataset format

Alpaca-style rows are instruction, input, output. Unsloth’s notebooks format them into a single text field and append the tokenizer EOS. Qwen2.5 Instruct’s EOS is <|im_end|> (id 151645 on this run). TRL 0.24 rejects a fake <EOS_TOKEN> that is not in the vocab.

This run loaded yahma/alpaca-cleaned (51,760 train rows) and kept the first 1,024. Full-epoch training is num_train_epochs=1 with max_steps unset — not this smoke.

LoRA / QLoRA config

load_in_4bit=True is QLoRA (4-bit base + LoRA adapters). r=16, lora_alpha=16, lora_dropout=0, bias="none", use_gradient_checkpointing="unsloth". Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.

Unsloth printed 8,798,208 trainable parameters of 502,830,976 (1.75%). max_seq_length=2048. Padding-free training auto-enabled.

Train

TRL 0.24’s SFTTrainer takes processing_class=tokenizer, not tokenizer=. Batch 2 × grad accum 4 = effective 8. max_steps=60, optim="adamw_8bit", learning_rate=2e-4, report_to="none".

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP \
  '~/unsloth_env/bin/python ~/unsloth_train.py'

Put unsloth_train.py on the VM (same logic as below). First step was slower (~6.5 s); later steps ~1.05 s.

import unsloth  # before trl / transformers / peft
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Qwen/Qwen2.5-0.5B-Instruct",
    max_seq_length=2048,
    load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
)

alpaca = (
    "Below is an instruction that describes a task, paired with an input that "
    "provides further context. Write a response that appropriately completes "
    "the request.\n\n### Instruction:\n{}\n\n### Input:\n{}\n\n### Response:\n{}"
)
eos = tokenizer.eos_token  # <|im_end|> on Qwen2.5 Instruct

def fmt(batch):
    texts = [
        alpaca.format(i, x, o) + eos
        for i, x, o in zip(batch["instruction"], batch["input"], batch["output"])
    ]
    return {"text": texts}

ds = load_dataset("yahma/alpaca-cleaned", split="train").select(range(1024))
ds = ds.map(fmt, batched=True)

trainer = SFTTrainer(
    model=model,
    processing_class=tokenizer,
    train_dataset=ds,
    args=SFTConfig(
        dataset_text_field="text",
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=5,
        max_steps=60,
        learning_rate=2e-4,
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="linear",
        seed=3407,
        output_dir="outputs",
        report_to="none",
        eos_token=eos,
    ),
)
trainer.train()
model.save_pretrained("lora_model")
tokenizer.save_pretrained("lora_model")

Save the adapter

model.save_pretrained("lora_model") wrote:

That is the adapter, not a merged 16-bit model. Merge is optional (FAQ). Checkpoint-60 also landed under outputs/.

Generate from the adapter

FastLanguageModel.for_inference(model)
prompt = alpaca.format("Explain LoRA fine-tuning in one sentence.", "", "")
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=64, use_cache=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))

This smoke returned a completion. Quality after 60 steps on 0.5B is not a product claim — it only proves generate runs on the adapter.

GPU util

nvidia-smi memory.used:

Used rose during the job. 0.5B QLoRA barely taxes a 48 GB L40. Size the SKU for the 7B–8B job you actually want, not this smoke.