Examples

Fine-tune with a Run Job

Train on a GPU you pay for only while training runs.

This example fine-tunes a small language model with LoRA as a Run Job. The training script and the trained adapter live on a network volume.

What you need

  • The CLI, signed in.
  • Credit for about an hour of one GPU. The run itself takes a few minutes.

Step 1: Create a network volume

Open the Storage page, click Create volume, name it training, set Capacity to 20, and click Create volume. It's ready for files after about a minute.

Step 2: Upload the training script

Save this as finetune.py. It trains a LoRA adapter for Qwen/Qwen2.5-0.5B on 1,000 instruction examples and saves it to the volume.

finetune.py
import torch
from datasets import load_dataset
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer, DataCollatorForLanguageModeling, Trainer, TrainingArguments

model_id = "Qwen/Qwen2.5-0.5B"
output_dir = "/workspace/outputs/qwen-lora"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="cuda")
model = get_peft_model(model, LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"], task_type="CAUSAL_LM"))
model.print_trainable_parameters()

dataset = load_dataset("yahma/alpaca-cleaned", split="train[:1000]")

def tokenize(example):
    text = f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}{tokenizer.eos_token}"
    return tokenizer(text, truncation=True, max_length=512)

dataset = dataset.map(tokenize, remove_columns=dataset.column_names)

trainer = Trainer(
    model=model,
    args=TrainingArguments(
        output_dir=output_dir,
        per_device_train_batch_size=8,
        num_train_epochs=1,
        learning_rate=2e-4,
        bf16=True,
        logging_steps=25,
        save_strategy="no",
        report_to=[],
    ),
    train_dataset=dataset,
    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
)
trainer.train()
model.save_pretrained(output_dir)
print(f"Saved the adapter to {output_dir}")

Upload it to the volume. In the console, open the volume and click Upload files, or use the CLI:

featherless-cloud volume cp finetune.py vol://training/

Step 3: Run the training job

Submit the script as a Run Job on one GPU, with the volume mounted at /workspace. The command installs the libraries the script needs, then runs it. HF_HOME keeps the downloaded model on the volume, so the next run reuses it.

featherless-cloud exec \
  --name finetune \
  --template featherless/rocm-pytorch \
  --gpu-count 1 \
  --volume training:/workspace \
  --env HF_HOME=/workspace/hf \
  --timeout 3600 \
  -- bash -c "pip install -q transformers peft datasets accelerate && python /workspace/finetune.py"

The CLI streams the output while the job runs. It ends like this:

trainable params: 1,081,344 || all params: 495,114,112 || trainable%: 0.2184
{'loss': '1.492', 'grad_norm': '0.6243', 'learning_rate': '0.0001616', 'epoch': '0.2'}
...
{'train_runtime': '8.94', 'train_samples_per_second': '111.9', 'train_steps_per_second': '13.98', 'train_loss': '1.458', 'epoch': '1'}
Saved the adapter to /workspace/outputs/qwen-lora
Run Job jbkbsmhf6s: succeeded

The job also appears on the console's Run Jobs page, with its logs. If you close the terminal, the job keeps running; follow it again with featherless-cloud runjob logs <job-id> --follow.

Step 4: Download the adapter

featherless-cloud volume cp -r vol://training/outputs/qwen-lora .
ls qwen-lora
README.md  adapter_config.json  adapter_model.safetensors

Load it with PEFT on top of the base model, or keep it on the volume and mount the volume into an instance that serves the model.

Clean up

The Run Job's compute stopped when the script finished. The volume keeps billing until you delete it:

featherless-cloud volume delete training

Next steps

Submit a Run Job

Time limits, retries and detached runs.

Transfer files

Sync datasets and checkpoints with a volume.