llamafactory

# LlamaFactory Fine-Tuning

LlamaFactory fine-tunes open-weight LLMs (LoRA/QLoRA and full-parameter; SFT, DPO and more) through the `llamafactory-cli` command driven by YAML configs.

## Before you start

If the user's message only invokes this skill (e.g. "use llamafactory skill") without a concrete request, ask the user what they want to fine-tune. Do not run any command until the goal is clear.

Confirm before training:

- GPU memory (`nvidia-smi`) — it bounds the model size and method; LoRA needs far less than full fine-tuning.
- The base model: a Hugging Face id or a local path.
- The dataset: where it lives and which format it is in.
- The goal: SFT with LoRA is the usual starting point.

## Install

```bash
git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git
cd LlamaFactory
pip install -e .
pip install -r requirements/metrics.txt   # optional: evaluation metrics
```

## Data

Register every dataset in `data/dataset_info.json`; the alpaca and sharegpt formats are supported. A minimal local entry:

```json
"my_dataset": { "file_name": "my_dataset.json" }
```

alpaca rows carry `instruction` / `input` / `output`; sharegpt rows carry a `conversations` list. Put the data file under `data/` next to the registry.

## Train

Training is driven by a YAML config. Start from the shipped example `examples/train_lora/qwen3_lora_sft.yaml`, or save a minimal config as `my_sft.yaml`, e.g. for [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B):

```yaml
model_name_or_path: Qwen/Qwen3-1.7B
trust_remote_code: true
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 8
lora_target: all
dataset: my_dataset
template: qwen3
output_dir: saves/qwen3-1.7b/lora/sft
learning_rate: 1.0e-4
num_train_epochs: 3.0
bf16: true
```

```bash
llamafactory-cli train my_sft.yaml
```

`llamafactory-cli webui` launches the no-code web UI for the same workflow.

## Merge and export

Merge the LoRA adapter into the base weights for standalone serving. Start from `examples/merge_lora/qwen3_lora_sft.yaml`, pointing `model_name_or_path`, `adapter_name_or_path` and `template` at your run (never merge into a quantized base):

```yaml
model_name_or_path: Qwen/Qwen3-1.7B
adapter_name_or_path: saves/qwen3-1.7b/lora/sft
template: qwen3
trust_remote_code: true
export_dir: saves/qwen3-1.7b-sft-merged
```

```bash
llamafactory-cli export my_merge.yaml
```

## Try the result

Both commands take an inference config — derive it from `examples/inference/qwen3_lora_sft.yaml`, again pointing the model, adapter and template at your run:

```yaml
model_name_or_path: Qwen/Qwen3-1.7B
adapter_name_or_path: saves/qwen3-1.7b/lora/sft
template: qwen3
infer_backend: huggingface
trust_remote_code: true
```

```bash
llamafactory-cli chat my_infer.yaml   # interactive chat with the tuned model
llamafactory-cli api my_infer.yaml    # OpenAI-compatible API server
```

## Close the loop

Serve the merged export as a standalone endpoint — vLLM serves the export directory directly, while Ollama needs an import first (a `Modelfile` with `FROM /path/to/export`, then `ollama create`; supported model architectures only) — then register the endpoint with PenguinHarness so agents can build, evaluate and tune AI apps on the fine-tuned model end to end.