高级主题检查点
从 Checkpoint 恢复训练
这一页对应 mint-quickstart 中的 advanced/checkpoint.py resume。
两种恢复模式
- 仅恢复权重:脚本会先尝试
create_training_client_from_state(path)自动识别 model/rank;如果对原始 checkpoint 路径做 metadata 查询时返回404,就回退到create_lora_training_client(...) + load_state(path),并使用MINT_BASE_MODEL/MINT_LORA_RANK(或默认值)。 - 连 optimizer 一起恢复:
create_lora_training_client(...)再配合load_state_with_optimizer(path),可以保留 optimizer momentum,但需要MINT_BASE_MODEL和MINT_LORA_RANK。
按所在区域选择 MinT 域名:
- 境内:
https://mint-cn.macaron.xin/ - 境外:
https://mint.macaron.xin/
命令
# 仅恢复权重
export MINT_API_KEY=sk-...
python advanced/checkpoint.py resume tinker://<run-id>/weights/<checkpoint-name>
# 连 optimizer 一起恢复
export MINT_API_KEY=sk-...
export MINT_BASE_MODEL=Qwen/Qwen3-0.6B
export MINT_LORA_RANK=16
python advanced/checkpoint.py resume tinker://<run-id>/weights/<checkpoint-name> --with-optimizer --steps 3常用参数:
--steps:恢复后继续跑多少个 SFT step--lr:这些 step 使用的学习率--save-name:恢复完成后新保存的 checkpoint 名称
核心 API
tc = service_client.create_training_client_from_state(resume_path)
tc = service_client.create_lora_training_client(base_model=model, rank=rank)
tc.load_state(resume_path).result()
tc = service_client.create_lora_training_client(base_model=model, rank=rank)
tc.load_state_with_optimizer(resume_path).result()预期输出
[resume] path=tinker://.../weights/my-ckpt-state with_optimizer=False steps=3
[resume] creating training client from state (optimizer resets)...
[resume] auto-detect state metadata lookup returned 404; retrying with explicit model/rank from env/defaults
[resume] fallback to explicit training client: model=Qwen/Qwen3-0.6B rank=16
[resume] loading state from tinker://.../weights/my-ckpt-state...
[resume] loaded, running 3 SFT step(s)...
[resume] step 1/3 done
[resume] saved: tinker://.../weights/resumed-checkpoint常见失败
- checkpoint 路径不存在或无效
- 使用
--with-optimizer时,没有匹配的MINT_BASE_MODEL/MINT_LORA_RANK - checkpoint 的 adapter 形状与新 client 不匹配
- 当前账号下 base model 不可用