Evaluating with adapters

[maanasharma5]

Hello,

Is there a way to evaluated a model that we have trained an e.g. LoRA adapter on? Thanks

[Hjh233]

You can first merge the weights of the LoRA adapters with the pre-trained model, as outlined in this discussion. Then, set the model path to point to the merged model.

[coder23j]

You can just add --lora_path arguments

CUDA_VISIBLE_DEVICES=0 \
python main.py mmlu --model_name llama \
  --model_path /path/to/basemodel \
  --lora_path /path/to/loraweight

if you change lora model, just export PYTHONPATH="$PYTHONPATH:/path/contains/changed-transformers-peft"