How to evaluate toxicity task on local hf-llama2-7B？

[AboveParadise]

Here is my code:

#!/bin/bash
dt-run +toxicity=realtoxicityprompts-toxic  \
    ++model=hf/../llama/llama-2-7b-hf \
    ++toxicity.n=25 \
    ++toxicity.template=1

and the bug is
Traceback (most recent call last):
File "/mnt/disk1/yg/DecodingTrust/src/dt/main.py", line 42, in main
perspective_module.main(perspective_args(**perspective_config))
File "/mnt/disk1/yg/DecodingTrust/src/dt/perspectives/toxicity/text_generation_hydra.py", line 29, in main
generator = Chat.from_helm(OPTS, conv_template=args.conv_template, cache=dirname, api_key=args.key)
File "/mnt/disk1/yg/DecodingTrust/src/dt/chat.py", line 41, in from_helm
return HFChat(model_name.replace("hf/", "").rstrip("/"), **kwargs)
File "/mnt/disk1/yg/DecodingTrust/src/dt/chat.py", line 364, in init
self.conv_template = get_conv_template(conv_template)
File "/mnt/disk1/yg/DecodingTrust/src/dt/conversation.py", line 284, in get_conv_template
return conv_templates[name].copy()
KeyError: None

How can i fix it?

[danielz02]

Thanks for your interest. To specify a local HF model, please use hf//path/to/local/hf/model.

[AboveParadise]

Thanks for your reply, but I have already used my local hf-llama2-7b model's location ../llama/llama-2-7b-hf, which is a folder downloaded from huggingface, is there any format mistake?

[danielz02]

Please try this

#!/bin/bash
dt-run +toxicity=realtoxicityprompts-toxic  \
    ++model=hf//../llama/llama-2-7b-hf \
    ++toxicity.n=25 \
    ++toxicity.template=1

[AboveParadise]

Thx, but I've already tried this and I got this bug:
Could not parse model name: '/../llama/llama-2-7b-hf'; Expected format: [namespace/]model_name[@revision]
Error executing job with overrides: ['+toxicity=realtoxicityprompts-toxic', '++model=hf//../llama/llama-2-7b-hf', '++toxicity.n=25', '++toxicity.template=1']
Traceback (most recent call last):
File "/mnt/disk1/yg/DecodingTrust/src/dt/main.py", line 42, in main
perspective_module.main(perspective_args(**perspective_config))
File "/mnt/disk1/yg/DecodingTrust/src/dt/perspectives/toxicity/text_generation_hydra.py", line 29, in main
generator = Chat.from_helm(OPTS, conv_template=args.conv_template, cache=dirname, api_key=args.key)
File "/mnt/disk1/yg/DecodingTrust/src/dt/chat.py", line 41, in from_helm
return HFChat(model_name.replace("hf/", "").rstrip("/"), **kwargs)
File "/mnt/disk1/yg/DecodingTrust/src/dt/chat.py", line 361, in init
raise ValueError("Unable to retrieve model config")
ValueError: Unable to retrieve model config

It seems worse than the former one, how could I fix it? FYI, my ../llama/llama-2-7b-hf/ is as below:

(llmbench) [root@gpu24 DecodingTrust]# ls ../llama/llama-2-7b-hf/
config.json             gitattributes.txt                 pytorch_model-00002-of-00002.bin  special_tokens_map.json  tokenizer.json
generation_config.json  pytorch_model-00001-of-00002.bin  pytorch_model.bin.index.json      tokenizer_config.json    tokenizer.model

[AboveParadise]

I think the main problem is there is no conv_template in the args

[danielz02]

Could you try using absolute path?