Skip to content

No data under regulation folder #2

@prasoonvarshney

Description

@prasoonvarshney

The README notes regulation evaluation as

cd regulation

# Attack evaluation on regulatory compliance
python evaluate_attack.py --model_id OpenSafetyLab/MD-Judge-v0_2-internlm2_7b

# Conversation evaluation
python evaluate_conversation.py --model_id meta-llama/Llama-Guard-3-8B

# Query evaluation
python evaluate_query.py --model_id allenai/wildguard

The evaluate_attack.py file in turn tries to access dataset_path = f"{base_dir}/dataset/regulation/gdpr/query/unsafe.jsonl"

However, there is no such file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions