I saw that it was committed, but llama.py did not change. When does it apply?

[suhyun0115]

I keep getting a warning that there are 2 bos tokens, but this branch seems to be the solution.
9d053d6

and If it is committed, how should I apply it to this code?

def load_model(repo_id, filename, chat_format = 'llama-3'):
model = Llama.from_pretrained(
repo_id=repo_id,
filename=filename,
n_gpu_layers=-1,
chat_format = chat_format,
)
return model

messages = [
{"role": "system", "content": "You are a helpful assistant."},
{
"role": "user",
"content": "hello"
}
]
llm.create_chat_completion(
messages=messages,
max_tokens=2048,
)

[CISC]

You're looking at a single commit in #1439 which is not the final solution.

When this PR gets merged and a new release is made the changes will be active, this is up to @abetlen to decide.