Unofficial re-implementation of Trusting Your Evidence: Hallucinate Less with Context-aware Decoding
- Context-Aware Generation: Enhances the relevance of the generated text by incorporating additional context into the decoding process.
- Currently, this projecit is in a stage where it can only perform simple task generation.
- Available decoding strategy : Greedy Decoding, Top-p Sampling, Top-k Sampling
- Implement based on huggingface transformers' generation_utils.py
cad_model = CAD(model_name="huggyllama/llama-13b", device=0)
contexts = ['Write a quote that ends in the word "early":']
input_texts = ['Better late than']
outputs = cad_model.generate(
input_texts=input_texts,
use_context_aware=True,
contexts=contexts,
max_length=20,
alpha=0.5,
decoding_strategy='top_p',
top_p_value=0.9,
use_repetition_penalty=True,
repetition_penalty_value=1.5,
)
print(cad_model.tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
@misc{shi2023trusting,
title={Trusting Your Evidence: Hallucinate Less with Context-aware Decoding},
author={Weijia Shi and Xiaochuang Han and Mike Lewis and Yulia Tsvetkov and Luke Zettlemoyer and Scott Wen-tau Yih},
year={2023},
eprint={2305.14739},
archivePrefix={arXiv},
primaryClass={cs.CL}
}