Skip to content

Commit

Permalink
Fix mirostatv2. (ggerganov#338)
Browse files Browse the repository at this point in the history
  • Loading branch information
ycros authored Jul 23, 2023
1 parent f036109 commit 56995ca
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions gpttype_adapter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,11 @@ llama_token sample_token_mirostat_v2(llama_token_data_array * candidates, std::m
candidates->size = std::distance(candidates->data, std::find_if(candidates->data, candidates->data + candidates->size, [&](const llama_token_data & candidate) {
return -log2f(candidate.p) > *mu;
}));

if (candidates->size == 0) {
candidates->size = 1;
}

// Normalize the probabilities of the remaining words
llama_sample_softmax(nullptr, candidates);
// Sample the next word X from the remaining words
Expand Down

0 comments on commit 56995ca

Please sign in to comment.