Skip to content

Latest commit

 

History

History
149 lines (141 loc) · 4.64 KB

152.md

File metadata and controls

149 lines (141 loc) · 4.64 KB

Results of GPT-4 on HumanEval/152

Date: 18 July, 2023
Model: gpt-4
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 256

def compare(game,guess):
    """I think we all remember that feeling when the result of some long-awaited
    event is finally known. The feelings and thoughts you have at that moment are
    definitely worth noting down and comparing.
    Your task is to determine if a person correctly guessed the results of a number of matches.
    You are given two arrays of scores and guesses of equal length, where each index shows a match. 
    Return an array of the same length denoting how far off each guess was. If they have guessed correctly,
    the value is 0, and if not, the value is the absolute difference between the guess and the score.
    
    
    example:

    compare([1,2,3,4,5,1],[1,2,3,4,2,-2]) -> [0,0,0,0,3,3]
    compare([0,5,0,0,0,4],[4,1,1,0,0,-2]) -> [4,4,1,0,0,6]
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def compare(game,guess):
    """I think we all remember that feeling when the result of some long-awaited
    event is finally known. The feelings and thoughts you have at that moment are
    definitely worth noting down and comparing.
    Your task is to determine if a person correctly guessed the results of a number of matches.
    You are given two arrays of scores and guesses of equal length, where each index shows a match. 
    Return an array of the same length denoting how far off each guess was. If they have guessed correctly,
    the value is 0, and if not, the value is the absolute difference between the guess and the score.
    
    
    example:

    compare([1,2,3,4,5,1],[1,2,3,4,2,-2]) -> [0,0,0,0,3,3]
    compare([0,5,0,0,0,4],[4,1,1,0,0,-2]) -> [4,4,1,0,0,6]
    """return [abs(x-y) for x,y in zip(game,guess)]

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate([1,2,3,4,5,1],[1,2,3,4,2,-2])==[0,0,0,0,3,3], "This prints if this assert fails 1 (good for debugging!)"
    assert candidate([0,0,0,0,0,0],[0,0,0,0,0,0])==[0,0,0,0,0,0], "This prints if this assert fails 1 (good for debugging!)"
    assert candidate([1,2,3],[-1,-2,-3])==[2,4,6], "This prints if this assert fails 1 (good for debugging!)"
    assert candidate([1,2,3,5],[-1,2,3,4])==[2,0,0,1], "This prints if this assert fails 1 (good for debugging!)"

    # Check some edge cases that are easy to work out by hand.
    assert True, "This prints if this assert fails 2 (also good for debugging!)"

Runs

✅ Run 1

Start time: 2023-07-18 17:47:30
Duration: 2.58s
Completion tokens: 27
Result: Pass

def compare(game,guess):
    return [abs(a-b) for a, b in zip(game, guess)]

✅ Run 2

Start time: 2023-07-18 17:47:27
Duration: 2.76s
Completion tokens: 27
Result: Pass

def compare(game,guess):
    return [abs(a-b) for a, b in zip(game, guess)]

✅ Run 3

Start time: 2023-07-18 17:47:24
Duration: 3.48s
Completion tokens: 28
Result: Pass

def compare(game,guess):
    return [abs(a - b) for a, b in zip(game, guess)]

✅ Run 4

Start time: 2023-07-18 17:47:21
Duration: 3.16s
Completion tokens: 33
Result: Pass

def compare(game,guess):
    return [abs(a-b) if a!=b else 0 for a,b in zip(game,guess)]

✅ Run 5

Start time: 2023-07-18 17:47:18
Duration: 2.37s
Completion tokens: 27
Result: Pass

def compare(game,guess):
    return [abs(a-b) for a, b in zip(game, guess)]

✅ Run 6

Start time: 2023-07-18 17:47:15
Duration: 2.8s
Completion tokens: 28
Result: Pass

def compare(game,guess):
    return [abs(a - b) for a, b in zip(game, guess)]

✅ Run 7

Start time: 2023-07-18 17:47:13
Duration: 2.56s
Completion tokens: 27
Result: Pass

def compare(game,guess):
    return [abs(a-b) for a, b in zip(game, guess)]

✅ Run 8

Start time: 2023-07-18 17:47:10
Duration: 2.81s
Completion tokens: 27
Result: Pass

def compare(game,guess):
    return [abs(a-b) for a, b in zip(game, guess)]

✅ Run 9

Start time: 2023-07-18 17:47:07
Duration: 3.07s
Completion tokens: 27
Result: Pass

def compare(game,guess):
    return [abs(a-b) for a, b in zip(game, guess)]

✅ Run 10

Start time: 2023-07-18 17:47:04
Duration: 3.05s
Completion tokens: 27
Result: Pass

def compare(game,guess):
    return [abs(a-b) for a, b in zip(game, guess)]