23.md

Results of GPT-3.5 on HumanEval/23

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 69

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """return len(string)

Unit tests

METADATA = {
    'author': 'jt',
    'dataset': 'test'
}


def check(candidate):
    assert candidate('') == 0
    assert candidate('x') == 1
    assert candidate('asdasnakj') == 9

Runs

✅ Run 1

Start time: 2023-07-18 17:26:06
Duration: 2.24s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 2

Start time: 2023-07-18 17:26:04
Duration: 2.23s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 3

Start time: 2023-07-18 17:26:02
Duration: 2.3s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 4

Start time: 2023-07-18 17:25:59
Duration: 2.26s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 5

Start time: 2023-07-18 17:25:50
Duration: 9.44s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 6

Start time: 2023-07-18 17:25:48
Duration: 2.0s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 7

Start time: 2023-07-18 17:25:45
Duration: 2.41s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 8

Start time: 2023-07-18 17:25:43
Duration: 2.37s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 9

Start time: 2023-07-18 17:25:40
Duration: 2.17s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)

✅ Run 10

Start time: 2023-07-18 17:25:38
Duration: 2.33s
Completion tokens: 42
Result: Pass

def strlen(string: str) -> int:
    """ Return length of given string
    >>> strlen('')
    0
    >>> strlen('abc')
    3
    """
    return len(string)