16.md

Results of GPT-3.5 on HumanEval/16

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 96

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """return len(set(string.lower()))

Unit tests

METADATA = {
    'author': 'jt',
    'dataset': 'test'
}


def check(candidate):
    assert candidate('') == 0
    assert candidate('abcde') == 5
    assert candidate('abcde' + 'cade' + 'CADE') == 5
    assert candidate('aaaaAAAAaaaa') == 1
    assert candidate('Jerry jERRY JeRRRY') == 5

Runs

✅ Run 1

Start time: 2023-07-18 17:27:25
Duration: 3.36s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 2

Start time: 2023-07-18 17:27:22
Duration: 2.97s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 3

Start time: 2023-07-18 17:27:19
Duration: 3.31s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 4

Start time: 2023-07-18 17:27:16
Duration: 2.93s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 5

Start time: 2023-07-18 17:27:13
Duration: 2.99s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 6

Start time: 2023-07-18 17:27:10
Duration: 3.05s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 7

Start time: 2023-07-18 17:27:06
Duration: 3.23s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 8

Start time: 2023-07-18 17:27:03
Duration: 3.03s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 9

Start time: 2023-07-18 17:27:00
Duration: 3.02s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))

✅ Run 10

Start time: 2023-07-18 17:26:57
Duration: 3.26s
Completion tokens: 71
Result: Pass

def count_distinct_characters(string: str) -> int:
    """ Given a string, find out how many distinct characters (regardless of case) does it consist of
    >>> count_distinct_characters('xyzXYZ')
    3
    >>> count_distinct_characters('Jerry')
    4
    """
    return len(set(string.lower()))