Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start testing on 3.13 beta build #3185

Draft
wants to merge 3 commits into
base: develop
Choose a base branch
from
Draft

Start testing on 3.13 beta build #3185

wants to merge 3 commits into from

Conversation

nateprewitt
Copy link
Contributor

This PR will start provisional testing on the recently released beta for Python 3.13. This will be used to catch any issues early before the official release in October.

@nateprewitt
Copy link
Contributor Author

nateprewitt commented May 23, 2024

It looks like all 3.13 builds are failing on these two tests due to increased memory usage with paginators:

FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_paginator_memory_constant
FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_waiter_memory_constant

We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.

Example Stacktrace
Memory increases ~10.8MB which exceeds our arbitrary 10MB threshold. I'd like to see what our change looks like in 3.12 to determine if we barely crossed a threshold we were already approaching or if this is a significant increase.


self = <tests.functional.leak.test_resource_leaks.TestDoesNotLeakMemory testMethod=test_create_single_paginator_memory_constant>

    def test_create_single_paginator_memory_constant(self):
        self.cmd('create_paginator', 's3', 'list_objects')
        self.cmd('free_paginators')
        self.record_memory()
        for _ in range(100):
            self.cmd('create_paginator', 's3', 'list_objects')
            self.cmd('free_paginators')
        self.record_memory()
        start, end = self.memory_samples
>       self.assertTrue((end - start) < self.MAX_GROWTH_BYTES, (end - start))
E       AssertionError: False is not true : 10878976

@jonathan343
Copy link
Contributor

We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.

I had a few minutes to compare the delta values between 3.12 a 3.13. The results are shown below:

Test Python 3.12 Memory Delta (B) Python 3.13 Memory Delta (B) % Increase
test_create_memory_clients_in_loop -9,052,160 176,128 -101.95 ??
test_create_memory_paginators_in_loop 1,503,232 1,937,408 28.88
test_create_memory_waiters_in_loop 1,675,264 1,794,048 7.09
test_create_single_client_memory_constant 1,785,856 10,338,304 478.90
test_create_single_paginator_memory_constant 2,551,808 11,886,592 365.81
test_create_single_waiter_memory_constant 2,490,368 11,870,208 376.64

There is definitely weird something going on with the 3.13 interpreter. I don't have to time to do a deeper dive now, but wanted to add this information here for future reference.

Notes:

  1. What is going on with test_create_memory_clients_in_loop??

Copy link
Contributor

@jonathan343 jonathan343 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Python 3.13.0 release candidate 3 was released on 10/01/2024 and made available in GitHub CI with actions/python-versions#308. I tested locally and this should resolved the current failures we're seeing. I requested some minor changes.

@@ -14,7 +14,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13-dev']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13-dev']
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13']

@@ -68,7 +68,7 @@ def test_create_single_waiter_memory_constant(self):
self.cmd('create_waiter', 's3', 'bucket_exists')
self.cmd('free_waiters')
self.record_memory()
for _ in range(100):
for _ in range(500):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for _ in range(500):
for _ in range(100):

@@ -93,7 +93,7 @@ def test_create_single_paginator_memory_constant(self):
self.cmd('create_paginator', 's3', 'list_objects')
self.cmd('free_paginators')
self.record_memory()
for _ in range(100):
for _ in range(500):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for _ in range(500):
for _ in range(100):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants