[Azure] Optimize autostopping speed for azure

[Michaelvll]

This is to reduce the autostopping speed for azure.

Previously, auto stopping takes 5m11s, and with the current branch, it takes 39s (8x faster)

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

[cblmemo]

Thanks for the fix! LGTM.