Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support the metadata directory (as per PEP-517) for build_wheel #4647

Merged
merged 8 commits into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions newsfragments/1825.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Re-use pre-existing ``.dist-info`` dir when creating wheels via the build backend APIs (PEP 517) and the `metadata_directory` argument is passed -- by :user:`pelson`.
abravalheri marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 4 additions & 1 deletion setuptools/build_meta.py
Original file line number Diff line number Diff line change
Expand Up @@ -417,9 +417,12 @@ def build_wheel(
config_settings: _ConfigSettings = None,
metadata_directory: StrPath | None = None,
):
cmd = ['bdist_wheel']
if metadata_directory:
cmd.extend(['--dist-info-dir', metadata_directory])
with suppress_known_deprecation():
return self._build_with_temp_dir(
['bdist_wheel'],
cmd,
'.whl',
wheel_directory,
config_settings,
Expand Down
24 changes: 21 additions & 3 deletions setuptools/command/bdist_wheel.py
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,13 @@ class bdist_wheel(Command):
None,
"Python tag (cp32|cp33|cpNN) for abi3 wheel tag [default: false]",
),
(
"dist-info-dir=",
None,
"directory where a pre-generated dist-info can be found (e.g. as a "
"result of calling the PEP517 'prepare_metadata_for_build_wheel' "
"method)",
),
]

boolean_options = ["keep-temp", "skip-build", "relative", "universal"]
Expand All @@ -243,6 +250,7 @@ def initialize_options(self) -> None:
self.format = "zip"
self.keep_temp = False
self.dist_dir: str | None = None
self.dist_info_dir = None
self.egginfo_dir: str | None = None
self.root_is_pure: bool | None = None
self.skip_build = False
Expand All @@ -261,8 +269,9 @@ def finalize_options(self) -> None:
bdist_base = self.get_finalized_command("bdist").bdist_base
self.bdist_dir = os.path.join(bdist_base, "wheel")

egg_info = cast(egg_info_cls, self.distribution.get_command_obj("egg_info"))
egg_info.ensure_finalized() # needed for correct `wheel_dist_name`
if self.dist_info_dir is None:
egg_info = cast(egg_info_cls, self.distribution.get_command_obj("egg_info"))
egg_info.ensure_finalized() # needed for correct `wheel_dist_name`

self.data_dir = self.wheel_dist_name + ".data"
self.plat_name_supplied = bool(self.plat_name)
Expand Down Expand Up @@ -447,7 +456,16 @@ def run(self):
f"{safer_version(self.distribution.get_version())}.dist-info"
)
distinfo_dir = os.path.join(self.bdist_dir, distinfo_dirname)
self.egg2dist(self.egginfo_dir, distinfo_dir)
if self.dist_info_dir:
# Use the given dist-info directly.
pelson marked this conversation as resolved.
Show resolved Hide resolved
log.debug(f"reusing {self.dist_info_dir}")
shutil.copytree(self.dist_info_dir, distinfo_dir)
# Egg info is still generated, so remove it now to avoid it getting
# copied into the wheel.
Comment on lines +463 to +464
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know why egg info is still generated? Is it because other commands might have generated it? What happens if the line for rmtree is removed (will the tests fail)?

While I'm okay with this approach, I'm also somewhat apprehensive, as it is perpetuating and reinforcing the expectation that egg-info is generated, even when it's not needed, making it harder in the future to correct this suboptimal approach. For example, this project also has the goal to someday obviate egg-info altogether, and to generate dist-info directly, and adding this additional handling of egg-info will likely complicate that work (though maybe not too much).

I wonder - how much do other commands from bdist_wheel depend on the generation of egg-info and could those commands get by without generating it if bdist_wheel could pass through the instruction to re-use existing dist-info?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know why egg info is still generated?

Looks like install calls install_egg_info (a registered sub-command, which is automatically run), which then calls egg_info. I wasn't brave enough to try to touch this.

What happens if the line for rmtree is removed (will the tests fail)?

I've added a specific test for this. Previously the test would have passed.

this project also has the goal to someday obviate egg-info altogether

Yes, I appreciate that. IMO, this change does add an additional place where you have to think about egg-info, but I believe it is already necessary to deal with bdist_wheel wrt egg-info, and the added code-path now explicitly rejects its use - I think that ultimately makes that easier to remove egg-info in the future.

shutil.rmtree(self.egginfo_dir)
else:
# Convert the generated egg-info into dist-info.
self.egg2dist(self.egginfo_dir, distinfo_dir)

self.write_wheelfile(distinfo_dir)

Expand Down
25 changes: 25 additions & 0 deletions setuptools/tests/test_bdist_wheel.py
Original file line number Diff line number Diff line change
Expand Up @@ -619,3 +619,28 @@ def _fake_import(name: str, *args, **kwargs):
monkeypatch.delitem(sys.modules, "setuptools.command.bdist_wheel")

import setuptools.command.bdist_wheel # noqa: F401


def test_dist_info_provided(dummy_dist, monkeypatch, tmp_path):
monkeypatch.chdir(dummy_dist)
distinfo = tmp_path / "dummy_dist.dist-info"

distinfo.mkdir()
(distinfo / "METADATA").write_text("name: helloworld", encoding="utf-8")

# We don't control the metadata. According to PEP-517, "The hook MAY also
# create other files inside this directory, and a build frontend MUST
# preserve".
(distinfo / "FOO").write_text("bar", encoding="utf-8")

bdist_wheel_cmd(bdist_dir=str(tmp_path), dist_info_dir=str(distinfo)).run()
expected = {
"dummy_dist-1.0.dist-info/FOO",
"dummy_dist-1.0.dist-info/RECORD",
}
with ZipFile("dist/dummy_dist-1.0-py3-none-any.whl") as wf:
files_found = set(wf.namelist())
# Check that all expected files are there.
assert expected - files_found == set()
# Make sure there is no accidental egg-info bleeding into the wheel.
assert not [path for path in files_found if 'egg-info' in str(path)]
Loading