Python package names not normalized

[maxrake]

Python package names not normalized

[cd-work]

#1273 has been dismissed, is this even still useful without it?

[maxrake]

#1273 has been dismissed, is this even still useful without it?

I still think so. Here is another example that shows how the results of parsing the same input package results in different names, where the only difference is whether it gets parsed as a lockfile or manifest.

❯ phylum --version
phylum v5.8.1

❯ cat requirements.txt
RuAmEl...YAML---CLib==0.2.8

❯ phylum parse --lockfile-type pip requirements.txt
[
  {
    "name": "RuAmEl...YAML---CLib",
    "version": "0.2.8",
    "type": "pypi",
    "lockfile": "requirements.txt"
  }
]

❯ vim requirements.txt

❯ cat requirements.txt
RuAmEl...YAML---CLib

❯ phylum parse --lockfile-type pip requirements.txt
Generating lockfile for manifest "requirements.txt" using Pip…
[
  {
    "name": "ruamel.yaml.clib",
    "version": "0.2.8",
    "type": "pypi",
    "lockfile": "requirements.txt"
  }
]

The expectation is that name of the parsed package is normalized and will be the same in both cases...ruamel-yaml-clib in this case.

[kylewillmon]

My concern here is that "normalization" is not the correct behavior. As can be seen from your example:

❯ phylum parse --lockfile-type pip requirements.txt
Generating lockfile for manifest "requirements.txt" using Pip…
[
  {
    "name": "PyYAML",
    "version": "6.0.1",
    "type": "pypi",
    "lockfile": "requirements.txt"
  },
  {
    "name": "Pillow",
    "version": "10.1.0",
    "type": "pypi",
    "lockfile": "requirements.txt"
  }
]

These package names are provided by pip because lockfile generation happened, but they are not "normalized". Instead, they are the canonical names for these packages... The normalization rules that you provide are only used for deduplication.

The canonical name for a package can only be known by looking up that package in the registry. Which leaves us with two options:

1. Output the normalized name instead of the canonical name
2. Make network requests to the registry to find the canonical name

Option 2 is clearly outside the scope of a lockfile parser and Option 1 leads to results that are technically incorrect...

And just like #1273, this issue really only exists when using requirements.txt files as lockfiles... Because any lockfile generator should convert package names to the canonical name.

So unless there is an option that I am missing, I don't think there is any reasonable action that we can take to resolve this issue.

[maxrake]

Okay...points taken. This issue will be closed.