python/mypy_extensions

Extensions for mypy

Mypy Extensions

The mypy_extensions module defines extensions to the Python standard library typing module that are supported by the mypy type checker and the mypyc compiler.

mypy_extensions's Issues

Deprecate `mypy_extensions.TypedDict`?

mypy_extensions.TypedDict is problematic for several reasons:

• mypy_extensions.TypedDict is missing quite a few features that have been added to typing.TypedDict at runtime (and have been backported to typing_extensions.TypedDict).
• I think the existence of both mypy_extensions.TypedDict and typing_extensions.TypedDict has the potential to be pretty confusing for users.
• It's quite annoying and error-prone that at typeshed, we have to remember to keep typing._TypedDict, typing_extensions._TypedDict and mypy_extensions._TypedDict all in sync. Unfortunately, we can't put them all in _typeshed and have all three modules import _TypedDict from _typeshed, as the three classes all have slightly subtle differences. (E.g. mypy_extensions._TypedDict doesn't have the __required_keys__ and __optional_keys__ ClassVars that both typing._TypedDict and typing_extensions._TypedDict have.)
• Mypy now maintains its own copy of the mypy-extensions stubs anyway, so its stubs for mypy_extensions are no longer automatically updated with each typeshed sync. That means that even if we update the stubs for mypy_extensions.TypedDict in typeshed (as we did in python/typeshed#10565), those updates are no longer of any benefit to mypy users unless mypy maintainers remember to copy across the changes to their forked version of the mypy_extensions stubs.

I propose that we deprecate mypy_extensions.TypedDict, and steer people towards typing_extensions.TypedDict instead: it's up-to-date with the features on typing.TypedDict, it's much more comprehensively tested, and it has up-to-date stubs.

Thoughts?

.travis.yml: The 'sudo' tag is now deprecated in Travis CI

Travis are now recommending removing the sudo tag.

"If you currently specify sudo: false in your .travis.yml, we recommend removing that configuration"

install_requires in setup.py invalid

Currently it state it requires a python version 3.5 or lower

It also currently fails running:

python3 setup.py
error in mypy_extensions setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Expected ',' or end-of-list in typing >= 3.5.3; python_version < "3.5" at ; python_version < "3.5"

pipenv lock sha256 hash doesn't match

I used a pipenv to manage my packages.

In Pipenv.lock, there is

        "mypy-extensions": {
            "hashes": [
                "sha256:a161e3b917053de87dbe469987e173e49fb454eca10ef28b48b384538cc11458"
            ],
            "version": "==0.4.2"
        },

But recently when I use pipenv sync, I got the install error.

Installing initially failed dependencies...
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 2611, in do_sync
[pipenv.exceptions.InstallError]:       system=system,
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 859, in do_install_dependencies
[pipenv.exceptions.InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 763, in batch_install
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python2.7/site-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Collecting mypy-extensions==0.4.2 (from -r /var/folders/dz/75zwm6t9767g27bmhk4gcq440000gn/T/pipenv-WW4j5b-requirements/pipenv-4mvKDb-requirement.txt (line 1))', '  Using cached https://files.pythonhosted.org/packages/55/aa/2df16f9c6215b10c472fd7564b5f2c5644c13ab9c26d83b1c9373c11d97d/mypy_extensions-0.4.2-py2.py3-none-any.whl']
[pipenv.exceptions.InstallError]: ['ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.', '    mypy-extensions==0.4.2 from https://files.pythonhosted.org/packages/55/aa/2df16f9c6215b10c472fd7564b5f2c5644c13ab9c26d83b1c9373c11d97d/mypy_extensions-0.4.2-py2.py3-none-any.whl#sha256=c77e34350c01523cd63b6dc30b36515107b2c3cbacc8392d51d33ea18d59f8f1 (from -r /var/folders/dz/75zwm6t9767g27bmhk4gcq440000gn/T/pipenv-WW4j5b-requirements/pipenv-4mvKDb-requirement.txt (line 1)):', '        Expected sha256 a161e3b917053de87dbe469987e173e49fb454eca10ef28b48b384538cc11458', '             Got        c77e34350c01523cd63b6dc30b36515107b2c3cbacc8392d51d33ea18d59f8f1']

which the expected sha256 has is dismatched.

I wonder if the pypi updated that version.

Default Values for TypedDict

It would be nice if you could set default values for a TypedDict:

class Coordinate(TypedDict):
    x: int = 0
    y: int = 0
    z: int = 0

# current behavior
assert Coordinate() == {}
# desired behavior
assert Coordinate() == {"x": 0, "y": 0, "z": 0}

Make 1.0 release

Summary of changes in the release:

• Add mypyc native int types i64, i32, i16 and u8
• Drop Python 2 and 3.4 support
• mypy_extensions is no longer considered experimental
• Officially support Python versions up to 3.11 in metadata

sdist is missing tests

The sdist package at PyPI is missing tests. Please add tests to sdist package to make downstream testing easier. Thank you.

README TLDR

Hi,

Would it be possible to get a TLDR in the README of what extensions are supported?

get_type_hints crashes on TypedDict with non-builtin types from other modules

Here's a min repro of the crash:

$ cat a.py
from __future__ import annotations

from typing import Any
from mypy_extensions import TypedDict


class ATypedDict(TypedDict):
    f: Any

$ cat b.py
from a import ATypedDict
from typing import get_type_hints

print(getattr(ATypedDict, '__module__'))
get_type_hints(ATypedDict)

$ python b.py
importlib._bootstrap
Traceback (most recent call last):
  File "b.py", line 5, in <module>
    get_type_hints(ATypedDict)
  File "/usr/lib/python3.7/typing.py", line 973, in get_type_hints
    value = _eval_type(value, base_globals, localns)
  File "/usr/lib/python3.7/typing.py", line 260, in _eval_type
    return t._evaluate(globalns, localns)
  File "/usr/lib/python3.7/typing.py", line 464, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'Any' is not defined

Here's my understanding of the issue:

When using from __future__ import annotations, type annotations become ForwardRefs. get_type_hints resolves these using globals and locals. Generally, calling get_type_hints on a class doesn't require globals or locals because get_type_hints retrieves the __dict__ of class's module. However, it seems like, in some cases, the __module__ for TypedDict classes is not set to the module it is defined in, thus when get_type_hints tries to resolve the type annotation it fails.

This issue does not occur with builtin types because they are always available.

0.4.2 resubmitted to pypi?

pipenv install is currently failing for me because the hash for 0.4.2 has a different hash to what was found the first time it was locked with pipenv lock.

0.4.2 was released on 3rd October 2019

However another wheel was uploaded yesterday on the 17th October 2019

Which happens to coincide with the release of 0.4.3

It seems like 0.4.2 was re-submitted (potentially with changes) and the hash has changed for that reason. The worst case scenario would be that the package was resubmitted with malicious changes. The simplest explanation would just be that it was simply re-uploaded with some changes by mistake.

I cant be sure without confirmation here though :/

Also migrate the tests from mypy

I missed migrating the tests in when this was pulled out from mypy, since they live in mypy/tests/testextensions.py and not in the extensions directory

Mark repo as defunct?

Looks like this has been superseded by https://github.com/python/typing/tree/master/typing_extensions

Build wheels for public releases

Hi, while installing mypy I stumbled upon an error while installing and it seems that mypy-extensions is causing this.

I try to install mypy (and thus mypy-extensions) within a fresh virtual environment from powershell on a Windows 10 (1903) machine. The error can be reproduced when just installing mypy-extensions.

• Python Version: 3.7.4
• pip version: 19.3

Other packages install just fine.

Here is what I've done to reproduce it:

PS F:\Programmierung\YTThingy> virtualenv venv
Using base prefix 'c:\\users\\username\\appdata\\local\\programs\\python\\python37'
New python executable in F:\Programmierung\YTThingy\venv\Scripts\python.exe
Installing setuptools, pip, wheel...
done.
PS F:\Programmierung\YTThingy> .\venv\Scripts\activate
(venv) PS F:\Programmierung\YTThingy> python -V
Python 3.7.4
(venv) PS F:\Programmierung\YTThingy> Get-Command pip
CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     pip.exe                                            0.0.0.0    F:\Programmierung\YTThingy\venv/Script...


(venv) PS F:\Programmierung\YTThingy> Get-Command python
CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     python.exe                                         3.7.415... F:\Programmierung\YTThingy\venv/Script...


(venv) PS F:\Programmierung\YTThingy> python -m pip --version
pip 19.3 from F:\Programmierung\YTThingy\venv\lib\site-packages\pip (python 3.7)
(venv) PS F:\Programmierung\YTThingy> python -m pip install mypy-extensions
Collecting mypy-extensions
  Using cached https://files.pythonhosted.org/packages/1e/2f/7bba47eb58f62a473387cd7658dedd0bedb4b0fa9d530bbbfa0a6d23034a/mypy_extensions-0.4.2.tar.gz
    ERROR: Error [WinError 87] Falscher Parameter while executing command python setup.py egg_info
ERROR: Could not install packages due to an EnvironmentError: [WinError 87] Falscher Parameter
(venv) PS F:\Programmierung\YTThingy> python -m pip --no-cache-dir install mypy-extensions
Collecting mypy-extensions
  Downloading https://files.pythonhosted.org/packages/1e/2f/7bba47eb58f62a473387cd7658dedd0bedb4b0fa9d530bbbfa0a6d23034a/mypy_extensions-0.4.2.tar.gz
    ERROR: Error [WinError 87] Falscher Parameter while executing command python setup.py egg_info
ERROR: Could not install packages due to an EnvironmentError: [WinError 87] Falscher Parameter

(It's a german System. "Falscher Parameter" == "wrong parameter")

Convert CI from Travis to GitHub Actions?

The project has a .travis.yml file, but it seems non-functional -- at least it doesn't look like it's triggered in CI. Would you be interested in replacing it with GitHub Actions? I can send a PR doing this if so.

Make a release that supports Python 3.9

`__module__` is set wrongly when using class definition syntax

$ cat foo.py
from mypy_extensions import TypedDict


class Person(TypedDict):
    name: str

Host = TypedDict("Host", {"name": "str"})

$ python
Python 3.6.5 (default, Jun 17 2018, 12:13:06)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import foo
>>> foo.Host
<class 'foo.Host'>
>>> foo.Host.__module__
'foo'
>>> foo.Person
<class '_frozen_importlib.Person'>
>>> foo.Person.__module__
'_frozen_importlib'
>>>

Expected behavior is that __module__ should be set correctly when either definition syntax is used.

I haven't experimented yet, but I expect that with the class definition syntax, we don't need a manual frame-walking override of __module__ at all, since Python will set it correctly for us.

test_typeddict_errors fails on python3.11

Heads up, we're seeing a test failure with python 3.11.0rc2 and mypy_extensions. Python 3.11 will be released in October.

======================================================================
FAIL: test_typeddict_errors (testextensions.TypedDictTests.test_typeddict_errors)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/source/tests/testextensions.py", line 91, in test_typeddict_errors
    with self.assertRaises(TypeError):
AssertionError: TypeError not raised

----------------------------------------------------------------------
Ran 7 tests in 0.003s

FAILED (failures=1)

Expose required/non-required keys for mixed TypedDict or TypedDict base class, in runtime

MyPy documentation mentions it's possible to mix required and non-required items in TypedDict:

class MovieBase(TypedDict):
    name: str
    year: int

class Movie(MovieBase, total=False):
    based_on: str

https://mypy.readthedocs.io/en/latest/more_types.html#mixing-required-and-non-required-items

Unfortunately, I couldn't find how to see which of Movie keys are required, in runtime. I can only get a list of all fields (__annotations__) and the fact Movie is non-total (__total__).

>>> from mypy_extensions import TypedDict
>>> class MovieBase(TypedDict):
...     name: str
...     year: int
... 
>>> class Movie(MovieBase, total=False):
...     based_on: str
... 
>>> Movie.__annotations__
{'based_on': <class 'str'>, 'name': <class 'str'>, 'year': <class 'int'>}
>>> Movie.__total__
False

Alternatively I could poke into original base classes to find out which keys are required, but I it doesn't work:

>>> Movie.__bases__
(<class 'dict'>,)

I'm trying to write a function that checks if data matches its TypedDict type, in runtime. I can't implement it without knowing which keys are required.

Introduce TypeAssert to constraint checker to certain type with a proxy check

This is a follow-up to python/mypy#14846.

The suggestion is to add TypeAssert generic typing constraint utility that enforces checker to cast type per the value. This is a suggestion by @erictraut, and he explained it in https://mail.python.org/archives/list/[email protected]/thread/JTO3WKRKQFW5YFP3ZSMNTRIEA5NPJEM6/ . He supplied following example as well:

def verify_no_nones(val: list[None | T]) -> TypeAssert[list[T]]:
    if None in val:
        raise ValueError()

def func(x: list[int | None]):
    verify_no_nones(x)
    reveal_type(x)  # list[int]

def assert_is_one_dimensional(val: tuple[T, ...] | T) -> StrictTypeAssert[tuple[T] | T]:
    if isinstance(val, tuple) and len(val) != 1:
        raise ValueError()

def func(x: float, y: tuple[float, ...]):
    assert_is_one_dimensional(x)
    reveal_type(x)  # float

assert_is_one_dimensional(y)
    reveal_type(y)  # tuple[float]

I'm growing in need of this functionality by the day, and suggestion of @erictraut seems to be the solution that works perfect for me as well. I want to request getting this accepted to mypy_extensions, that is maintainers giving green-light to work on such a feature. I would be more than happy to work on this as I'm the person who is in need of it. I might need guidance though.

Do I have the OK to work on this?

update version

setup.py show version 1.0.0-dev0 but actual released version is 1.0, so now you should change to other value, example: 1.0.1-dev.

Missing 1.0.0 tag

Could you please push the tag corresponding to the 1.0.0 release on PyPI? We're using tags for packaging for Gentoo.