conda-forge/python-feedstock

A conda-smithy repository for python.

About python-feedstock

Feedstock license: BSD-3-Clause

Home: https://www.python.org/

Package license: Python-2.0

Summary: General purpose programming language

Development: https://docs.python.org/devguide/

Documentation: https://www.python.org/doc/versions/

Python is a widely used high-level, general-purpose, interpreted, dynamic programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than would be possible in languages such as C++ or Java. The language provides constructs intended to enable clear programs on both a small and large scale.

Current build status

Travis

Azure

Variant Status linux_64_build_typedebugchannel_targetsconda-forge_python_debug linux_64_build_typereleasechannel_targetsconda-forge_main linux_aarch64_build_typedebugchannel_targetsconda-forge_python_debug linux_aarch64_build_typereleasechannel_targetsconda-forge_main linux_ppc64le_build_typedebugchannel_targetsconda-forge_python_debug linux_ppc64le_build_typereleasechannel_targetsconda-forge_main osx_64_build_typedebugchannel_targetsconda-forge_python_debug osx_64_build_typereleasechannel_targetsconda-forge_main osx_arm64_build_typedebugchannel_targetsconda-forge_python_debug osx_arm64_build_typereleasechannel_targetsconda-forge_main win_64

Current release info

Name	Downloads	Version	Platforms

Installing python

Installing python from the conda-forge channel can be achieved by adding conda-forge to your channels with:

conda config --add channels conda-forge
conda config --set channel_priority strict

Once the conda-forge channel has been enabled, libpython-static, python can be installed with conda:

conda install libpython-static python

or with mamba:

mamba install libpython-static python

It is possible to list all of the versions of libpython-static available on your platform with conda:

conda search libpython-static --channel conda-forge

or with mamba:

mamba search libpython-static --channel conda-forge

Alternatively, mamba repoquery may provide more information:

# Search all versions available on your platform:
mamba repoquery search libpython-static --channel conda-forge

# List packages depending on `libpython-static`:
mamba repoquery whoneeds libpython-static --channel conda-forge

# List dependencies of `libpython-static`:
mamba repoquery depends libpython-static --channel conda-forge

About conda-forge

conda-forge is a community-led conda channel of installable packages. In order to provide high-quality builds, the process has been automated into the conda-forge GitHub organization. The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock.

A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services. Thanks to the awesome service provided by Azure, GitHub, CircleCI, AppVeyor, Drone, and TravisCI it is possible to build and upload installable packages to the conda-forge anaconda.org channel for Linux, Windows and OSX respectively.

To manage the continuous integration and simplify feedstock maintenance conda-smithy has been developed. Using the conda-forge.yml within this repository, it is possible to re-render all of this feedstock's supporting files (e.g. the CI configuration files) with conda smithy rerender.

For more information please check the conda-forge documentation.

Terminology

feedstock - the conda recipe (raw material), supporting scripts and CI configuration.

conda-smithy - the tool which helps orchestrate the feedstock. Its primary use is in the construction of the CI .yml files and simplify the management of many feedstocks.

conda-forge - the place where the feedstock and smithy live and work to produce the finished article (built conda distributions)

Updating python-feedstock

If you would like to improve the python recipe or build a new package version, please fork this repository and submit a PR. Upon submission, your changes will be run on the appropriate platforms to give the reviewer an opportunity to confirm that the changes result in a successful build. Once merged, the recipe will be re-built and uploaded automatically to the conda-forge channel, whereupon the built conda packages will be available for everybody to install and use from the conda-forge channel. Note that all branches in the conda-forge/python-feedstock are immediately built and any created packages are uploaded, so PRs should be based on branches in forks and branches in the main repository should only be used to build distinct package versions.

In order to produce a uniquely identifiable distribution:

• If the version of a package is not being increased, please add or increase the build/number.
• If the version of a package is being increased, please remember to return the build/number back to 0.

Feedstock Maintainers

• @chrisburr
• @isuruf
• @jakirkham
• @katietz
• @mbargull
• @mingwandroid
• @msarahan
• @ocefpaf
• @pelson
• @scopatz
• @xhochy

python-feedstock's Issues

Other build flags?

Noted these additions made by Travis CI in PR ( travis-ci/cpython-builder#15 ). Was curious if it would be worthwhile to add some of these other options like --with-wide-unicode, --enable-loadable-sqlite-extensions, or --with-computed-gotos.

Building gdbm support

Currently the dbm module doesn't have gdbm support built. Would be pretty useful if we could get this to work.

Python 3.6.0b3 in conflict with itself

Maybe I am trying to install the beta version in the wrong way, but when trying to install the beta version, I get:

joris@joris-XPS-13-9350:~/scipy$ conda create -n py36 python=3.6.0b3 -c conda-forge/label/prerelease
Using Anaconda API: https://api.anaconda.org
Fetching package metadata .............
Solving package specifications: ....

The following specifications were found to be in conflict:
  - python 3.6.0b3*
Use "conda info <package>" to see the dependencies for each package.

The same if I just specify 3.6 instead of 3.6.0b3

This is on Linux (Ubuntu, 64 bit)

3.5.3-1 include tk8.6 but tk8.5 is pinned

when trying to build a library with win I get errors because there are two versions of tk available. One is directly shiped with the python packacge and this is 8.6.

Python 3.3

Would be nice to build Python 3.3, although this may be a low priority.

Source code for Python 3.3.6

This might require some none standard setting in the CI script as conda-forge is not currently building Python 3.3 packages.

Build on OSX 10.12.6 with Xcode 9.2 fails

Hello,

I am trying to debug an issue with segfaults when using GStreamer through PyGobject in conda. I have narrowed down the problem to only the conda Python binary- it works everywhere else but on Python installed through conda on OSX. Interestingly, the OSX conda build is the only one to use an old XCode compiler (600 ...), so I thought this was the likely culprit. However, when I try to build the main Python recipe, it fails.

clang-4.0: warning: no such sysroot directory: '/opt/MacOSX10.9.sdk' [-Wmissing-sysroot]

Hacked away at it by changing the conda_build_config.yaml to have /Library/Developer/CommandLineTools/SDKs/ as the location of the MacOSX sdk, but I still get a failure:

dyld: lazy symbol binding failed: Symbol not found: _utimensat
  Referenced from: /Users/kpashov/anaconda3/conda-bld/python_1522354442033/work/build-static/./python.exe
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _utimensat
  Referenced from: /Users/kpashov/anaconda3/conda-bld/python_1522354442033/work/build-static/./python.exe
  Expected in: /usr/lib/libSystem.B.dylib

I have found that this is due to some "improvements" of the SDK added by Apple: https://trac.macports.org/ticket/54893

Anyway, how do I compile Python on my machine with a newer compiler? Alternatively, where in conda can I find a version of Python compiled with a slightly newer compiler so I can test?

Python 3.6.4 is failing on Windows

Windows is failing with:

(C:\bld\python_1514037886491\_b_env) C:\bld\python_1514037886491\work\Python-3.6.4\PCbuild>"C:\Program Files (x86)\MSBuild\14.0\Bin\amd64\MSBuild.exe"  "C:\bld\python_1514037886491\work\Python-3.6.4\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m /p:Configuration=Release /p:Platform=x64 /p:IncludeExternals=true /p:IncludeSSL=true /p:IncludeTkinter=true /p:UseTestMarker= /p:GIT="C:\Program Files\Git\cmd\git.exe"          
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Platforms\x64\PlatformToolsets\v140\Toolset.targets(36,5): error MSB8036: The Windows SDK version 10.0.15063.0 was not found. Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution". [C:\bld\python_1514037886491\work\Python-3.6.4\PCbuild\pythoncore.vcxproj]

@isuruf do you have an idea if we can solve that without the "Retarget solution" suggested in the error? (So we can solve this without a Windows machine.)

OSX min version of conda-forge Python ahead of supported Anaconda min version

The minimum OSX version of software that can be built on the conda-forge channel's version of python is 10.9, but the minimum version of OSX supported from the default/anaconda channel's python is 10.7. This has caused a problem with a package where we are trying to slowly migrate to conda-forge, but now can no longer support the older OSX versions.

Having conda-forge be ahead of the default channel is fine, but because this is the fundamental python package we are talking about, I feel that there should be minimal to no differences between the channels which can cause unexpected behavior like this. We are looking at increasing the minimum OSX version of the package I am seeing the problem on, but I still wish to discuss the posiblity of re-aligning the minimum versions.

My guess is that Travis' Xcode6.4 only supports 10.9 at the earliest so that is causing the python built here to only build packages on that version or later. If this is not really feasible or possible, that is fine too, but it should probably be pointed out that there is a difference.

INSTSONAME in Makefile does not point to a .so file?

I'm not sure if it's a python or a conda build thing, so please refer me to the right channel if this is not the right place to address this issue.

After I installed Python 3.6.5 in Linux (through conda from anaconda build hc3d631a_2) I see the following in [environment root]/lib/python3.6/config-3.6m-x86_64-linux-gnu/Makefile:

LIBRARY=	libpython$(VERSION)$(ABIFLAGS).a
LDLIBRARY=      libpython$(VERSION)$(ABIFLAGS).a
BLDLIBRARY=     $(LDLIBRARY)
PY3LIBRARY=     
DLLLIBRARY=	
LDLIBRARYDIR=   
INSTSONAME=	$(LDLIBRARY)

Does this look odd as the INSTSONAME is not a .so? The reason for this is that I think some autoconf script (such as Vim when compiling with python support) would read the INSTSONAME from the Makefile and use that as the dynamic library name. And thus it will try to load the .a instead of .so and failed (?).

Here is the same file for Python 3.5.2 (lib/python3.5/config-3.5m):

LIBRARY=	libpython$(VERSION)$(ABIFLAGS).a
LDLIBRARY=      libpython$(LDVERSION).so
BLDLIBRARY=     -L. -lpython$(LDVERSION)
PY3LIBRARY=     libpython3.so
DLLLIBRARY=	
LDLIBRARYDIR=   
INSTSONAME=	libpython$(LDVERSION).so.1.0

It looks like this is what the python configure.ac is setting:

https://github.com/python/cpython/blob/master/configure.ac

My question is, should the LDLIBRARY point to a .so?

Build py3x with profiler guided optimization

This should make python a tad faster

Windows runtimes missing

Seems that Windows runtimes are not being pulled in by Python 3.5 or 3.4 AFAICT. Please see these builds for reference. Should these be added as build and run dependencies? It's possible I'm missing something as I'm not a Windows expert. So, please don't hesitate to correct me.

cc @msarahan @jjhelmus

python=3.7.0=hfd72cd7_0 breaks gcc linker

There is a regression in python=3.7.0=hfd72cd7_0 (conda-forge tip) from python=3.7.0=hc3d631a_0 (anaconda tip) that breaks the gcc linker.

I have a Python package with a trivial C module.
My setup.py:

      ext_modules=[Extension('xarray_extras.kernels.np_to_csv',
                             ['xarray_extras/kernels/np_to_csv.c'])],

Works:

$ conda create -y -c conda-forge -n t1 python=3.7.0=hc3d631a_0 gcc_linux-64
$ conda activate t1
$ python setup.py build_ext

/home/crusaderky/anaconda3/envs/t1/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -fPIC -I/home/crusaderky/anaconda3/envs/t1/include/python3.7m -c xarray_extras/kernels/np_to_csv.c -o build/temp.linux-x86_64-3.7/xarray_extras/kernels/np_to_csv.o
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/xarray_extras
creating build/lib.linux-x86_64-3.7/xarray_extras/kernels
x86_64-conda_cos6-linux-gnu-gcc -pthread -shared -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/home/crusaderky/anaconda3/envs/t1/lib -L/home/crusaderky/anaconda3/envs/t1/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/home/crusaderky/anaconda3/envs/t1/lib -L/home/crusaderky/anaconda3/envs/t1/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 build/temp.linux-x86_64-3.7/xarray_extras/kernels/np_to_csv.o -o build/lib.linux-x86_64-3.7/xarray_extras/kernels/np_to_csv.cpython-37m-x86_64-linux-gnu.so

Broken:

$ conda create -y -c conda-forge -n t2 python=3.7.0=hfd72cd7_0 gcc_linux-64
$ conda activate t2
$ python setup.py build_ext

/home/crusaderky/anaconda3/envs/t2/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -g -fwrapv -O3 -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -fPIC -I/home/crusaderky/anaconda3/envs/t2/include/python3.7m -c xarray_extras/kernels/np_to_csv.c -o build/temp.linux-x86_64-3.7/xarray_extras/kernels/np_to_csv.o
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/xarray_extras
creating build/lib.linux-x86_64-3.7/xarray_extras/kernels
gcc -pthread -shared -L/home/crusaderky/anaconda3/envs/t2/lib -Wl,-rpath=/home/crusaderky/anaconda3/envs/t2/lib,--no-as-needed -L/home/crusaderky/anaconda3/envs/t2/lib -Wl,-rpath=/home/crusaderky/anaconda3/envs/t2/lib,--no-as-needed -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 build/temp.linux-x86_64-3.7/xarray_extras/kernels/np_to_csv.o -L/home/crusaderky/anaconda3/envs/t2/lib -lpython3.7m -o build/lib.linux-x86_64-3.7/xarray_extras/kernels/np_to_csv.cpython-37m-x86_64-linux-gnu.so

    gcc: error: unrecognized command line option ‘-fstack-protector-strong’
    gcc: error: unrecognized command line option ‘-fno-plt’
    error: command 'gcc' failed with exit status 1

Notice the change from x86_64-conda_cos6-linux-gnu-gcc to just gcc when linking.

The output of env is identical in the two environments.

Debug builds?

Any idea how much work it would be to have a, say, 3.5-debug Python version? Maybe this is already available somewhere.

64-bit Windows build is failing

Currently the 64-bit windows build is failing.

link.exe /RELEASE /RELEASE /NOLOGO /MACHINE:AMD64 -dll C:\conda\conda-bld\work\Python-3.5.1\externals\tk-8.6.4.2\win\Release_AMD64_VC13\tkstub86.lib C:\conda\conda-bld\work\Python-3.5.1\externals\tcl-core-8.6.4.2\win\Release_AMD64_VC13\tclstub86.lib kernel32.lib  advapi32.lib user32.lib gdi32.lib comdlg32.lib  -out:Release_AMD64_VC13\tix84.dll @C:\Users\appveyor\AppData\Local\Temp\1\nm37AC.tmp
LINK : fatal error LNK1181: cannot open input file 'C:\conda\conda-bld\work\Python-3.5.1\externals\tk-8.6.4.2\win\Release_AMD64_VC13\tkstub86.lib' [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
NMAKE : fatal error U1077: '"c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\x86_amd64\link.exe"' : return code '0x49d' [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
  Stop.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: The command "setlocal [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: if not exist "C:\conda\conda-bld\work\Python-3.5.1\externals\tcltk64\lib\tix8.4.3\tix84.dll" goto build [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: goto :eof [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: :build [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: set VCINSTALLDIR=c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\ [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: cd /D "C:\conda\conda-bld\work\Python-3.5.1\externals\tix-8.4.3.6\win" [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: nmake /nologo -f makefile.vc MACHINE=AMD64 DEBUG=0 NODEBUG=1 TCL_MAJOR=8 TCL_MINOR=6 TCL_PATCH=4 BUILDDIRTOP="Release_AMD64_VC13" TCL_DIR="C:\conda\conda-bld\work\Python-3.5.1\externals\tcl-core-8.6.4.2" TK_DIR="C:\conda\conda-bld\work\Python-3.5.1\externals\tk-8.6.4.2" INSTALL_DIR="C:\conda\conda-bld\work\Python-3.5.1\externals\tcltk64" all install [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.MakeFile.Targets(38,5): error MSB3073: " exited with code 2. [C:\conda\conda-bld\work\Python-3.5.1\PCbuild\tix.vcxproj]
  _tkinter.c
  tkappinit.c
     Creating library C:\conda\conda-bld\work\Python-3.5.1\PCBuild\amd64\_tkinter.lib and object C:\conda\conda-bld\work\Python-3.5.1\PCBuild\amd64\_tkinter.exp
  Generating code
  Finished generating code
  _tkinter.vcxproj -> C:\conda\conda-bld\work\Python-3.5.1\PCBuild\amd64\_tkinter.pyd

C:\conda\conda-bld\work\Python-3.5.1\PCbuild>if errorlevel 1 exit 1 
Command failed: C:\Windows\system32\cmd.exe /c call bld.bat
Command exited with code 1

The last time this was working was in the staged-recipe PR. Need to track down what has changed between then and now. I'll be gone next week so if anyone wants to jump on this feel free.

Package missing in current linux-64 channels (python 3.5)

I'm getting an error any time I try to update/install a package using conda.

((nsp)) fresnel ➜ ~  conda update --all
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata .............
Solving package specifications ....
Warning: Package missing in current linux-64 channels: 
  - conda-forge::python 3.5* (target=conda-forge::python-3.5.1-0.tar.bz2), skipping
Fetching package metadata .............
Solving package specifications ....
Warning: Package missing in current linux-64 channels: 
  - conda-forge::python 3.5* (target=conda-forge::python-3.5.1-0.tar.bz2), skipping
Error: Package missing in current linux-64 channels: 
  - conda-forge::python 3.5* (target=conda-forge::python-3.5.1-0.tar.bz2)

I've tried forcing a reinstall of python:

((nsp)) fresnel ➜ ~  conda install -f python=3.5
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata .............
Solving package specifications .............

Package plan for installation in environment /home/luke/software/python/miniconda/envs/nsp:

The following NEW packages will be INSTALLED:

    python: 3.5.1-0 conda-forge

Proceed ([y]/n)? y

Pruning extracted packages from the cache ...

But then a conda update --all returns the same error as before.

dyld: Symbol not found: _fdopendir$INODE64

I got this error on OSX 10.9:

dyld: Symbol not found: _fdopendir$INODE64
  Referenced from: /Users/ci/miniconda3/envs/fk/lib/libpython3.6m.dylib
  Expected in: /usr/lib/libSystem.B.dylib

any idea?

$ sw_ver
ProductName:	Mac OS X
ProductVersion:	10.9.5
BuildVersion:	13F1077

$ nm /Users/ci/miniconda3/envs/fk/lib/libpython3.6m.dylib | grep fdopendir
                 U _fdopendir$INODE64

$ nm /usr/lib/libSystem.B.dylib | grep fdopendir
# nothing

Python 3.5.2 build 3 does not place compiled Python files in __pycache__ directory

Ths conda-forge python 3.5.2_3 packages do not have the .pyc files in the standard library in __pycache__ directory, rather they placed next to the .py directory. The 3.5.2_2 build has the in the correct location.

For example:

$ tar tvf ~/Downloads/osx-64-python-3.5.2-2.tar.bz2 | grep zipfile.*pyc
-rw-r--r--  0 travis staff    5000 Jul 25 20:38 lib/python3.5/test/__pycache__/test_zipfile64.cpython-35.opt-1.pyc
-rw-r--r--  0 travis staff    5000 Jul 25 20:38 lib/python3.5/test/__pycache__/test_zipfile64.cpython-35.pyc
-rw-r--r--  0 travis staff    5000 Jul 25 20:38 lib/python3.5/test/__pycache__/test_zipfile64.cpython-35.opt-2.pyc
-rw-r--r--  0 travis staff   44344 Jul 25 20:38 lib/python3.5/__pycache__/zipfile.cpython-35.opt-2.pyc
-rw-r--r--  0 travis staff   49866 Jul 25 20:38 lib/python3.5/__pycache__/zipfile.cpython-35.opt-1.pyc
-rw-r--r--  0 travis staff   49948 Jul 25 20:38 lib/python3.5/__pycache__/zipfile.cpython-35.pyc
-rw-r--r--  0 travis staff   75071 Jul 25 20:38 lib/python3.5/test/__pycache__/test_zipfile.cpython-35.opt-2.pyc
-rw-r--r--  0 travis staff   77015 Jul 25 20:38 lib/python3.5/test/__pycache__/test_zipfile.cpython-35.pyc
-rw-r--r--  0 travis staff   77015 Jul 25 20:38 lib/python3.5/test/__pycache__/test_zipfile.cpython-35.opt-1.pyc

vs.

$ tar tvf ~/Downloads/osx-64-python-3.5.2-3.tar.bz2 | grep zipfile.*pyc
-rw-r--r--  0 travis staff    4968 Sep  8 09:39 lib/python3.5/test/test_zipfile64.pyc
-rw-r--r--  0 travis staff    5044 Sep  8 09:38 lib/python3.5/test/__pycache__/test_zipfile64.cpython-35.opt-2.pyc
-rw-r--r--  0 travis staff    5044 Sep  8 09:37 lib/python3.5/test/__pycache__/test_zipfile64.cpython-35.opt-1.pyc
-rw-r--r--  0 travis staff   44388 Sep  8 09:38 lib/python3.5/__pycache__/zipfile.cpython-35.opt-2.pyc
-rw-r--r--  0 travis staff   49910 Sep  8 09:37 lib/python3.5/__pycache__/zipfile.cpython-35.opt-1.pyc
-rw-r--r--  0 travis staff   49911 Sep  8 09:39 lib/python3.5/zipfile.pyc
-rw-r--r--  0 travis staff   75115 Sep  8 09:38 lib/python3.5/test/__pycache__/test_zipfile.cpython-35.opt-2.pyc
-rw-r--r--  0 travis staff   76983 Sep  8 09:39 lib/python3.5/test/test_zipfile.pyc
-rw-r--r--  0 travis staff   77059 Sep  8 09:37 lib/python3.5/test/__pycache__/test_zipfile.cpython-35.opt-1.pyc

Unfortunately this results in these .pyc files being included in the correct locations in any conda package which are build with this python build in the build environment. This can bloat the package side by ~900 kB.

I believe this is just a flux of the particular version of conda-build used the build. Rolling the build number should fix this. I will investigate.

Python 3.6 pre-releases

Would be nice to build a pre-release of Python 3.6 for testing. Perhaps including it in a specific pre-release channel, conda-forge/conda-forge.github.io/#370.

Source code for Python 3.6.0a1

Python 3.6 release schedule

What new in Python 3.6 might be helpful for writing tests.

Apply find_library patch

Would be good to port the change in commit ( AnacondaRecipes@68802d1 ) to our python. It avoids needing a nasty hack of setting DYLD_LIBRARY_PATH on macOS to ensure find_library searches the conda environment associated.

xref: ContinuumIO/anaconda-issues#1716

OS X Python 3.6.0a3 package was not uploaded

Looks like the Python 3.6.0a3 package in the prerelease branch was built on Travis CI but the upload command failed:

$ ./ci_support/upload_or_check_non_existence.py ./recipe conda-forge --channel=prerelease
Traceback (most recent call last):
  File "./ci_support/upload_or_check_non_existence.py", line 10, in <module>
    from binstar_client.utils import get_binstar
ImportError: No module named 'binstar_client'

Python 2.7 from conda-forge breaks cython builds using mingw

Moving this from the conda-forge google group to here on @pelson's suggestion:
https://groups.google.com/forum/#!topic/conda-forge/vMHyvb3ogK4

I came across a bug on Windows where the python 2.7 build causes compilation failures when using cython with mingw. I create an conda environment:

conda install -n foo -c conda-forge python libpython mingw cython

And then I try to compile a cython .pyx file that's something trivial like:

test.pyx:

class A(object):
pass
cdef double func(int x):
return 1.3 * x

Then if I run:

cythonize -bi test.pyx

I get the following error:

Compiling C:\Users\josh\Desktop\test.pyx because it changed.
[1/1] Cythonizing C:\Users\josh\Desktop\test.pyx
running build_ext
building 'test' extension
C:\Users\josh\Miniconda2\envs\foo\Scripts\gcc.bat -mdll -O -Wall -IC:\Users\josh\Miniconda2\envs\foo\include -IC:\Users\josh\Miniconda2\envs\foo\PC -c C:\Users\josh\Desktop\test.c -o c:\users\josh\des
ktop\test.o
C:\Users\josh\Desktop\test.c:659:15: warning: '__pyx_f_4test_func' defined but not used [-Wunused-function]
writing c:\users\josh\desktop\test.def
C:\Users\josh\Miniconda2\envs\foo\Scripts\gcc.bat -shared -s c:\users\josh\desktop\test.o c:\users\josh\desktop\test.def -LC:\Users\josh\Miniconda2\envs\foo\libs -LC:\Users\josh\Miniconda2\envs\foo\PC
build\amd64 -LC:\Users\josh\Miniconda2\envs\foo\PC\VS9.0\amd64 -lpython27 -lmsvcr90 -o C:\Users\josh\Desktop\test.pyd
c:\users\josh\desktop\test.o:test.c:(.text+0x307): undefined reference to `__imp_Py_InitModule4'
collect2.exe: error: ld returned 1 exit status
error: command 'C:\\Users\\josh\\Miniconda2\\envs\\foo\\Scripts\\gcc.bat' failed with exit status 1

If I then go back and replace the python in the environment (which was from conda-forge) using:

conda install -c defaults python

Then the code correctly compiles.

Some google searches say that you need to define MS_WIN64. I'm not sure that it is totally fair to say that the conda-forge build "breaks" compilation using mingw, but it is nonetheless surprising when you pull in python from conda-forge after using the defaults version and code compilation starts breaking where in previously worked. I'm not sure if conda-forge should or can adjust some build parameter so that it plays nicely with the defaults libpython/mingw.

Josh

OS X "framework build"

There has been various discussions about building Python on OS X as a "framework build" so that it can properly access the Windows manager. I believe this would also mean that python and pythonw are no longer the same and that a python.app binary is include in the package.

• conda-forge mailing list discussion
• conda mailing list discussion
• conda-forge mailing list post on python.app issue

The anaconda-recipes repository has a python-3.5 recipe which provides details on how Continuum has been building Python and may be of help.

Issue #15 is related and can likely be incorperated into this.

Python doesn't require `vc` package

Regarding the VC features, the python package is not requiring the vc package (recipe), so, for example, it is allowed to install python 3.5 side-by-side with the vc 10 package, so both vc14 and vc10 features would be tracked.

What's the idea? Can this block be added to the python recipe?

requirements:
  build:
    - vc 9  # [win and py27]
    - vc 10  # [win and py34]
    - vc 14  # [win and py35]
  run:
    - vc 9  # [win and py27]
    - vc 10  # [win and py34]
    - vc 14  # [win and py35]

testing that conda works with this package

python is a core package: updating python in the root env (and getting a bad version) could potentially kill the whole conda system making a reinstall necessary. It would be nice if this package could include at least a test (outside of meta.yml, in the CI config directly) that conda itself works with this package: installing this package into the root env and installing an unrelated package and then downgrading python to an older version.

recall command

On a fresh install of python from conda-forge

The following NEW packages will be INSTALLED:

    ncurses:    5.9-9         conda-forge
    openssl:    1.0.2j-0      defaults   
    pip:        8.1.2-py27_0  conda-forge
    python:     2.7.12-1      conda-forge
    readline:   6.2-2         defaults   
    setuptools: 27.2.0-py27_0 defaults   
    sqlite:     3.13.0-1      conda-forge
    tk:         8.5.19-0      conda-forge
    wheel:      0.29.0-py27_0 conda-forge
    zlib:       1.2.8-3       conda-forge

My "up arrow" key returns the string

>>> ^[[A

right arrow

>>> ^[[C

etc. ...
at the command prompt, not the previous command. Not as effect I get with the system python

Any ideas on how could this be different please?

thank you
marqh

Remove Windows dependencies pulled in using SVN

The windows build pulls in various third party libraries (zlib, openssl, bzip2, tcl/tk, sqlite, and xz) using SVN. Providing these libraries as conda packages and linking against these would be preferred.

Running Python's test suite

Currently we don't run Python's test suite. Given how important the Python interpreter is to everything at conda-forge, maybe we should be doing this.

Two solutions for python on osx

Warning: 2 possible package resolutions (only showing differing packages):
  - python-3.5.1-0.tar.bz2, sqlite-3.9.2-0.tar.bz2
  - python-3.5.0-1.tar.bz2, sqlite-3.11.1-0.tar.bz2

We will want to move our pinning forwards to resolve this (there is no major urgency on this though)

Using wrong Tcl/Tk on Mac

On Mac this is picking up the system framework for Tcl/Tk when it should be picking up our build. We should fix this.

sys.path will use user site-packages over the environment's site-packages

Hello,

I'm following up on a recommendation from @jakirkham in the R-base feedstock in this issue:
conda-forge/r-base-feedstock#37 where we see similar behavior in R with respect to .libPaths().

According to PEP 370 there are platform-specific paths in which users can install packages which will be included in sys.path. The unfortunate result is that if you have a package installed there, it may be incompatible with a given conda environment's binaries and cause issues. Worse, the user-package is always preferred in import-order, so this will impact every environment (including root).

I'm kind of torn as to whether this is or is not a bug, but I'm reporting it in any case for further discussion.

---

Here is a demonstration showing this in a root environment from defaults and a conda-forge environment using the Python from this feedstock.

---

boring conda-info for completeness

16:33:55 ~
§ conda info

     active environment : None
       user config file : /home/evan/.condarc
 populated config files : 
          conda version : 4.4.11
    conda-build version : 3.0.28
         python version : 3.6.1.final.0
       base environment : /opt/Miniconda3  (read only)
           channel URLs : https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
          package cache : /opt/Miniconda3/pkgs
                          /home/evan/.conda/pkgs
       envs directories : /home/evan/.conda/envs
                          /opt/Miniconda3/envs
               platform : linux-64
             user-agent : conda/4.4.11 requests/2.14.2 CPython/3.6.1 Linux/4.15.7-1-ARCH arch/ glibc/2.26
                UID:GID : 1000:100
             netrc file : None
           offline mode : False

 

Set up the user-site packages and add a Hello World package:

16:33:59 ~
§ mkdir -p ~/.local/lib/python3.6/site-packages
16:34:07 ~
§ echo "print('Hello World')" > ~/.local/lib/python3.6/site-packages/hello.py

Try it in the root environment:

16:34:13 ~
§ which python
/opt/Miniconda3/bin/python
16:34:20 ~
§ python
Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:09:58) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/opt/Miniconda3/lib/python36.zip', '/opt/Miniconda3/lib/python3.6', '/opt/Miniconda3/lib/python3.6/lib-dynload', '/home/evan/.local/lib/python3.6/site-packages', '/opt/Miniconda3/lib/python3.6/site-packages']
>>>             
>>> import hello
Hello World
>>> 

Note the end of sys.path: '/home/evan/.local/lib/python3.6/site-packages', '/opt/Miniconda3/lib/python3.6/site-packages'] the .local site-packages takes precedence over the environment's.

Try it in a fresh conda-forge environment:

16:34:40 ~
§ conda create -n example -c conda-forge python=3.6

Boring output of install plan

Solving environment: done

## Package Plan ##

  environment location: /home/evan/.conda/envs/example

  added / updated specs: 
    - python=3.6


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2018.1.18          |           py36_0         143 KB  conda-forge
    setuptools-38.5.2          |           py36_0         525 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         668 KB

The following NEW packages will be INSTALLED:

    ca-certificates: 2018.1.18-0      conda-forge
    certifi:         2018.1.18-py36_0 conda-forge
    ncurses:         5.9-10           conda-forge
    openssl:         1.0.2n-0         conda-forge
    pip:             9.0.1-py36_1     conda-forge
    python:          3.6.4-0          conda-forge
    readline:        7.0-0            conda-forge
    setuptools:      38.5.2-py36_0    conda-forge
    sqlite:          3.20.1-2         conda-forge
    tk:              8.6.7-0          conda-forge
    wheel:           0.30.0-py36_2    conda-forge
    xz:              5.2.3-0          conda-forge
    zlib:            1.2.11-0         conda-forge

Proceed ([y]/n)? y


Downloading and Extracting Packages
certifi 2018.1.18: ##################################################### | 100% 
setuptools 38.5.2: ##################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use:
# > source activate example
#
# To deactivate an active environment, use:
# > source deactivate
#

 

Run the same test:

16:35:21 ~
§ source activate example
(example) 16:35:27 ~
§ which python
/home/evan/.conda/envs/example/bin/python
(example) 16:35:29 ~
§ python
Python 3.6.4 | packaged by conda-forge | (default, Dec 23 2017, 16:31:06) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/evan/.conda/envs/example/lib/python36.zip', '/home/evan/.conda/envs/example/lib/python3.6', '/home/evan/.conda/envs/example/lib/python3.6/lib-dynload', '/home/evan/.local/lib/python3.6/site-packages', '/home/evan/.conda/envs/example/lib/python3.6/site-packages']
>>> 
>>> import hello
Hello World
>>> 

Same behavior for .local site-packages as in root.

ncurses requirement not specified in os-x packages

The os-x python packages requires ncurses but fails to declare this requirement in meta.yaml. Specifically, the _ncurses _curses_panel and readline modules link against libncursesw.5.dylib.

This causes errors when the python package is installed with a version of ncurses that does not provide the libncursesw.5.dylib, see conda-forge/ncurses-feedstock#37

~$ conda create -c conda-forge -n py36_cf python=3.6
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment /Users/jhelmus/anaconda/envs/py36_cf:

The following NEW packages will be INSTALLED:

    ca-certificates: 2017.7.27.1-0 conda-forge
    ncurses:         5.9-10        conda-forge
    openssl:         1.0.2l-0      conda-forge
    python:          3.6.3-0       conda-forge
    readline:        6.2-0         conda-forge
    sqlite:          3.13.0-1      conda-forge
    tk:              8.5.19-2      conda-forge
    xz:              5.2.3-0       conda-forge
    zlib:            1.2.8-3       conda-forge

Proceed ([y]/n)? y

#
# To activate this environment, use:
# > source activate py36_cf
#
# To deactivate an active environment, use:
# > source deactivate
#

~$ conda inspect linkages --show-files -n py36_cf python
python
------

ncurses-5.9-10:
    libncursesw.5.dylib (lib/libncursesw.5.dylib) from lib/python3.6/lib-dynload/_curses.cpython-36m-darwin.so
    libncursesw.5.dylib (lib/libncursesw.5.dylib) from lib/python3.6/lib-dynload/_curses_panel.cpython-36m-darwin.so
    libncursesw.5.dylib (lib/libncursesw.5.dylib) from lib/python3.6/lib-dynload/readline.cpython-36m-darwin.so
    libpanelw.5.dylib (lib/libpanelw.5.dylib) from lib/python3.6/lib-dynload/_curses_panel.cpython-36m-darwin.so

Maintaining many versions

We need to be careful to distinguish between versions we want to support, and versions that we want to have built in the past.

The history has been really interesting to see Python 1 grow, but we don't need to maintain recipes for it. Instead, I propose we build them (i.e. merge), tag the commit that we used, and then delete the branch. Ideally, all of this would be done with a common (read: linear) git history, so that we can diff the build between two versions.

Thoughts @jjhelmus?

Numpy cannot find the MKL DLLs

Quote from @ SylvainCorlay in this comment.

@jakirkham @pelson

@aburgm just told me that the issue comes from the conda-forge package for python itself.

"Numpy cannot find the MKL DLLs which are in Library/bin. It works without Library/bin in PATH for the Python from the defaults channel but when using Python from conda-forge it needs Library/bin in the PATH. "

Sorry I'm missing some relevant context I think. (though maybe you have told me this before. sorry) What is this in relation to? Did we have some traceback of the error? What dependencies from which channels were in the environment? Based on the path issue, what version of conda was involved?

Should add that we don't package NumPy for Windows and have no MKL package. Don't think it is relevant here, but figured it should be mentioned anyways.

Also, am moving this issue to the Python issue tracker.

Build against xz 5.2.2

The latest release of xz is version 5.2.2. A build should be created which uses this version of xz.

Error when install openssl

None pkg can be installed to the conda env in my PC. The computer is offline, and the error messages are as follow:

$ conda install openssl --offline
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata:
Solving package specifications: .
Error: Dependencies missing in current linux-64 channels:

• conda -> requests -> python 3.5* -> openssl 1.0.2d
• conda -> python 3.5* -> openssl 1.0.2d
• conda -> pycosat -> python 3.5* -> openssl 1.0.2d
• conda -> conda-env -> python 3.5* -> openssl 1.0.2d
• conda -> pyyaml -> python 3.5* -> openssl 1.0.2d
• python 3.5* -> openssl 1.0.2d
• requests (target=requests-2.9.1-py35_0.tar.bz2) -> python 3.5* -> openssl 1.0.2d
• conda-env (target=conda-env-2.4.5-py35_0.tar.bz2) -> python 3.5* -> openssl 1.0.2d
• pycosat (target=pycosat-0.6.1-py35_0.tar.bz2) -> python 3.5* -> openssl 1.0.2d
• pyyaml (target=pyyaml-3.11-py35_1.tar.bz2) -> python 3.5* -> openssl 1.0.2d

Building ndbm support

Currently the dbm module doesn't have ndbm support built (on non-macOS platforms). Would be pretty useful if we could get this to work.

py 3.6 branch

hello,
I saw there's a prerelease branch, now 3.6.0 is out I guess we could have a 3.6 branch

Python 2.7.13

A new minor release of Python 2.7 has been released, 2.7.13. The 2.7 should be updated with this new version.

Python 3.4

Would be nice to build Python 3.4.

Source code for Python 3.4.4

anaconda-recipes python-3.4 recipe

Python 2.6

Would be nice to build Python 2.6, although this may be a low priority.

Source code for Python 2.6

This might require some none standard setting in the CI script as conda-forge is not currently building Python 2.6 packages.

nis doesn't build on macOS 10.11

This is more of an FYI at this point, but I'm not seeing nis build on 10.11. As we are not building on that platform yet, this is not a problem we need to concern ourselves with. However, at some point, we will. So we should keep this in mind.

Remove need for OS X failed extension hack

Currently a hack is being use to properly build some of the OS X Python extension modules. Specifically, during the make command the build script checks to see if each extension module is importable.Those that fail are renamed and not copied into the correct location. To get these modules to work the conda build script explicitly copies the files into the correct location. The post build step adjusts the library search paths in these extensions and so that they are importable, which is explicitly tested by run_tests.py.

I think these are failing because the system libraries are shadowing the conda installed libraries. Adjusting the rpath or library search path during the build may solve this issue.

Building Python 2.7 with tracemalloc support

The tracemalloc module is an extremely valuable feature in Python 3. It is possible to build Python 2 with tracemalloc support. Though it requires patching the Python 2 build. Raising this issue to see if this approach would be viable.

ref: http://pytracemalloc.readthedocs.io/install.html#manual-installation

Apply no frameworks search patch

We should be apply this patch. Basically, this stops really weird things from happening like imports coming from the system python while using python from a conda install. See this related discussion in gitter from start to end. This is only one problem that can occur without this simple patch.

Python 3.7.0 in the prerelease label

It would be great to provide Python 3.7.0 package in the prerelease label.

PEP 537 has the release schedule for the new version:

Actual :
...
* 3.7.0 beta 3: 2018-03-29

Expected:

* 3.7.0 beta 4: 2018-04-30
* 3.7.0 candidate 1: 2018-05-21
* 3.7.0 candidate 2: 2018-06-04 (if necessary)
* 3.7.0 final: 2018-06-15

Python 3.5.3

It would be nice to get Python 3.5.3. I guess it only needs the version bumped up, but I am not sure.

Python 2.7

Would be nice to build Python 2.7.

Source code for Python 2.7.11

anaconda-recipes python-2.7 recipe

~20% slower than defaults build

Using python 3.6 on Linux, pyperformance gives significantly slower speeds on a variety of benchmarks with conda-forge's package than with the one from defaults:

This might just be from the new compilers, but some of the new optimizations may only be possible because defaults statically links the python executable.

Maybe we should be doing that, and/or also enabling more compiler optimizations. (Probably should wait to work on that until conda-forge is using the updated compilers, though.)

Python 3.5.2-1 broken on Win64

The version string seems to be incorrect - see conda/conda#3022

"Bus Error" with conda-forge python packages

It seems the conda-forge recipes need to not detect binary files. This setting changed in conda-build 2.0 from opt-in to opt-out, so this will need to change with the migration to conda-build 2.

conda/constructor#57 (comment)