conan-io/hooks

Official Conan client hooks

hooks's Issues

[conan center] check URL points to the https://github.com/conan-io/conan-center-index

check that URL attribute of conanfile.py points to the https://github.com/conan-io/conan-center-index (not to the bincrafters or conan-community, for instance)

[conan center] check there is no author attribute

[conan-center hook] Add .cc to the list of C++ extensions

@uilianries said in #45:

could you please add ".cc". It's the Google's C++ extension. Also, ".c++" is a valid extension, according CMake.

[conan center] check there is topics attribute

Tox should run the latest Conan version

Instead of keep updating manually, we could define "latest":

https://github.com/conan-io/conan-package-tools/blob/develop/tox.ini#L5

[bug] Conan Center hook is not able to deal with shell script recipe

conan-io/conan-center-index#156 (comment)

[conan center] check there is no imports in test_package/conanfile.py

seems like imports method is not needed in test_package/conanfile.py, if we use run_environment?

binary sign hook

add hook to sign conan libraries
should invoke signtool for Windows/WindowsStore, and codesign for Macos/iOS/watchOS/tvOS.
probably, should be no-op for other platforms (like Linux).
some environment variables should specify path to the sign keys.

Create tests for Bintray plugin

Related issue: #2

Use same approach from #17 to design the tests.

[conan-center hook] Include check about recipe license being exported

There is check about package license but no one for the recipe license that should be exported with the conanfile.py

[conan center] check homepage attribute is specified

[bintray-update] Map SPDX licenses

Hi!

There is a lack between Bintray and SPDX licenses, where Bintray only supports a limited number of licenses, but any Conan recipe could have one or multiple licenses, following the SPDX format.

This feature intents to map SPDX licenses to Bintray format, fixing possible lacks.
For example:

SPDX GPL-2.0-or-later -> Bintray GPL-2.0

Regards!

[binary linter] verify OSX/iOS minimal version

Documentation about hooks

It may be too early in the process for this, but in artifactory plugins, we've given each plugin its own directory so that its test and readme functions can be clearly defined on an individual plugin-by-plugin basis. This allows us to test one plugin at a time etc.
Also this mandates an individual readme per plugin which tries to summarize what each plugin does, so a user can decide whether or not they want an individual plugin. Given how early we are in building plugins, that may be premature. On the other hand, having been responsible for the effort of creating those readmes a few years later in a repository without that documentation, it is definitely easier for everyone to do it up-front, so I figured I'd mention it now.

[binary linter] verify Android API level

[binary linter] verify standard library used (libc++ vs libstdc++)

clarify requirements

this topic actually has multiple questions to clarify

• can plug-in depend on other python libraries (e.g. lief, requests)? if so, how to specify them (e.g. requirements.txt)?
• can plug-in have build_requirements? (e.g. if it depend on external tool like git, ninja or whatever)
• can plug-in have system_requirements? (as above, to install system tools, like git)
• can plug-in have python_requirements? (to share code between plug-ins)
• can plug-in have requirements?

UPDATE: it turns out, hooks cannot use any of conan dependencies, like requirements, system_requirements, python_requirements or build_requerements, as they invoke conan methods, which has to be hooked, and it goes into infinite vicious circle.
moreover, hooks should be very careful choosing which conan API to use, in order to avoid such infinite recursion.

/cc @solvingj @Croydon @uilianries @lasote @memsharded @danimtb @jgsogo

Handle configurations of hooks

As soon as hooks start to proliferate there will be more and more environment variables to configure them, I think we should consider that risk and think about alternatives, there could be a better approach. It is time for brainstorming:

• Do nothing, hooks will start to introduce env variables (there are already some of them), I think it could be a mess in the long term

• Use a json file to handle the configuration of all the hooks (shareable with a conan config install), the HookManager could pass those variables in every function call:

  hooks_config.json

  {
      "hook_name_1": {
          "key1": "value",
          "key2": "value"
      },
      "hook_name_2": {
          "...": "..."
      }
  }

  hooks/hook_name_1.py

  ...
  def pre_export(..., config):
      assert config == {"key1": "value", "key2": "value"}
      ...
  
  def post_package(..., config):
      ...

• add them to conan.conf file inside the cache

• ...more ideas?

[conan center] check meta lines

check the following meta lines are not in conanfile.py and test_package/conanfile.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4

• shebang
• vim
• coding header

[conan center] check CMakeLists.txt file starts with cmake_minimum_required

if there is CMakeLists.txt file near to the conanfile.py, it should start from the cmake_minimum_required at very beginning
same for test_package/CMakeLists.txt (if exists)

[conan-center] Validate conandata.yml keys and fail if unknown

• For the conan-center one
• Validate the structure failing if unknown one:
  • "sources" (already documented)
  • "paches" (look latest proposal conan-io/conan-center-index#90)

How to test plugins?

Hi!

It would be nice add some tests for new plugins but using conan config to install, all tests will be copied also.

Is there some way to ignore those files using conan config install ?

[suggestion] A hook to check for Recipe's typos

Hi, friends.
Moved from conan-io/conan#5747

Users do typos from time to time, especially with exports_sources. Would be cool to catch that.

[binary linter] verify debug information available for Debug and RelWithDebInfo

[binary linter] verify ELF ABI

[bintray-update] Wont work with conan_server

Hi!

When running conan_server, the hooks bintray-update shown:

[HOOK - bintray_update.py] post_upload_recipe(): ERROR: Could not extract subject and repo from http://0.0.0.0:9300: Invalid pattern

We need to check before to extract the information if the endpoint is Bintray.

Hook to test conans.tools import

We should check that tools are imported conans. Related to: https://github.com/conan-community/community/issues/116 and conan-io/conan#4068

[conan center] check there is no system_requirements

I think we have previously agreed to disallow system_requirements in conan-center-index
so, it would be nice to check in hook there are no system_requirements methods
all exceptions will be white-listed in hook I guess

Should we rename hooks?

When I started the conan-center hook I thought it would make sense to call it the same way as the remote conan-center.

Having a look at the existing hooks I would like to have the naming "somehow" unified.

This is my proposal:

• binary-linter.py --> binary_linter.py
• bintray_update.py --> bintray_updater.py
• github-updater.py --> github_updater.py

Should we rename conan-center.py to something like conan-center_reviewer.py or conan-center_checker.py ?

cc/ @conan-io/community-leaders

[binary linter] verify endianess

[conan center] check that test_package uses run_environment instead of RunEnvironment

check that test_package/conanfile.py uses run_environment=True instead of RunEnvironment helper

[conan-center] settings.os should not be mandatory

Hi!

When building an installer package (bincrafters/pkg-config_installer), only the settings os_build and arch_build are available. However, the hook conan-center requires settings.os as well:

ERROR: [HOOK - conan-center.py] post_build(): 'settings.os' doesn't exist
'settings' possible configurations are ['arch_build', 'compiler', 'os_build']
Traceback (most recent call last):
  File "/opt/python/pyenv/versions/3.6.0/lib/python3.6/site-packages/conans/client/hook_manager.py", line 56, in execute
    method(output, **kwargs)
  File "/home/uilian/.conan/hooks/conan-center.py", line 121, in post_build
    if not _files_match_settings(conanfile, conanfile.build_folder):
  File "/home/uilian/.conan/hooks/conan-center.py", line 209, in _files_match_settings
    os = _get_os(conanfile)
  File "/home/uilian/.conan/hooks/conan-center.py", line 243, in _get_os
    if hasattr(conanfile.settings, attrib):
  File "/opt/python/pyenv/versions/3.6.0/lib/python3.6/site-packages/conans/model/settings.py", line 257, in __getattr__
    self._check_field(field)
  File "/opt/python/pyenv/versions/3.6.0/lib/python3.6/site-packages/conans/model/settings.py", line 253, in _check_field
    raise undefined_field(self._name, field, self.fields, self._parent_value)
conans.errors.ConanException: 'settings.os' doesn't exist
'settings' possible configurations are ['arch_build', 'compiler', 'os_build']

We should check if os or os_build are available.

Regards!

Create a post package hooks that checks for external "broken" links

Checks for link files that point to files outside of the package folder

Link to original issue: conan-io/conan#4147

Credentials of bintray plugin env vars

About these variables:

CONAN_LOGIN_USERNAME: Bintray login username
CONAN_PASSWORD: Bintray API KEY

I know that these variables are used in CPT, but even these I would like to rename them.
I think would make more sense to name them as "BINTRAY_XXX" even check if there are some "standard" variables maybe in the jfrog cli?

[binary linter] verify fPIC

[conan-center_reviewer] Should we consider fPIC for installer and header-only?

Hi!

Question about the hook conan-center_reviewer. It checks the fPIC, but some recipes does not have options, like installers and header-only. Maybe we could skip fPIC check (including log) if there is no option attribute computed.

Regards!

Document 'export_metadata' hook

There is no mention to it in the README.

[conan center] check there is no version if all is used

if recipe is in all directory, it shouldn't specify any version

make hook from readme generator

it would be nice to automatically update README to synchronize it with conanfile, so README always points to the correct name, version, description, etc.
could be made from: https://github.com/bincrafters/conan-readme-generator

[binary-linter] Reduce verbosity of output

Outputing 2 lines for every file in package folder is too much for a reasonable output:

[HOOK - conanio/hooks/binary-linter.py] post_package(): conan binary linter plug-in
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\conaninfo.txt" is not a executable, skipping...
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\conanmanifest.txt" is not a executable, skipping...
[HOOK - conanio/hooks/binary-linter.py] post_package(): checking file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr110.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr110d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr100.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr100d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr130.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr130d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr120.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr120d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" imports library "vcruntime140.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "vcruntime140d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" imports library "vcruntime140.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "vcruntime140d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr90.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr90d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr80.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msvcr80d.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "cygwin1.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msys-1.0.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\bin\docopt.dll" doesn't import library "msys-2.0.dll"
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\include\docopt.h" is not a executable, skipping...
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\include\docopt_util.h" is not a executable, skipping...
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\include\docopt_value.h" is not a executable, skipping...
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\lib\docopt.lib" is not a executable, skipping...
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\licenses\LICENSE-Boost-1.0" is not a executable, skipping...
[HOOK - conanio/hooks/binary-linter.py] post_package(): file "C:\Users\danimtb\.conan\data\docopt\0.6.2\danitmb\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9\licenses\LICENSE-MIT" is not a executable, skipping...

All those is not a executable, skipping... could be omitted

Centrally installed hooks vs. build reproducibility

Continuing discussion from #21 (comment)

In my opinion, hooks, as they are currently, should be akin to git hooks: not necessary for functioning, but allow complementing functionality. They should never affect the resulting build.

If I relied on a hook for capturing some build-time information in my package (as suggested on Slack, see my earlier comment, I would need a specific hook installed to be able to reproduce that package. And that requirement is not apparent from the conanfile (nor is it enforced by conan). My recipe is no longer portable, I need to take specific steps (installing a hook) before I can rebuild it.

Furthermore, it seems to me that once a hook is installed it affects all builds? (I might be wrong about this). This would make it impossible to perform functionality on just a couple of recipes.

I would propose a different, simpler model, based on function overrides. So hook functions like pre_export, post_export etc. would be simply member functions in Conanfile. That would make using the hooks very visible and environment-independent (if I clone the repo on a different machine, I don't need to perform additional hidden steps, it's there in the conanfile). This model also allows global overrides, by relying on python_requires and an intermediary base class:

class MyConanfile(Conanfile):
  def post_source(self):
    print("I'm in all recipes derived from MyConanfile")

Again, this preserves the build reproducibility.

Add Appveyor CI

We need to run tests against Conan versions in Windows too.

conan-center hook fails on symlinks

ERROR: [HOOK - conan-center.py] pre_export(): [Errno 62] Too many levels of symbolic links: '/Users/sse4/bincrafters/test-symlinks/test'
Traceback (most recent call last):
  File "/Users/sse4/bincrafters/conan/conans/client/hook_manager.py", line 58, in execute
    method(output=output, **kwargs)
  File "/Users/sse4/.conan/hooks/conan-center.py", line 58, in wrapper
    ret = func(output, *args, **kwargs)
  File "/Users/sse4/.conan/hooks/conan-center.py", line 142, in pre_export
    @run_test("RECIPE FOLDER SIZE", output)
  File "/Users/sse4/.conan/hooks/conan-center.py", line 67, in tmp
    ret = func(out)
  File "/Users/sse4/.conan/hooks/conan-center.py", line 151, in test
    total_size += os.path.getsize(file_path)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/genericpath.py", line 57, in getsize
    return os.stat(filename).st_size
OSError: [Errno 62] Too many levels of symbolic links: '/Users/sse4/bincrafters/test-symlinks/test'

repro:

mkdir test-symlinks
cd test-symlinks
conan new test/1.0@user/testing
ln -s test test
conan create . user/testing

[conan center] check for self.settings.compiler.cppstd

setting self.settings.compiler.cppstd should be removed, same check as self.settings.compiler.libcxx

[github-updater] Increase output information

The github-updater hook works silently and in my case it only outputs:

[HOOK - conanio/hooks/github-updater.py] pre_export(): no topics to update

I don't know if it is done on purpose, but it should say what this it is updating in github.

[suggestion] Move the check for mixed settings into hooks

conan-io/conan#4602

There is no need to scare consumers of the recipe by these warnings.

[binary linter] verify shared library exports

[conan center] check recipe doesn't export LICENSE.md / LICENSE

check that recipe doesn't export the LICENSE.md (or just LICENSE) file
the following shouldn't be in conanfile.py:

export = ["LICENSE.md"]

[Conan Center] fPIC pre-export check is weak

Coming from conan-io/conan-center-index#123 (comment) cc/ @ericLemanissier

In conan-io/conan-center-index#82, the PR passed without the conan center hook erroring on fPIC not managed correctly (KB-H007).

I suspect this was because the hook checks literally for "fpic" and the recipe has 'fpic' as showed here:

hooks/hooks/conan-center.py

Line 150 in cb81c49

if '"fpic"' in low:

This check should be improved.

SPDX license check hook

hook to verify license attribute of conanfile matches SPDX identifier
from:
conan-io/conan#3498
conan-io/conan#2342

/cc @liberforce @weatherhead99