Giter Site home page Giter Site logo

python-medikit / medikit Goto Github PK

View Code? Open in Web Editor NEW
38.0 4.0 4.0 578 KB

Strongly opinionated python project management.

Home Page: https://python-medikit.github.io/

Makefile 3.22% Python 91.82% Jinja 4.96%
python python3 release-engineering release-automation release-management

medikit's Introduction

✚ medikit ✚

Strongly opinionated python 3.5+ project management.

Continuous Integration Status

Coverage Status

Documentation Status

License Status

Medikit is the first-aid toolkit to manage your project's boilerplate, like package files, versions, config, test suite, runners, ...

This package helps you create python (or not) source trees using best practices (or at least the practices we consider as best for us) in a breeze.

Don't worry about setting up git, a makefile, usual project targets, unit tests framework, pip, wheels, virtualenv, code coverage, namespace packages, setup.py files ...

Medikit's got you covered on all this, using one simple and fast command.

Install

Before installing the package, you must make sure that pip and virtualenv are installed and available to be used in your current environment.

pip install medikit

Now, you may want to bootstrap a python package source tree.

mkdir my.awesome.pkg
cd my.awesome.pkg
medikit init .

You're done with the bootstrap. You can now run:

make install
make test
git commit -m 'Damn that was fast ...'

Happy?

Update

If you change the Projectfile content, or update the library, you will need to run the generator again.

medikit update

To better control what changes are made, I suggest that you run it on a clean git repository, then look at the dofferences using:

git diff --cached

You can then commit the generated changes.

Gotchas

As the headline says, we have made strong opinionated choices about how a project tree should be organized.

For example, we choose to use make to provide the main project entrypoints (install, test). We also choose to use git. And pytest. And to put root package in the project root (as opposed to a src dir or something like this). Etc.

For beginners, that's a good thing, because they won't have to ask themselves questions like "What should I put in setup.py ?" or "Should I create a «src» dir or not ?". For more advanced users, it can be either a good thing if you agree with our choices, or a bad one ...

Architecture

Medikit uses a single configuration file in your project, called Projectfile.

This file contains all the specific of your project:

  • What features you wanna use.
  • The customizations of those features
  • The additional event listeners (more on this later) you need.
  • The eventual pipelines that you need.

At its heart, medikit uses an "event dispatcher" model.

An update will dispatch a "medikit.on_start" event, and features that you required can listen (react) to this event by adding jobs to run in response. They also can dispatch their own events.

As a result, you'll get your projects files updated, that will be a combination of all the events listeners executed.

It means two things:

  • Unlike usual project templates and generators, it can both bootstrap and update your project, as best practice evolves.
  • It's not a dependency of your project. Once it has run, you can forget it. Either you choose to maintain your project assets with it and you'll need it installed while updating, or you can remove it and just keep the generated files.

F.A.Q

  • I'm using PasteScript, isn't that enough?
    • PasteScript with the basic_package template will only generate a very minimalistic tree, while we install a few tools and generate more boilerplate than it does. The fact is, we were using it before but still had a lot of repeated actions to do then, and the exact aim of this project is to automate the whole. Also, PasteScript cannot update a project once generated, while we do.
  • Should I use it?
    • You're a grown man, right?
  • Is it stable / production ready?
    • Not really relevant to this project, as it's more a development tool than something you'll use in operations. However, please note that on some points and until version 1.0, we will tune things and change the way it works to find the most flexible way to operate. Thus, if you relly on a specific implementation, updates may break things. The good news is that you'll be able to review changes using git diff --cached, and either rollback or report issues saying how much you're disappointed (and why, don't forget the why, please).
  • Can I contribute?
    • Yes, but the right vs wrong choices decision is up to us. Probably a good idea to discuss about it (in an issue for example) first.
  • Can you include feature «foo»?
    • Probably, or maybe not. Come on github issues to discuss it, if we agree on the fact this feature is good for a lot of usages, your patch will be welcome. Also, we're working on a simple way to write "feature plugins", so even if we don't agree on something, you'll be able to code and even distribute addons that make things work the way you like.
  • Do you support python 3?
    • Of course, and for quite some times we decided to only support python 3, as we think the "10 years incubation period" we just had is a sufficient maturation period to just forget about python 2.

License

License Status

medikit's People

Contributors

ericbolo avatar fossabot avatar hartym avatar pyup-bot avatar sfermigier avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

medikit's Issues

Django: use yapf to adjust settings?

Following code kept for reference even if not working, but it would be amazing to be able to tune settings (like ensure that something is in INSTALLED_APPS). Maybe not achievable easily and safely, though.

    # Use yapf to adjust settings (for example, add some apps).
    @subscribe('medikit.feature.django.on_configure')
    def on_django_configure(self, event):
        original_source, newline, encoding = yapf_api.ReadFile('config/settings.py')
        print(original_source, newline, encoding)
        reformatted_code, encoding, has_change = yapf_api.FormatFile(
            'config/settings.py',
            in_place=True,
        )
        print(reformatted_code, encoding, has_change)

Supports working out of git ?

Should be already enabled by the git feature, but we need to test that, and maybe have command line flags to avoid updating git repo if unwanted.

Website: entry points by role

  • Guide for project's operators (make help, etc.)
  • Guide for project's developers (make update, etc.)
  • Guide for project's maintainers (features, etc.)

Release pipeline: when publishing to pypi, "bad marshal data (unknown type code)" error

Probably linked to releasing with multiple version targets. Works with 3.5/3.6 but complains with 3.7. Need bytecode cache clearing between versions and/or regeneration? Maybe it's just because py3.7 is not released yet?

Traceback (most recent call last):
  File "setup.py", line 78, in <module>
    download_url='https://github.com/python-medikit/medikit/archive/{version}.tar.gz'.format(version=version),
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 209, in run
    os.path.join(archive_root, 'EGG-INFO'), self.zip_safe()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 245, in zip_safe
    return analyze_egg(self.bdist_dir, self.stubs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 355, in analyze_egg
    safe = scan_module(egg_dir, base, name, stubs) and safe
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 392, in scan_module
    code = marshal.load(f)

Better config using «mock recorders».

Mock/MagicMock calls are recorded in order, we may be able to improve the configuration process with a really nice syntax by recording all calls to mock objects then replaying them on real configurator objects once the config parsing is finished.

No python Projectfiles

Let's see if we can use projectfiles for no-python projects, like okdocker or maybe a node only project.

Initial Update

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create seperate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

Check system & dependency make target ?

A lightweight "make check" task could run a lightweight environment check :

  • Are all (system) deps satisfied ?
  • Are versions correct ?
  • etc.

Can enhance the DX / onboarding for medikit-unaware developpers on managed projects.

extras_require is buggy with old setuptools versions

Michael had an old setuptools version, and looks that setup.py was not parseable.

(bonobo)bash-3.2$ pip install --upgrade setuptools
Collecting setuptools
  Using cached setuptools-36.6.0-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 18.2
    Uninstalling setuptools-18.2:
      Successfully uninstalled setuptools-18.2
Successfully installed setuptools-36.6.0

(Do not work with 18.2, but works with 36.6)

Error was:

(bonobo)bash-3.2$ pip install --editable bonobo
Obtaining file:///Users/misha/pyconde17/bonobo
    Complete output from command python setup.py egg_info:
    error in bonobo setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /Users/misha/pyconde17/bonobo/

Remove tornado dependency

Tornado is only used to format logs nicely, and it's a bit overkill to depend on a web framework for this job. Let's code a minimal and similar log formatter and remove it.

Pipelines polishing

Pipelines are very raw, for now. Enhancements that would be a quick win:

  • Documentation
  • Interractive steps / confirmation steps
  • Default pipeline in a better place
  • Ability to release to PyPI from pipeline (not default, of course)
  • Ability to enhance pipeline from features, by sending events
  • List pipelines
  • Pipelines shortcuts in makefiles ?
  • Pipeline status while running (like a wizard, step x on y ....)
  • Pipelines status : show available and started pipelines
  • Better pipeline console, no reset of logger
  • When pipeline ends, make it end so one can relaunch pipeline.

Release pipeline: Sanity check before accepting a version number

While releasing, no check is done on version number. Add the following:

  • If version is lower than current, warn the user he's probably making a mistake and ask for "hard" confirmation.
  • If version is equal to current, warn and refuse if already pushed
  • If there is a version jump, warn
  • Add default ? Next patch ?
  • Add "patch"/"p", "minor"/"m", "major"/"M" string to compute the version instead of typing it.
Current version: 0.5.8 Git version: 0.5.8
Next version? 0.5.9

"Nice fatals"

Some fatals are really normal, and the stack trace dump misleads the user into thinking there is something wrong with the software.

For example:

 FileExistsError  Already started, use `medikit pipeline release start --force` to force a restart, or use `medikit pipeline release continue`.

Find a better way to output this, while not hiding anything (stack trace should be accessible, even if maybe hidden by default).

Maybe this should be a https://github.com/python-mondrian/mondrian feature ?

Warnings in generated file.

This file is generated by medikit ... Recipe is Projectfile, regenerate using "make update", "make update-requirements" or medikit update (if make update broken).

Find a way to allow prereleases, but on an optin basis.

Easy to do the following, but one should opt-in to prereleases.

--- a/medikit/feature/python.py
+++ b/medikit/feature/python.py
@@ -309,7 +309,7 @@ class PythonFeature(Feature):
                     tmpfile.name, finder=repository.finder, session=repository.session, options=pip_options
                 )
             )
-            resolver = Resolver(constraints, repository, prereleases=False, clear_caches=False, allow_unsafe=False)
+            resolver = Resolver(constraints, repository, prereleases=True, clear_caches=False, allow_unsafe=False)

             self.render_file_inline(
                 'requirements{}.txt'.format('-' + extra if extra else ''),

pipelines continue although there is an error.

pip._vendor.pkg_resources.ContextualVersionConflict: (stevedore 1.28.0 (/Users/rd/.virtualenvs/rdnet/lib/python3.6/site-packages), Requirement.parse('stevedore<1.28,>=1.27'), {'bonobo'})
make: *** [install] Error 2
Make('clean install') is complete, moving forward.

Allow to list events in command line

Something like edgy-project list-event should list all available events, by feature, so it's easier to customise Projectfile. Available event content would be nice to have, too.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.