autopub/autopub

Automatically publish package releases upon pull request merge

AutoPub

AutoPub enables project maintainers to release new package versions to PyPI by merging pull requests.

Environment

AutoPub is intended for use with continuous integration (CI) systems such as GitHub Actions, CircleCI, or Travis CI. Projects used with AutoPub are built via build and published via Twine. Contributions that add support for other CI and build systems are welcome.

Configuration

AutoPub settings can be configured via the [tool.autopub] table in the target project’s pyproject.toml file. Required settings include Git username and email address:

[tool.autopub]
git-username = "Your Name"
git-email = "[email protected]"

Release Files

Contributors should include a RELEASE.md file in their pull requests with two bits of information:

• Release type: major, minor, or patch
• Description of the changes, to be used as the changelog entry

Example:

Release type: patch

Add function to update version strings in multiple files.

Usage

The following autopub sub-commands can be used as steps in your CI flows:

• autopub check: Check whether release file exists.
• autopub prepare: Update version strings and add entry to changelog.
• autopub build: Build the project.
• autopub commit: Add, commit, and push incremented version and changelog changes.
• autopub githubrelease: Create a new release on GitHub.
• autopub publish: Publish a new release.

For systems such as Travis CI in which only one deployment step is permitted, there is a single command that runs the above steps in sequence:

• autopub deploy: Run prepare, build, commit, githubrelease, and publish in one invocation.

autopub's Issues

Explicit no release

Apologies if I missed something, but there is no explicit option in the RELEASE.md file for not triggering a release, right?

(Actually, it seems that by not writing one of the three release types, a release is averted, but that results in an error, so it is not very graceful.)

The motivation is documenting changes in the changelog.

Maybe it would be sufficient to update the README of this project to make it clear that this is possible already.

autopub fails to push release to PyPI

So I just pushed an update to (Pelican's) image-process, and autopub errored out saying the git tag didn't exist (but it does, it created it), and so didn't add the wheel to the GitHub release or push the wheel to PyPI.

• GitHub Action run -- https://github.com/pelican-plugins/image-process/runs/5613836396?check_suite_focus=true
• Release -- https://github.com/pelican-plugins/image-process/releases/tag/3.0.0

Any ideas on how to fix this? I think it's something in autopub that broke....

Add option to increase verbosity/logging

This is a little improvement that comes in my mind...
It should be nice to have an option to increase verbosity and logging...

Include dates with release

One of the pieces I look for in changelog is the date of the releases. This helps me understand how recently a project has been update, and so allows me to guess what versions of Python it might support, for example.

Would it be possible for AutoPub to automatically include the date the release is cut when it updates the changelog? I envisions this either as an extension to the header (current just the release version) or as a single line just under the header.

Support python 3.6?

Hi :)
Just noted that autopub requires python 3.7, unfortunately, I have some environment that still uses python 3.6 (as AWS Elastic Beanstalk only recently provides support for python 3.7) so my library that is used as a dependency in those environments can't be in python 3.7...

Would it be difficult in your opinion @justinmayer to provide support for python 3.6 as well?

Thanks :)

Update "contributed by" message to mention contributor

Not long ago GitHub released an updated to releases that shows an avatar of the people mentioned in the release:

See https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository (step 7)

It would be nice if we updated our message to do a @mention to the user so they get recognised in the release too 😊

It would be best to include all the people that have contributed to the PR (reviews and commits) but maybe that's more for a plugin, what do you think?

Issue with release `Release with tag_name 0.xxx.0 not found`

I've recently been getting the following error (it happened in the last 2 releases):

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.12/x64/bin/autopub", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/autopub/autopub.py", line 83, in main
    arguments.func(arguments)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/autopub/autopub.py", line 33, in githubrelease
    create_github_release()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/autopub/create_github_release.py", line 50, in create_github_release
    asset_pattern="dist/*",
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/github_release.py", line 116, in with_check_for_credentials
    return func(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/github_release.py", line 469, in gh_release_create
    gh_asset_upload(repo_name, tag_name, asset_pattern, dry_run=dry_run)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/github_release.py", line 116, in with_check_for_credentials
    return func(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/github_release.py", line 724, in gh_asset_upload
    upload_url = get_release_info(repo_name, tag_name)["upload_url"]
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/github_release.py", line 272, in get_release_info
    raise Exception('Release with tag_name {0} not found'.format(tag_name))
Exception: Release with tag_name 0.100.0 not found

See an example here: https://github.com/strawberry-graphql/strawberry/runs/5434280998?check_suite_focus=true

Not sure if something changed in GitHub's API 🤔

De-couple commit tagging from GitHub release function

Current Flow

The current flow looks something like:

1. autopub prepare: Increment version number, append to CHANGELOG, etc.
2. autopub build: Build the package
3. autopub commit: git commit the changes from the first step
4. autopub githubrelease: git tag and publish release to GitHub (single step)
5. autopub publish: Publish package to PyPI

The reason for this order is that the autopub githubrelease step needs both the built package as well as the Git tag.

Issue

Let's say you want to build the package after the commit has been tagged. Since the package must be built before it is handed to GitHub release process, there is currently no easy way to do things in this order in an AutoPub context because the autopub githubrelease step performs both tagging and releasing in a single step.

Proposal

If the git tag step were to be moved into a separate function, along with a corresponding new command autopub tag, then an alternative flow supporting the above use case could look something like:

1. autopub prepare: Increment version number, append to CHANGELOG, etc.
2. autopub commit: git commit the changes from the first step
3. autopub tag: git tag the above commit, with tag as return value
4. autopub build: Build the package
5. autopub githubrelease --tag 4.9: Pass tag return value to GitHub release function
6. autopub publish: Publish package to PyPI

(If no --tag flag is passed to autopub githubrelease, then that step would continue to function as before, with tag+release performed in a single step, for backwards compatibility for folks who currently expect autopub githubrelease to do what it currently does.)

Make GitHub release optional

Hello,

I would like to use this tool to automate the versioning and releases of my project, but as our softwore is not open source and is not hosted on GitHub should be better to not have to install githubrelease library in the deploy process...

Maybe just a try/except in the import could already fix...

Ideas/proposal for AutoPub 1.0

Hi everyone, I wanted to write this a long time ago. I've been using autopub for a while on Strawberry's repo and I think it is time to think about AutoPub 1.0.

I'm going to list some of the use cases we have in Strawberry, and then some use cases we might want to make autopub more flexible.

• update the GitHub release with the information of PR author and with a link to the PR
  • Personally I'd also list reviewers and commiters, we do get PRs that have commits from multiple people sometimes
• send a tweet after the PR has been released, to thank the authors
  • again, we should do this for reviewers and committers too
• pre-release
  • we currently have a manual command that creates a pre-release, this was done by @marcoacierno (thanks again) and it has been very helpful. I think we could have a autopub pre-release

and some other use cases I think we should support

• other package managers other than poetry
  • there's tools like hatch, pdm and probably more
• allow publishing on private repositories
• add support for CalVer (we know people that prefer CalVer to semver[1])
  • this might also mean we do some tweaks to the release file

I don't have a lot of ideas on how we build this, yet. I thought having a autopub-core that knows the basics (reading the release file, getting info about a PR, etc) and a few autopub-integration (eg autopub-poetry) that know how to use the individual package managers to build and publish. And for things that aren't related to autopub we can use plugins.

[1] maybe we should use frontmatter for defining metadata inside the release file :)

Vendor the `githubrelease` module

The current shipped githubrelease package on PyPI is lacking a fix for a problem affecting the use of implicitly-populated GITHUB_TOKEN environment variables. To be clear, the problem is already fixed via scikit-build/github-release#73 but isn't yet available in a shipped release on PyPI.

Moreover, there is a problem with that project's setup.py configuration that prevents installing directly from the githubrelease repository's master branch. Related PR: scikit-build/github-release#76

After thinking of the many hours I have spent dealing with this issue as well as other problems related to githubrelease, I am currently of the mind that we should vendor this module inside AutoPub. That would give us agency and direct control over its behavior, without having to be reliant on the upstream project.

Related discussion:

• pelican-plugins/photos#112
• pelican-plugins/photos#99 (comment)

Support for plugins

I've been working on an action to send tweets when we get new releases for Strawberry and my current implementation doesn't feel great.

Maybe autopub could support plugins so we do similar things more easily?

Just thinking out loud, maybe this is not that useful ^^

Question on Autopub compatibility with different changelog and release methods

Hello @justinmayer and @patrick91,
I'm closely following the exciting developments of Autopub, particularly the work on version 1. As an enthusiastic fan and prospective user, I have a couple of questions that I believe align with your current work.

1. Customizable Changelog Format:
Wagtail, the project I'm involved with, uses a different format for its changelog, as seen here: Wagtail CHANGELOG. Instead of a single PR per title, Wagtail lists multiple entries under a single release title, each prefixed with tags like 'Add:', 'Maintenance:', or 'Docs:'. Currently, this is managed manually without a strict format.

Question: Could Autopub support this changelog format? This feature would significantly enhance its applicability to projects like Wagtail. Perhaps through the new plugin system?

2. Separate Stages for Merging and Release:
Wagtail's workflow involves merging PRs and updating the changelog first, followed by a scheduled release according to the release plan, rather than releasing immediately after PR closure.

Question: Can Autopub accommodate this two-stage process – merging and changelog update first, followed by a later release on GitHub/PyPI?

I recognize these requests might extend beyond the immediate scope of the development of Version 1. However, I believe implementing these features could significantly expand Autopub's functionality, making it more suitable for diverse projects, including Wagtail.

Additionally, I wish to experiment with Autopub v1 with the beta project Strawberry-Wagtail, created by @patrick91. This application could serve as a novel test case and potentially pave the way for Autopub's future integration into Wagtail, especially if it can adapt to our specific changelog and release methodologies.

I'd appreciate your thoughts on these matters. Thanks!

Allow to publish prerelease from PR

Hi @justinmayer 👋

just wanted to get your opinion on adding a check for PRs that allow to trigger a prerelease from a PR, maybe by adding a label or a comment (from one of the core devs).

What do you think? I think it might be extremely useful for big releases 😊

Publish new package version on S3 private Bucket

We start to use s3pypi to upload a new version on our private s3 bucket.
For now, I wrote a little poetry script to run the command to push the generated tar.gz file of poetry to build on s3, but could be nice maybe integrate this step as a command of autopub (instead of github)

Migrate commans Cleo command-line

Refactoring/improve the commands using Cleo like poetry does see commands.

I think Cleo is a nice tool to write command line commands like poetry, and it enables to write tests! I think we could think to migrate to it in order to clean up some scripts and start to write some tests :)

What do you think @justinmayer?

Add contributors names to changelog

Hi @justinmayer would be possible to add an option (or make it default) that tells the bot to add the contributor name/username to the changelog? Would be nice to give recognition to people that send PRs :)

/cc @marcoacierno

404 Client Error: Not Found for url

Maybe I have some wrong tokens though, do you know what's the issue?
https://github.com/nrbnlulu/qtgql/actions/runs/4052819281/jobs/6972689626

Creating virtualenv qtgql-vCo0FpIN-py3.11 in /home/runner/.cache/pypoetry/virtualenvs
Building qtgql (0.103.0)
To https://github.com/nrbnlulu/qtgql
   1b126c3..18e9b1a  HEAD -> main
To https://github.com/nrbnlulu/qtgql
 * [new tag]         0.103.0 -> 0.103.0
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.1/x64/bin/autopub", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/autopub/autopub.py", line 83, in main
    arguments.func(arguments)
  File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/autopub/autopub.py", line 33, in githubrelease
    create_github_release()
  File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/autopub/create_github_release.py", line 45, in create_github_release
    gh_release_create(
  File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/github_release.py", line 132, in with_check_for_credentials
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/github_release.py", line 545, in gh_release_create
    response.raise_for_status()
  File "/opt/hostedtoolcache/Python/3.11.1/x64/lib/python3.11/site-packages/requests/models.py", line 10[21](https://github.com/nrbnlulu/qtgql/actions/runs/4052819281/jobs/6972689626#step:5:22), in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.github.com/repos/nrbnlulu/qtgql/releases

Note that I tried to re-run this workflow and it caused another error.
https://github.com/nrbnlulu/qtgql/actions/runs/4052819281/jobs/6972716375

Ensure `CHANGELOG.md` has a trailing newline

If CHANGELOG.md does not exist, AutoPub will create it when AutoPub is used to publish the first release. But when it creates and populates that file for the first time, AutoPub appears to do so without ensuring that there is a trailing newline, which subsequently causes a CI error because the end-of-file-fixer hook fails:

fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing CHANGELOG.md

Example: https://results.pre-commit.ci/run/github/495147975/1682663315.jFdnAyZWRyKjOY8ywe3eiQ

Write CHANGELOG header by default

By mistake, I didn't include the initial header in my CHANGELOG.md

CHANGELOG
=========

Autopub is looking for those lines in order to understand where to start writing the RELEASE notes... As it didn't find them continue to overwrite the release lifting only the lasted int he CHANGELOG.md file...

I think the TODO should be:

• create CHANGELOG.md file if not found
• create with the initial header
• Mention in the README/docs how to write initial CHANGELOG header [and mention the options changelog-header and changelog-file]

Improve documentation?

It was my first time with AutoPub and, perhaps, I am stupid, but I did not realise after reading the (terse) documentation how it works. Given that you can't easily revert something that has been publicly released, a better more transparent documentation would save some head-ache for the newcomers.

Particularly, I suggest to explicitly state that AutoPub is using semantic versioning and will bump the version according to the release type (major.minor.patch). If there were any previous GitHub releases, AutoPub will take that as the starting point for the version calculation. Also, that it will tag the repository with the updated version it calculated before publishing the package to PyPI.

If the README.md had something like the above, I would not end up with version 2.0.0 for my initial submission to PyPI :)))

protected main branch support?

remote: error: GH006: Protected branch update failed for refs/heads/main.        
remote: error: Changes must be made through a pull request.     

https://github.com/nrbnlulu/qtgql/actions/runs/4051372691/jobs/6969636236

Error with httpx

I have seen this before, but I can't recall how I resolved the issue. I have encountered this problem again, so I am posting it here. The error is: httpx.LocalProtocolError: Illegal header value b'Bearer '

The traceback: https://github.com/pelican-plugins/search/runs/7675586108?check_suite_focus=true

Can anyone think of a potential explanation for this problem, and perhaps how it should be resolved? (cc: @patrick91 @marcoacierno)

Not updating version in pyproject.toml

Hi again!

I'm trying to run autopub in a Jenkins job, almost everting fine but when it comes the time to upload the library after the poetry build, it seems the version in the file pyproject.toml is still the old one...

This is the script in the job:

python3.7 -m venv $WORKSPACE/myenv
. $WORKSPACE/myenv/bin/activate

# I know that poetry install will automate this couple of lines, It's just the first try...

pip install -r  $WORKSPACE/mylibrary/requirements.txt
pip install --pre autopub
pip install poetry
pip install githubrelease

cd $WORKSPACE/mylibrary/

# this to be sure that the dist directory is cleaned...
rm -rf $WORKSPACE/mylibrary/dist
export PROJECT_ROOT=$WORKSPACE"/mylibrary"

autopub check
autopub prepare


git remote -v
git push --dry-run
autopub commit																																																		
poetry build
poetry run publish   	

this is the output

+ autopub check
+ autopub prepare
+ git push --dry-run
Everything up-to-date
+ autopub commit
###########################
###########################
+ poetry build
Building mylibrary (0.1.1)
 - Building sdist
 - Built mylibrary-0.1.1.tar.gz

 - Building wheel
 - Built mylibrary-0.1.1-py3-none-any.whl
+ poetry run publish
error: mylibrary-0.1.1 already exists! You should use a different version (use --force to override).
Finished: SUCCESS

this is my pyproject.toml

[tool.poetry]
name = "mylibrary"
version = "0.1.1"

At the end of the building CHANGELOG is updated with the new version (0.1.2) but my pyproject.toml is untouched...

Any ideas why is not updating? Do I have to include other parameters somewhere??

Thank you :)