Comments (8)
We could have an action similar to https://github.com/MeltanoLabs/tap-dbt/blob/main/.github/workflows/release.yml that only requires pushing a commit with the version bump and then manually create a GitHub release. So a single PR can be used to bump the version, groom the changelog and then the release can be created referencing the PR branch.
There's already a MeltanoLabs PyPI token in that repo, and it might make sense to make it an org-level secret if possible.
from tap-github.
@edgarrmondragon - Strangely, this looks exactly like a flow I have built before, and still I did not know this was called "Git Flow". I previously used a "development" or "develop" branch but was not yet sure if we should advocate the same here. After reading that GitFlow is now a fairly well established pattern, I'm inclined to leverage this - or something similar - as our best practice for MeltanoLabs since there's plenty of material on this topic, and post-automation, this is very straightforward to maintain.
Here was the most concise summary I could find of the flow, <3 minutes: https://youtu.be/1SXpE08hvGs
Adding to the video above, my expectation is that all release flows will be automated with CI -perhaps incrementally automating them but I'm a way that all taps and targets here in MeltanoLabs would basically use the same CI workflow files.
Still up for discussion:
- Do we use a
develop
branch as default, vs merging directly tomain
? - How do we create, number, and manage our release branches (if applicable)?
from tap-github.
Update: The hypermodern python template has some automation templates for GitHub release management: https://github.com/cjolowicz/hypermodern-python/tree/master/.github
Uses:
- Release drafter action: https://github.com/release-drafter/release-drafter
This would be the release model, as far as I understand this approach:
- PRs created as usual, with extra labels.
- PRs will be categorized in the change log so the "adds" / "changes" / "breaks" / "skips-changelog" labels are needed on the PR for guidance on how the changelog will be populated.
- Changlog updates are not needed. Instead, the PR descriptions themselves can be groomed to proactively tidy what will become the changelog entry.
- PRs are merged to
main
as usual.- Instead of committing to a git artifact, changelog drafts are created as "Release Drafts" in GitHub using their Releases API.
- The Github CI flow auto-creates (or auto-updates) the latest unpublished changes on
main
as a "Release Draft".
- The "changelog" file is actually the release draft text. That text may optionally be groomed manually between releases.
- A "publish" action is simply to change the status on the "draft" release notes to "Published", triggering this flow: https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows#release
- Publishing a release draft kicks off another CI flow to actually handle the release (performing the PyPi publish, for instance).
What I really like about this model is that we don't require extra commits to the git repo for grooming the changelog, and the release notes of the project then serve that purpose.
Still researching: where and how the version bump occurs and is committed back to the main branch.
from tap-github.
Strawman for discussion:
- Add a manually triggered CI job called
create_release
with one inputrelease_version
(passed as "X.Y.Z"). - The pipeline runs bumbversion and translates commit history into corresponding changelog entries.
- The pipeline commits the result to a new branch (
release/vX.Y.Z
) and kicks off a PR to merge that tomain
branch. - Manual review and grooming of the changelog entries can be performed on the MR's branch, per usual.
- When ready, a repo administrator sends the
/publish
slash command in a comment on that PR. - The CI pipeline listens for slash commands.
- When the
/publish
slash command occurs on a branch named likerelease/*
and the role of the comment author isAdministrator
, then the CI pipeline triggers apoetry publish
. - Manually merging the PR after a successful publish will bring
main
branch up-to-date with the latest changelog and version bumps.
from tap-github.
@edgarrmondragon - Nice! I just had a look now at the tap-dbt workflow. Probably comes as no surprise that I'm also thinking of how to make this a part of the cookiecutter eventually... 😅
I think part of the hurtle of tap developers getting to "stable" is building their own publish and versioning workflows. When those don't exist (like for my tap-athena WIP), it can create a bad experience for those users who might otherwise feel okay being early adopters.
from tap-github.
The more I read about GitFlow the more I like the idea. In particular the automation it allows:
When the source code in the
develop
branch reaches a stable point and is ready to be released, all of the changes should be merged back intomaster
somehow and then tagged with a release number. How this is done in detail will be discussed further on.Therefore, each time when changes are merged back into
master
, this is a new production release by definition. We tend to be very strict at this, so that theoretically, we could use a Git hook script to automatically build and roll-out our software to our production servers everytime there was a commit onmaster
.
(from https://nvie.com/posts/a-successful-git-branching-model/)
from tap-github.
This is great! I've also used gitflow in the past and it worked pretty well for us. I agree for this use case its common and well documented so its better than bringing our own workflow.
The only thing that I'm not clear on is how keeping main
as the default branch would work. I think making develop
the default branch is how gitflow was intended to be used. My understanding of how the release drafter would work in this context is that as we merge PRs into the default branch (develop for us) its keeping a draft release PR (to main branch) up to date, then when were ready we would merge that PR which includes all develop
features, version bump, changelogs. That merge to main would kick off the publish steps. I'm not totally clear if the draft releaser can do develop/main though, it kind of seems like it wants to only use the default branch for everything which conflicts with gitflow.
Does that make sense? Was that what you guys were thinking or am I misinterpreting anything?
from tap-github.
Theres an issue already for this but I couldnt find it. Do we have any thoughts on what we do it the pypi namespace is already taken for a package? For example the singer-io variant exists for tap-google-analytics
so we cant publish https://github.com/MeltanoLabs/tap-google-analytics. What about just github tagged releases? I think its less than ideal for dependency resolution in things like poetry (any other major issue with this?) but at least users can pin a version like git+https://github.com/MeltanoLabs/[email protected]
. Or the other option is updating the namespace with a meltanolabs prefix tap-google-analytics-mlabs
or meltano-labs-tap-google-analytics
? They get kind of ugly/verbose but it could still be the best options. What do you think?
from tap-github.
Related Issues (20)
- Passing a username as "organizations" config value crashes the tap HOT 5
- KeyError: `commit_timestamp` HOT 5
- Field `fetched_at` in stream `extra-metrics` can be formatted as a date-time string
- Releases stream has 10,000 record limit HOT 3
- The 'pull_number' field not being populated for the 'pull_request_commits' stream HOT 5
- If a member is part of multiple teams, they will only be listed once HOT 2
- ValueError: not enough values to unpack (expected at least 1, got 0) in repository_streams HOT 1
- Incremental replication doesn't respect the current state HOT 1
- Use pre-commit.ci to lint project
- Stream `extra_metrics` fails on repos with large number of issues/PRs HOT 1
- Drop support for python 3.7 HOT 1
- Invalid SCHEMA messages are produced for deselected streams HOT 3
- Replace use of `get_next_page_token` in the tap HOT 2
- Workflow streams incorrectly claim to support incremental loading
- Hard to tell if API token is valid or not HOT 1
- Add `files` property to `CommitsStream` HOT 1
- Experiencing 401 Bad Credentials when credentials are valid
- Document `api_url_base` setting for Enterprise Server installations
- SDK Version pointing to a specific commit HOT 1
- Loader 'target-jsonl' is not known to Meltano.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tap-github.