Comments (4)
Thanks for explaining where it comes from. From my perspective, just having the issues
as child of repo
instead of the current setup works fine, because it covers our use case 😅 I don't think there would be many cases where you specifically want "less data for your quota", but if that pops up someday, we can figure out something?
I can try and implement this, and follow a similar logic for other streams where a similar situation exists.
from tap-github.
@laurentS - Thanks for raising this! I wonder if it's best to just support issue_comments
in this case as a child of repo
, driven by the {repo}/issues/coments
endpoint instead of being a child of issues
(aka, a child of repo
instead of a grandchild of repo
). Performance would be improved, I think, to make a paginated repo-level call, rather than a separate round of calls for each issue, and it would also address your point about rate limits.
At the time when I was developing the first version of this tap, we were needing an example case to prove parent-child and grandparent-parent-child relationships for the SDK, and for this reason we probably chose a less-than-ideal retrieval path - in the interest of proving the more complex use case.
Given all the above, I think I'd be inclined to take your proposed logic in place of how we have it set now. Wdyt?
from tap-github.
Was this resolved in #9? We could close then if so 👀
from tap-github.
Indeed, thanks for pointing it out.
from tap-github.
Related Issues (20)
- Passing a username as "organizations" config value crashes the tap HOT 5
- KeyError: `commit_timestamp` HOT 5
- Field `fetched_at` in stream `extra-metrics` can be formatted as a date-time string
- Releases stream has 10,000 record limit HOT 3
- The 'pull_number' field not being populated for the 'pull_request_commits' stream HOT 5
- If a member is part of multiple teams, they will only be listed once HOT 2
- ValueError: not enough values to unpack (expected at least 1, got 0) in repository_streams HOT 1
- Incremental replication doesn't respect the current state HOT 1
- Use pre-commit.ci to lint project
- Stream `extra_metrics` fails on repos with large number of issues/PRs HOT 1
- Drop support for python 3.7 HOT 1
- Invalid SCHEMA messages are produced for deselected streams HOT 3
- Replace use of `get_next_page_token` in the tap HOT 2
- Workflow streams incorrectly claim to support incremental loading
- Hard to tell if API token is valid or not HOT 1
- Add `files` property to `CommitsStream` HOT 1
- Experiencing 401 Bad Credentials when credentials are valid
- Document `api_url_base` setting for Enterprise Server installations
- SDK Version pointing to a specific commit HOT 1
- Loader 'target-jsonl' is not known to Meltano.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tap-github.