Comments (4)
Hello @gabrielsoltz, I'm sorry to hear you're experiencing problems.
I've been attempting to reproduce the issue you're facing. Despite several attempts, the query consistently returns the expected results.
> select count(*) from github_workflow where repository_full_name in (select name_with_owner from github_my_repository)
+-------+
| count |
+-------+
| 893 |
+-------+
Time: 175.6s. Rows fetched: 2. Hydrate calls: 2.
Could you please provide the contents of the log file from ~/.steampipe/logs/plugin-2024-*.log
for the plugin, along with the content of the workflow file that's causing the issue? Make sure to omit any sensitive information before sharing. This information will greatly assist us in further investigating the problem.
Alternatively, could you please verify whether any of your workflow files might have indentation issues?
I can remember, during my tests with the previous PR, I executed the query (select * from github_workflow where repository_full_name in (select name_with_owner from github_my_repository)
) across over 70+
repositories and 500+
workflow files without encountering any issues. The results were successful and I've included the query outcome in the PR for reference.
Thanks!
from steampipe-plugin-github.
HI @ParthaI thank you for verifying this, did you check your previous query using select *
instead of select count(*)
? This issue only happens when querying one of the content columns (workflow_content, workflow_content_json, and pipeline)... So I think that if you use count(*) it doesn't happen, you need to specify one of those columns in your query.
Here are the logs:
2024-02-16 15:20:55.327 UTC [ERROR] steampipe-plugin-github.plugin: [ERROR] 170809685441: github_workflow.decodeFileContentToPipeline: Pipeline conversion error=EOF
2024-02-16 15:20:55.327 UTC [ERROR] steampipe-plugin-github.plugin: [ERROR] 170809685441: transform decodeFileContentToPipeline returned error EOF
2024-02-16 15:20:55.327 UTC [ERROR] steampipe-plugin-github.plugin: [ERROR] 170809685441: failed to populate column 'pipeline': EOF
2024-02-16 15:20:55.327 UTC [WARN] steampipe-plugin-github.plugin: [WARN] 170809685441: QueryData StreamError failed to populate column 'pipeline': EOF (github-170809685441)
2024-02-16 15:20:55.327 UTC [WARN] steampipe-plugin-github.plugin: [WARN] 170809685441: streamRows execution has failed: github-170809685441 - calling queryCache.AbortSet (github: failed to populate column 'pipeline': EOF)
2024-02-16 15:20:55.328 UTC [WARN] steampipe-plugin-github.plugin: [WARN] 170809685441: QueryCache AbortSet - aborting request with error github: failed to populate column 'pipeline': EOF (1 subscriber) (github-170809685441)
2024-02-16 15:20:55.328 UTC [WARN] steampipe-plugin-github.plugin: [WARN] 170809685441: queryData.streamRows returned error: github: failed to populate column 'pipeline': EOF
2024-02-16 15:20:55.328 UTC [WARN] steampipe-plugin-github.plugin: [WARN] 170809685441: executeForConnection github returned error github: failed to populate column 'pipeline': EOF, writing to CHAN
2024-02-16 15:20:55.328 UTC [WARN] steampipe-plugin-github.plugin: [WARN] 170809685441: error channel received github: failed to populate column 'pipeline': EOF
2024-02-16 15:20:55.337 UTC [WARN] steampipe-plugin-github.plugin: [WARN] 170809685441: readAndStreamAsync failed to read previous rows from cache: github: failed to populate column 'pipeline': EOF publisher github-170809685441 (github-170809685441)
from steampipe-plugin-github.
Hi @ParthaI, I was able to identify the repository, triggering the error using the following script:
for i in $(steampipe query "select name_with_owner from github_my_repository" --output csv); do echo "Repo: $i" && steampipe query "select name, path, state, workflow_file_content from github_workflow where repository_full_name = '$i';"; done
You were right; there is one repository with an invalid YAML file, which is generating this problem.
I'll fix this on my side. As an idea, we can improve the error message when this happens.
I appreciate your help!
from steampipe-plugin-github.
@gabrielsoltz, I hope the issue has been resolved, as per our previous discussion. Hence, closing the issue for now, please feel free to reopen it if you encounter any further issues.
Thanks!
from steampipe-plugin-github.
Related Issues (20)
- Update plugin to use DefaultRetryConfig
- Remove `temp_clone_token` column from all repo tables.
- YAML unmarshal error in `github_workflow.pipeline` column HOT 6
- GitHub v3 API fields cause error in `github_search_repository` table HOT 1
- A way to query for assignee on Pull Requests HOT 1
- 502 Bad Gateway with repos with large amount of PRs HOT 2
- Strange behavior: "Variable $includeAllowUpdateBranch is used by anonymous query" HOT 1
- failed to populate column 'interaction_ability' HOT 3
- Variable definition error in github_branch table
- Table github_workflow fails with error when we pass all repositories HOT 6
- Using a Github App token with Steampipe doesn't work HOT 7
- Table github_search_user fails with error `Error: Variable $includeUserAnyPinnableItems is used by anonymous query but not declared`
- rate-limit makes the plugin unusable HOT 43
- Join between github_repository_dependabot_alert and github_search_pull_request results in error HOT 3
- Add table github_outside_collaborators HOT 15
- Error querying github_workflow for repository that contains dynamic workflows
- (Performance) issue with `github_outside_collaborators` table HOT 2
- Update the plugin docs to include instructions on how to use the rate limiters HOT 1
- Added support for plugin authentication using Github OAuth App. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from steampipe-plugin-github.