gusandrews/gitissuescraper

A tool to help scrape not just the issues on a GitHub project, but also the comments on each issue.

gitissuescraper's Issues

Terminology fix: it's not a "scraper".

Hey, Gus -- you're going to hate me for saying this now instead of earlier (actually, I did say it during the team call, but James and I spoke at the same time and I'm pretty sure no one heard me; I didn't repeat it then, as the topic moved on). Anyway:

A "scraper" is a program that gets information from human-readable sources that aren't intended for programs (details here and here). In other words, if your program ingests the GitHub issues page -- the same HTML page that you and I see in our browsers -- then it could "scrape" GitHub issues. But if you use the GitHub API to get issue information, then by definition that's not scraping.

Now, obviously, using the API is the right thing to do. I'm not saying any of that should change. I'm just saying don't call it a scraper or refer to what it does as scraping. If you call it a "scraper", people will get confused, because it's the opposite of a scraper -- one only writes scrapers when no API is available.

I realize that this involves not only changing the documentation and filenames and variable names in your code code, but the very name of the program itself. But actually, the name needed to change anyway, because the program is no longer only for Git issues (by the way, don't forget to update the README.md file too to indicate that support now extends beyond just GitHub).