Add possibility to monitor a list of URLs instead of single URL

[muety]

Add possibility to monitor a list of URLs instead of single URL

[muety]

Good idea! How would you want to proceed with tolerance and XPath parameters? Would you suggest to configure them separate for each URL (see below) or globally for all? What would be appropriate for your use case?

# (url, tolerance, xpath)
https://url1.com, 75, /body
https://url2.com, 0, //div[name="article-container"]
...

[coveritytest]

Separate for each URL would be nice of course :) What do you think of json as input?

{
  "url_list": [
    {
      "url": "https://url1.com",
      "tolerance": 75,
      "xpath": "/body"
    },
    {
      "url": "https://url2.com",
      "tolerance": 0,
      "xpath": "//div[name='article-container']"
    }
  ]
}

[muety]

A JSON-formatted scrape config like the one you suggested seems to be a good option.

I'm thinking about keeping the logic for performing multiple scrapes outside of the actual watcher and provide a Bash script instead. It would read JSON config, iterate the array of URLs and parameters and call watcher.py for each entry. Doing so wouldn't add additional complexity to the actual code and have control logic separated from business logic. The only drawback would be the fact that Windows users wouldn't be able to use that functionality unless we also provide a Batch- or PowerShell script.

[coveritytest]

Yeah that's a good point, might be better to do it this way. Maybe then CSV would be easier to parse in a Bash script. On the other hand, with tools like jq JSON shouldn't be a problem either.