Use location other than GitHub to download the rulesets

[cfabianski]

Use location other than GitHub to download the rulesets

[geeknik]

How often are the rules updated? If it's not hourly, here's a possible fix for settings.go without changing download locations.

--- a/settings.go
+++ b/settings.go
@@ -38,22 +38,39 @@
        for _, asset := range release.Assets {
            // we aren't expecting many found languages per repo
            for _, language := range foundLanguages {
                if asset.Name != language+".tar.gz" {
                    continue
                }

+               // Create file in rules dir
+               filepath, err := filepath.Abs(filepath.Join(bearerRulesDir, "source-"+language+".tar.gz"))
+               if err != nil {
+                   return tagName, err
+               }
+
+               // Check if the file exists and is not too old
+               if info, err := os.Stat(filepath); err == nil {
+                   // If the file is less than 1 hour old, use it instead of downloading
+                   if time.Since(info.ModTime()) < time.Hour {
+                       file, err := os.Open(filepath)
+                       if err != nil {
+                           return tagName, err
+                       }
+                       defer file.Close()
+
+                       if _, err = ReadRuleDefinitions(ruleDefinitions, file); err != nil {
+                           return tagName, err
+                       }
+
+                       continue
+                   }
+               }
+
                resp, err = http.Get(asset.BrowserDownloadUrl)
                if err != nil {
                    return tagName, err
                }
                defer resp.Body.Close()

-               // Create file in rules dir
-               filepath, err := filepath.Abs(filepath.Join(bearerRulesDir, "source-"+language+".tar.gz"))
-               if err != nil {
-                   return tagName, err
-               }
                file, err := os.Create(filepath)
                if err != nil {
                    return tagName, err
                }
                defer file.Close()

                // Copy the contents of the downloaded archive to the file
                if _, err := io.Copy(file, resp.Body); err != nil {
                    return tagName, err
                }

                // reset file pointer to start of file
                _, err = file.Seek(0, 0)
                if err != nil {
                    return tagName, err
                }

                if _, err = ReadRuleDefinitions(ruleDefinitions, file); err != nil {
                    return tagName, err
                }
            }
        }

        return release.TagName, nil
    }

[cfabianski]

Thanks for your suggestion @geeknik
This wouldn't work I'm afraid since we already have a cache in place for the rules locally as you can see here

[geeknik]

Thanks for your suggestion @geeknik This wouldn't work I'm afraid since we already have a cache in place for the rules locally as you can see here

Interesting. It doesn't seem to work though as I get rate limited after I run Bearer 3 or 4 times and then it won't run because it can't fetch the rules.