The fregot from fugue

Running the test suite ruins the history of `fregot repl`

The test suite actually invokes fregot repl; which is nice because we get black-box testing. However, the issue with this is that the repl history becomes essentially worthless. We should introduce something like --no-history flag that we can add during testing.

Fregot json unmarshaling behaves differently than unmarshling in OPA.

When I run the command fregot test lib rules tests/rules I get the error below.

fregot (eval error):
  "rules\tf\aws\s3\bucket_policy.rego" (line 17, column 12):
  builtin error:

    17| policy :=  json.unmarshal(input.policy)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Expected string but got null

  Stack trace:
    rule rules.tf_aws_s3_bucket_policy.policy at rules\tf\aws\s3\bucket_policy.rego:22:13
    rule rules.tf_aws_s3_bucket_policy.transport_policy at rules\tf\aws\s3\bucket_policy.rego:40:3
    rule rules.tf_aws_s3_bucket_policy.allow at tests/rules\tf\aws\s3\bucket_ssl_test.rego:20:7
    rule rules.tf_aws_s3_bucket_policy.test_s3_bucket_sse at tests:1:1

Line 17 of bucket_policy.rego looks like: policy := json.unmarshal(input.policy)

When I run the OPA version of the command above my test pass okay:

opa test .\tests\rules\tf\aws\s3 .\rules\ .\lib\
PASS: 2/2`

In case you need it, the structure of input is below.

= {
  "id": "aws_s3_bucket.no_policy",
  "acl": "private",
  "_provider": "aws",
  "website": [],
  "replication_configuration": [],
  "cors_rule": [],
  "tags": null,
  "bucket_prefix": null,
  "policy": null,
  "server_side_encryption_configuration": [],
  "grant": [],
  "object_lock_configuration": [],
  "logging": [],
  "lifecycle_rule": [],
  "_type": "aws_s3_bucket",
  "force_destroy": true
}

Policy will either be null or contain a string of JSON.

I'm running version fregot v0.13.4

Fregot doesn't recognize raw string declaration

Wanna try out the nice debug feature of fregot, but getting the error. Looks like raw string isn't supported yet?

fregot (fatal parse error):
  "policy/tags.rego" (line 30, column 12):
  parse failed:

    30|   re_match(`^([A-Z][A-Z0-9\-]+)+`, a)
                   ^

  unexpected character '`'```

top-level variable assignments don't work

x := "bar" outside a rule doesn't work with fregot:

$ cat foo.rego
package f

x := "bar"
$ fregot repl foo.rego
fregot (fatal parse error):
  "foo.rego" (line 3, column 3):
  parse failed:

    3| x := "bar"
         ^

  unexpected :=
  expecting var
$ opa run foo.rego
OPA 0.22.0-dev (commit ea20b6a2-dirty, built at 2020-06-25T12:02:21Z)

Run 'help' to see a list of commands and check for updates.

> data
{
  "f": {
    "x": "bar"
  }
}
>

Feature Request: Loading Data Files

Hi,

Just ran into Fregot and really like it. Although you can currently load input using :input and load a rego file, it would be great if you could tell it to load a data file. Is this currently planned? Thanks

"some a, b" not allowed

The rego code is silly but illustrates the difference:

$ cat foo.rego
package f

x := { "foo", "bar" }

allow {
        some a, b
        x[a] == x[b]
}
$ opa eval -d foo.rego data.f.allow
{
  "result": [
    {
      "expressions": [
        {
          "value": true,
          "text": "data.f.allow",
          "location": {
            "row": 1,
            "col": 1
          }
        }
      ]
    }
  ]
}
$ fregot eval data.f.allow foo.rego
fregot (fatal parse error):
  "foo.rego" (line 6, column 15):
  parse failed:

    6|  some a, b
              ^

  unexpected ,
  expected newline before next statement
$

Opa doesn't support var keyword on rego files.

Fregot, has a var (keyword) to declare variables, but when use this keyword on opa check, it show us this error message:
rego_parse_error: var cannot be used for rule name.

Polices that import data are not correctly parsed by fregot

While testing fregot, I came across this weird issue that any policy which tries to import something from the data section gives parsing errors in fregot.

Example policy

package test.policy

import input
import data.data_label

match[policy.name] {
   policies := data_label[input.cutomer_id]
   policy := policies[_]
   policy.name == input.name
}

When you try to load this policy to fregot, it complains with errors of type

repl% :load basic.rego
Loading basic.rego...
fregot (compile error):
  "basic.rego" (line 9, column 4):
  unknown variable:

    9|    policy := data_label[input.cutomer_id]
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Undefined variable: data_label

The problem goes away if you comment out the data import statement and use the data variable directly. Below is the edited version which works with fregot

package test.policy

import input

match[policy.name] {
   policies := data.data_label[input.cutomer_id]
   policy := policies[_]
   policy.name == input.name
}

Although this works, but this is not good for testing policies with fregot. OPA/rego policies frequently use the data section to import stuff. fregot should correctly interpret such imports. Otherwise, the policy needs to be modified every time someone needs to use fregot.

Error: Could not unify type string with empty

The test passes with opa, but fregot failed with typecheck error:

fregot (typecheck error):
  "policy/main_test.rego" (line 28, column 5):
  Unification error:

    28|     deny == set() with input as { "resource_changes": [{ "address": "module.one", "change": { "after": { "tags": { "product": "access-management", "component": "authentication-server", "asset": "service" }}}}]}
            ^^^^^^^^^^^^^

  Could not unify type string with empty

Load a directory in repl

Is this possible? I don't see anything in :help but could be missing something.

If it doesn't exist, could it?

PS You all are heroes. Thanks for making this.

`import input` not recognized

Fregot should allow import input as blah, not just import data.blah

fregot (fatal parse error):
  "policy/exception_logic_test.rego" (line 6, column 8):
  parse failed:

    6| import input as tfplan
              ^

  unexpected input (imports should start with `data.`)

How to import "io.jwt.decode" in 'fregot repl ' ?

Hello,
My policy depends on "io.jwt.decod" OPA builtins. I tried to debug the policy using "fregot repl" . While loading the policy file, it complains

unknown call:

    6|     [_, payload, _] := io.jwt.decode(token)
                              ^^^^^^^^^^^^^^^^^^^^

  Unknown call to io.jwt.decode

Is it possible to load the opa builtins before testing of policy starts ? There is no issue while testing the policy using "opa test".

Support `data.` paths in `with` statements

Currently fregot only supports with statements that change the input. paths; not data. paths.

Issue importing fixture data from json: "expected object at the top level"

While attempting to run fregot over a GCP resource policy library I'm building (based on https://github.com/forseti-security/policy-library), repl fails to start, with a parse error while attempting to load from a valid fixture JSON file.

opa loads the file's contents into the array data.test.fixtures.gcp_bq_c002_dataset_location.assets

fregot repl policies/lib policies/validator
fregot (parse error):
  "policies/validator/test/fixtures/gcp_bq_c002_dataset_location/assets/data.json" (line 1, column 1):
  parse failed:
     1| [{
     2|     "name": "//bigquery.googleapis.com/projects/my-project/datasets/good-location",
     3|     "asset_type": "bigquery.googleapis.com/Dataset",
     4|     "resource": {
     5|         "version": "v2",
     6|         "discovery_document_uri": "https://www.googleapis.com/discovery/v1/apis/bigquery/v2/rest",
     7|         "discovery_name": "Dataset",
     8|         "parent": "//cloudresourcemanager.googleapis.com/projects/100000000000",
     9|         "data": {
    10|             "creationTime": "1605589349335",
    11|             "datasetReference": {
    12|                 "datasetId": "2338365251144589",
    13|                 "projectId": "my-project"
    14|             },
    15|             "id": "my-project:2338365251144589",
    16|             "kind": "bigquery#dataset",
    17|             "lastModifiedTime": "1605589349335",
    18|             "location": "australia-southeast1"
    19|         },
    20|         "location": "australia-southeast1"
    21|     },
    22|     "ancestors": [)
    23|         "projects/100000000000",
    24|         "folders/38708975143",
    25|         "organizations/200000000000"
    26|     ],
    27|     "update_time": "2020-11-17T05:02:30.235818Z"
    28| },
    29| {
    30|     "name": "//bigquery.googleapis.com/projects/my-project/datasets/bad-location-asia-northeast2",
    31|     "asset_type": "bigquery.googleapis.com/Dataset",
    32|     "resource": {
    33|         "version": "v2",
    34|         "discovery_document_uri": "https://www.googleapis.com/discovery/v1/apis/bigquery/v2/rest",
    35|         "discovery_name": "Dataset",
    36|         "parent": "//cloudresourcemanager.googleapis.com/projects/100000000000",
    37|         "data": {
    38|             "creationTime": "1605589349335",
    39|             "datasetReference": {
    40|                 "datasetId": "2338365251144589",
    41|                 "projectId": "my-project"
    42|             },
    43|             "id": "my-project:2338365251144589",
    44|             "kind": "bigquery#dataset",
    45|             "lastModifiedTime": "1605589349335",
    46|             "location": "asia-northeast2"
    47|         },
    48|         "location": "asia-northeast2"
    49|     },
    50|     "ancestors": [
    51|         "projects/100000000000",
    52|         "folders/38708975143",
    53|         "organizations/200000000000"
    54|     ],
    55|     "update_time": "2020-11-17T05:02:30.235818Z"
    56| },
    57| {
    58|     "name": "//bigquery.googleapis.com/projects/my-project/datasets/bad-location-asia-northeast1",
    59|     "asset_type": "bigquery.googleapis.com/Dataset",
    60|     "resource": {
    61|         "version": "v2",
    62|         "discovery_document_uri": "https://www.googleapis.com/discovery/v1/apis/bigquery/v2/rest",
    63|         "discovery_name": "Dataset",
    64|         "parent": "//cloudresourcemanager.googleapis.com/projects/100000000000",
    65|         "data": {
    66|             "creationTime": "1605589349335",
    67|             "datasetReference": {
    68|                 "datasetId": "2338365251144589",
    69|                 "projectId": "my-project"
    70|             },
    71|             "id": "my-project:2338365251144589",
    72|             "kind": "bigquery#dataset",
    73|             "lastModifiedTime": "1605589349335",
    74|             "location": "asia-northeast1"
    75|         },
    76|         "location": "asia-northeast1"
    77|     },
    78|     "ancestors": [
    79|         "projects/100000000000",
    80|         "folders/38708975143",
    81|         "organizations/200000000000"
    82|     ],
    83|     "update_time": "2020-11-17T05:02:30.235818Z"
    84| }
    85| ]
  expected object at the top level

builtin parity with OPA

Made using this command:

jq -rn --argjson fregot "$(cat extra/capabilities-master.json)" --argjson opa "$(cat v0.24.0.json)" '$opa.builtins[].name | if . | IN($fregot.builtins[].name) then "- [x] \(.)" else "- [ ] \(.)" end'

using the capabilities file of OPA v0.24.0.

Error while parsing [_, cont] := walk(structure.spec.containers[_])

Come up with a problem, where standard opa bundle . works correctly, though fregot bundle -o bun.bundle.rego */**/*.rego doesn't.
The error I receive by fregot is

fregot (fatal parse error):
"Standards/STD-OSE/v1_0_0/OSE16.rego" (line 43, column 14):
parse failed:
43| [, value] := walk(structure.spec.containers[])
^
unexpected "]"

The problem that there is no error in this construction, so I need your help in realising what's going on.

Operator error: policy working on OPA playground but not with fregot

Hi,

I'm trying to write a simple policy in rego and I would like to use fregot to make some test and to help me to troubleshoot my policies.

I've wrote a simple policy which work in OPA Playground but not with fregot. When I tried to load it from repl I've this error:


fregot (compile error):
  "    satisfied := [good | labelProvided := labels[_]; (labelProvided == labelRequired)]" (line 1, column 55):
  invalid unification:

    1|     satisfied := [good | labelProvided := labels[_]; (labelProvided == labelRequired)]
                                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You can find the policy just below:

package k8srequiredlabels

deny {
    labels := {label | input.review.object.metadata.labels[label]}
    requiredLabels := ["entity", "site"]
    labelRequired := requiredLabels[_]
    satisfied := [good | labelProvided := labels[_]; good := (labelProvided == labelRequired)]
    not any(satisfied)
    msg := sprintf("Object of kind %v does not have the required labels set. got: %v and want: %v", [input.review.kind.kind, labels, requiredLabels])
}

Version: fregot v0.13.3

Thanks.

"fregot: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory"

Downloaded version v0.13.0 and have the following error on Ubuntu 18.04 LTS:

$ fregot
fregot: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory

I do have the packages libtinfo-dev and libtinfo5 installed:

$ dpkg -l | grep libtinfo
ii  libtinfo-dev:amd64                         6.1-1ubuntu1.18.04                                  amd64        developer's library for the low-level terminfo library
ii  libtinfo5:amd64                            6.1-1ubuntu1.18.04                                  amd64        shared low-level terminfo library for terminal handling
ii  libtinfo5:i386                             6.1-1ubuntu1.18.04                                  i386         shared low-level terminfo library for terminal handling

I tried symlinking libtinfo.so.6 as follows:

$ sudo ln -s /lib/x86_64-linux-gnu/libtinfo.so.5.9 /lib/x86_64-linux-gnu/libtinfo.so.6

But Fregot now complains about ncurses:

$ fregot
fregot: /lib/x86_64-linux-gnu/libtinfo.so.6: version `NCURSES6_TINFO_5.0.19991023' not found (required by fregot)

I do have ncurses packages installed:

ii  libncurses5:amd64                          6.1-1ubuntu1.18.04                                  amd64        shared libraries for terminal handling
ii  libncurses5:i386                           6.1-1ubuntu1.18.04                                  i386         shared libraries for terminal handling
ii  libncursesw5:amd64                         6.1-1ubuntu1.18.04                                  amd64        shared libraries for terminal handling (wide character support)
ii  mtr-tiny                                   0.92-1                                              amd64        Full screen ncurses traceroute tool
ii  ncdu                                       1.12-1                                              amd64        ncurses disk usage viewer
ii  ncurses-base                               6.1-1ubuntu1.18.04                                  all          basic terminal type definitions
ii  ncurses-bin                                6.1-1ubuntu1.18.04                                  amd64        terminal-related programs and man pages
ii  ranger                                     1.8.1-0.2                                           all          File manager with an ncurses frontend written in Python
ii  tig                                        2.3.0-1                                             amd64        ncurses-based text-mode interface for Git

Any idea?

Thank you

Fregot doesn't recognize non-rego files even when OPA does

First of all - very cool looking tool. It looks like it will solve a lot of my pain points with Rego.

Now to the issue I'm having. I have one of my policy files loading a YAML file as a data source. It looks like this:

package main
import data.exceptions.filename # this refers to the file at path policy/exceptions/filename.yaml
import data.common # just where I store some common functions

a_function (param) {
  common.list_contains_value(exceptions.filename.firstfield, "valuetocheck")
}

This works fine in good ol' opa test. But when I try to run repl or test commands with fregot on these files, I get this error:

fregot repl policy/main.rego policy/main_test.rego policy/common.rego policy/exceptions/filename.yaml --watch
fregot (interpreter error):
  Unknown rego file extension: policy/exceptions/filename.yaml , expected .rego or .bundle.rego

And of course, when I try to run it without specifying the YAML file, it doesn't know what I'm referring to.


fregot repl policy/main.rego policy/main_test.rego policy/common.rego --watch

fregot (compile error):
  "policy/main.rego" (line 10, column 3):
  unknown variable:

    10|   common.list_contains_value(exceptions[filename][firstfield], "valuetocheck")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Undefined variable: exceptions

What's the root cause? Am I doing something wrong? Or is this something you can fix?

Help would be much appreciated. Thanks!

"import input.foo" ignored?

Again comparing with opa eval:

$ cat -n input.json x.rego
     1  {"foo":"BAR"}
     1  package x
     2
     3  import input.foo
     4
     5  bar = foo
$ opa eval -d x.rego -i input.json data.x.bar --format=pretty
"BAR"
$ fregot eval -i input.json data.x.bar x.rego
fregot (typecheck error):
  "x.rego" (line 5, column 7):
  Unbound variables:

    5| bar = foo
             ^^^

  The variable foo is referenced, but it is never assigned a value
$

no 'array.slice' builtin

There might be others, I haven't checked. Just stumbled here:

$ cat slice.rego
package s

arr := ["foo", "bar", "baz"]
x := array.slice(arr, 0, 1)
$ opa eval -d slice.rego data.s.x --format=pretty
[
  "foo"
]
$ fregot eval data.s.x slice.rego
fregot (renamer error):
  "slice.rego" (line 4, column 6):
  unknown function:

    4| x := array.slice(arr, 0, 1)
            ^^^^^^^^^^^^^^^^^^^^^^

  Package array is not imported.
$

Alternative to conftest

Looks like it might be possible to use this as an alternative to the conftest tool? Have you considered the possibility?

I really like the idea of being able to interactively debug the rego tests (which conftest does not support since it doesn't have a REPL). I'm not sure what might be necessary to make that happen, but I think at the very least, data loading at #205 seems necessary.

Unable to test rule which returns a set due to unknown_variable issues

Trying fregot for the first time and running into Unknown variable issues even with very simple policies. Here is what my policy looks like

package test.policy

import input

match[policy.name] {
   policies := data.policies[input.PolicyBaseMsg.TenantId]
   policy := policies[_]
   policy.name == input.name
}

This policy uses the standard constructs of input and data from rego. I have the following 2 files

data.json

{"policies": {"d66688a0-214a-4bdc-aceb-389ccc956e56": {"Policy_1": {"name": "Talha"}}}}

input.json

{"name":"Talha", "PolicyBaseMsg": {"TenantId":  "d66688a0-214a-4bdc-aceb-389ccc956e56"}}

Now I am loading all this into fregot in the following way

fregot repl --watch
F u g u e   R E G O   T o o l k i t
fregot v0.12.0 repl - use :help for usage info
repl% 


repl% :load data.json 
Loading data.json...
OK


repl% :input input.json 
repl% 



repl% :load test.rego
Loading test.rego...
Loaded package test.policy



test.policy% match
fregot (eval error):
  "test.rego" (line 11, column 9):
  unknown variable:

    11| match [[policy.name, policy.responses]]  {
                ^^^^^^^^^^^

  Unknown variable: policy_0

  Stack trace:
    rule test.policy.match at match:1:1
test.policy%

Note that fregot is complaining about some unfamiliar variable policy_0

 Unknown variable: policy_0

There is no such variable in the entire rego policy. What might be happening here?
This policy run perfectly fine with normal OPA/Rego tools.

Fregot REPL Not Recognizing Rego Errors

I'm running through the Interactively Debugging the Rego Policy Language with Fregot tutorial and the REPL isn't entering error mode when the JSON error is encountered on line 10.

Steps to reproduce:

(1) Installed Fregot v0.13.4 (fregot-v0.13.4-linux-x86_64.tar.gz).
(2) Followed steps in tutorial down to Oh No, an Error!. No error is reported, just an empty return value.
(3) Run fregot repl demo.rego --watch and continue to follow the steps. Output is shown below.

F u g u e   R E G O   T o o l k i t
fregot v0.13.4 repl - use :help for usage info
repl% :load demo.rego
Loading demo.rego...
Loaded package fregot.examples.demo
fregot.examples.demo% :input repl_demo_input.json
fregot.examples.demo% :break deny
Set breakpoint at fregot.examples.demo.deny
fregot.examples.demo% deny
21|     ami = amis[ami]
        ^^^^^^^^^^^^^^^
fregot.examples.demo(debug)% :step
10|     ami = input.resource_changes.change.after.ami
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
fregot.examples.demo(debug)% :step
(debug) finished     <--- DID NOT ENTER ERROR MODE
fregot.examples.demo%

Additionally, evaluating input.resource_changes.change does not return the expected index type error:

fregot.examples.demo(debug)% input.resource_changes.change
{}     <--- DID NOT RETURN EXPECTED evalRefArg: cannot index array with a string ERROR
fregot.examples.demo(debug)%

Any ideas?

Load Packages referenced in a rego file.

I'm using fregot to test some conftest checks as an alternative to the opa cli.

It would be nice to be able to load a rego file and have it automatically load all the imported packages so I don't have to import them all myself.

LHS of `:=` is not just limited to variables

The following snippet should work, but it does not:

test_assign_array {
  [_, num, _] := json.unmarshal("[1, 2, 3]")
  num == 2
}

Parse error when doing a union of a set literal and a set var

The repro case for this issue is:

package repro

k := "foo"
k2 := {"bar", "baz"}
k3 := {k}|k2

I get a parse error when I try to eval k3 in that file:

$ fregot eval 'k3' repro.rego
fregot (fatal parse error):
  "repro.rego" (line 5, column 10):
  parse failed:

    5| k3 := {k}|k2
                ^

  unexpected |
  expecting var

When I enter each line in the repl, I get this behavior:

repl% k := "foo"
Rule k added
repl% k2 := {"bar", "baz"}
Rule k2 added
repl% k3 := {k}|k2
= true
| k3 = {
    "baz",
    "foo",
    "bar"
  }
repl% k3
fregot (compile error):
  "k3" (line 1, column 1):
  unknown variable:

    1| k3
       ^^

  Undefined variable: k3

Thanks in advance!

Getting error - printf: bad formatting char 'v'

Getting error - printf: bad formatting char 'v'

  "policy/main.rego" (line 14, column 12):
  builtin error:

    14|     msg := sprintf("Failed for the following resources: %v", [resources])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  printf: bad formatting char 'v'```

High CPU usage during REPL debug for large input file

Versions tested with: 0.11.1/0.12

While using fregot with a a custom set of rules and leveraging the regula library, the execution hangs while invoking the main deny. I let it run for 10 mins or so and it keep pretty high CPU usage the whole time.

Steps to recreate:
fregot repl --input tf-plan.json policy
:break data.rule.<rulename>.<etc>
data.main.deny

This seems to be due to a large tf-plan file (1.1 MB). Smaller files seem to work fine with the same library of rules.

Testing Regula advanced rules?

Hello,

Is it possible to use fregot test to run unit tests of Regula advanced rules making use of fugue.resources, fugue.deny_resource and similar functions?

So far, simply running fregot test raises errors because it's not aware of the definition of these functions:

$ frego test rules/aws_s3_bucket_public_access_block.rego
 "rules/aws_s3_bucket_public_access_block.rego" (line 12, column 11):
  unknown call:

    12| buckets = fugue.resources("AWS.S3.Bucket")
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  Unknown call to fugue.resources

...

Thanks!

fugue / fregot Goto Github PK

fregot's People

Contributors

Stargazers

Watchers

Forkers

fregot's Issues

Recommend Projects

Recommend Topics

Recommend Org