Cannot use Policy as code from CLI

[aroundthecode]

Cannot use Policy as code from CLI

[eyarz]

@zmotu I agree. this is a valid use case we didn't consider when we designed this functionality.

it's not a bug because this is the expected behavior. it should and will be changed for sure.
but, as you know, there is always the question of prioritization. Unfortunately, Datree's core maintainer team (@datreeio/developers) is not planning to address this soon.

if someone from the community wants to take the lead on the that and open a PR we will review and merge it for sure :))

[zmotu]

Thanks @eyarz, I have taken a shot at it as a first time contributor in #897.

[adifayer]

@aroundthecode According to the error you provided, it seems that your account is not in policy-as-code mode, which is required to be able to use a local policy file.
Switching PAC mode on is done via the dashboard, on the settings page. Here's a link to our docs: https://hub.datree.io/dashboard/policy-as-code

Let me know if it solved your issue :)

[aroundthecode]

the point is that I do not want to use an account to perform my validation since it's done by gitlab CLI in offline mode.

Can I use Policy-as-code in offline mode? I'm expecting it to be the very reason people want to pass local file instead of getting them remotaly.

[adifayer]

@aroundthecode You can not use offline mode without having policy-as-code mode on, therefore you have to use your Datree account in Gitlab by setting your token as an environment variable.

To summarize, here are the steps you need to take to use Datree offline within Gitlab:

1. Set Policy-as-code mode on for your account (via the dashboard)
2. Set your datree token as an env. variable in Gitlab (docs: https://hub.datree.io/cicd-examples/gitLab-ci-cd#to-do-list-for-setting-up-datree-in-your-ci)
3. Run your desired commands with the offline flags (--policy-config, --no-record)

[aroundthecode]

@adifayer thanks for your time in supporting me.

I still find quite misleading calling this "offline mode" when is mandatory to manage it from a remote web ui, is there any plan to manage this in a different way in the near future?

My point is that in the last day the default remote rules changed and I cannot enforce all of them without introducing issues in production-running container( I think that CONTAINERS_INCORRECT_READONLYROOTFILESYSTEM_VALUE is quite a strong assumption to be introduced as default rules).

At the moment I've no token in my pipeline since I was (wrongly) sure to be in a totally offline mode with no need to contact any remote site. Changing this means getting approval from Security team about the tool and meanwhile I've my CI blocked by datree so if there is no other way I'll probably have to remove datree from the flow just due to the new rules added.

Is there any alternative I didn't condered? thanks again.

[adifayer]

@aroundthecode Hey again! I made some checks again regading the offline CLI behavior, I'd love to clarify that-

1. If there's no internet access in your CI pipeline, datree should work offline out-of-the-box if you run the following command datree config set offline local and pass the --policy-config flag. Does your CI have internet access?
2. Regarding the addition of some default rules- do you pass the policy name explicitly via --policy flag? I suggest that you try to pass the policy name "Default", that way you will be able to return using the initial policy you had within your CI. Let me know if it fixed that issue :)

[aroundthecode]

HI @adifayer

by forcing Default policy via CLI I get the following differences:

+-----------------------------------+----------------------------------------------------------------+
| Enabled rules in policy "Default" | 21                                                             |
| Configs tested against policy     | 4                                                              |
| Total rules evaluated             | 21                                                             |
| Total rules skipped               | 0                                                              |
| Total rules failed                | 0                                                              |
| Total rules passed                | 21                                                             |
| See all rules in policy           | https://app.datree.io/login?t=niBiPLnjr8Fibb5HSBAmVh&p=Default |
+-----------------------------------+----------------------------------------------------------------+

while without forcing it:

+-----------------------------------+------------------------------------------------------+
| Enabled rules in policy "Starter" | 34                                                   |
| Configs tested against policy     | 4                                                    |
| Total rules evaluated             | 34                                                   |
| Total rules skipped               | 0                                                    |
| Total rules failed                | 1                                                    |
| Total rules passed                | 33                                                   |
| See all rules in policy           | https://app.datree.io/login?t=niBiPLnjr8Fibb5HSBAmVh |
+-----------------------------------+------------------------------------------------------+

So it seem the real default policy is "Starter" and not "Defaut", this solved my problem thanks a lot!
Can you please explain why this behaviour and which are the difference between the two policies?

Is there anywhere in docs some details on the logic and the timing online policies are updated to avoid this happening again ?

Thanks again for your time!

[adifayer]

@aroundthecode We made a release where we changed the default policy (the policy we validate against in cases where --policy flag wasn't passed), to have more active rules by default to cover security use cases. The update is relevant only for new users,
and since you haven't set your token, we don't recognize your account and detect you as a new user, therefore you got the new default policy (the policy for signed users will never be changed).
To avoid that, you can either set your token or just explicitly pass the --policy-flag.

[zmotu]

Thanks @adifayer for clarifying the above.

There still seems to be an outstanding use case:

CI environments that may not be air gapped (ie. have internet access) but do not want the additional dependency on the external API introduced in their CI pipelines either for security, reproducibility or reliability reasons.

I can confirm that when running in offline mode (datree config set offline local) but an internet connection is present, there continues to be the following error thrown.

to use --policy-config flag you must first enable policy-as-code mode: https://hub.datree.io/policy-as-code

Since this issue has been raised more than once (#642) can I confirm if this is considered a bug and if it will be addressed?

Datree Version: 1.8.24

[eyarz]

we will release a new version of the CLI on Sunday with the code changes that are related to this issue.
@zmotu thank you for the PR!!

[royhadad]

Released in version 1.8.39
https://github.com/datreeio/datree/releases/tag/1.8.39