Enhance `between --filter` option

[homeport]

Enhance `between --filter` option

[HeavyWombat]

Thanks for the feedback. I would like to have that option, too. Even started to work on it. The "problem" so to speak is the current path implementation. It is not really how I like it, not really flexible or good in terms of design. So I am not sure whether I want to hack the feature into the current design or if I finally do the redesign of the path implementation.

[shukla2009]

I am also interested in --exclude feature

[sachaos]

@shukla2009

I’ve added —exclude, —filter-regexp and —exclude-regexp options on my own fork.
This fork is not assumed to merge to upstream because I think @HeavyWombat had started to work on this issue and this issue will be resolved in the near future.

But if you want to try it before then, feel free to use it.

https://github.com/sachaos/dyff/tree/feature/filter-or-exclude-by-regexp

[HeavyWombat]

@sachaos You know what, if you have something already, why not just PR it to dyff here and we aim for a new release with those features you have. Ignore my previous comment. I just have some experiments laying around.

[HeavyWombat]

Yes, that was a bit of an oversight, when we did the --filter implementation. It is definitely on my list of things to be fixed soon. It is similar to #161.

[jrauschenbusch]

Exclusion of paths would also be very nice (e.g. filtering of server-side generated metadata by k8s controllers).

I also tried to use the filter as it is currently implemented using two files generated by kustomize, but it seems that this does not work correctly. There are multiple documents in a single file. Doing the dyff results in 1 diff. By using the reported path as filter no diff is reported.

[HeavyWombat]

Exclusion of paths would also be very nice (e.g. filtering of server-side generated metadata by k8s controllers).

I also tried to use the filter as it is currently implemented using two files generated by kustomize, but it seems that this does not work correctly. There are multiple documents in a single file. Doing the dyff results in 1 diff. By using the reported path as filter no diff is reported.

I was considering adding some implicit filtering in case the Kubernetes "mode" is used, i.e. generated metadata.

With regards to the issue you described at the end, any chance you have some simplified examples that you share?

[jrauschenbusch]

What you mean with "implicit filtering"? Would it also work with metadata (labels/annotations) attached by e.g. the kustomize-controller?

Here an example for the filter issue:

---
# a.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demo-app
  name: demo-app
  namespace: demo
spec:
  selector:
    matchLabels:
      app: demo-app
  template:
    spec:
      containers:
      - image: hashicorp/http-echo:0.2.1
        name: http-echo

---
# b.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demo-app
  name: demo-app
  namespace: demo
spec:
  selector:
    matchLabels:
      app: demo-app
  template:
    spec:
      containers:
      - image: hashicorp/http-echo:0.2.3
        name: http-echo

~ dyff between  --set-exit-code --omit-header a.yaml b.yaml                                                                      

spec.template.spec.containers.http-echo.image
  ± value change
    - hashicorp/http-echo:0.2.3
    + hashicorp/http-echo:0.2.1
~ echo $?
1

~ dyff between  --set-exit-code --omit-header --filter "spec.template.spec.containers.http-echo.image" a.yaml b.yaml                                                                      
~ echo $?
0

[sachaos]

@HeavyWombat Hi, I have same issue.
I would like to add --exclude option and filter by regular expression too.
If you don't mind, I would like to handle this issue.

[sachaos]

Thank you for your response! I understand.
Then I'll fork and write the code for my purpose, without thinking about merging it into upstream.

[sachaos]

@HeavyWombat Sorry, I was under the mistaken impression that you had already addressed this issue quite well and were in the process of fixing it from a more fundamental point of view.

I have submitted a PR.
Please review it as I would like to try to merge it.
#214

[HeavyWombat]

Thanks for the PR. Will have a look over the weekend. I really appreciate the feedback and support here.