tomwright/dasel

Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool. Supports conversion between formats and can be used as a Go package.

dasel

Dasel (short for data-selector) allows you to query and modify data structures using selector strings.

Comparable to jq / yq, but supports JSON, YAML, TOML, XML and CSV with zero runtime dependencies.

One tool to rule them all

Say good bye to learning new tools just to work with a different data format.

Dasel uses a standard selector syntax no matter the data format. This means that once you learn how to use dasel you immediately have the ability to query/modify any of the supported data types without any additional tools or effort.

Table of contents

• Dasel
• One tool to rule them all
• Quickstart
• Completion
• Issue vs discussion
• Features
• Table of contents
• Documentation
• Playground
• Benchmarks
• Pre-Commit

Quickstart

Dasel is available on homebrew, ASDF, scoop, docker, Nix or as compiled binaries from the latest release.

brew install dasel

You can also install a development version with:

go install github.com/tomwright/dasel/v2/cmd/dasel@master

For more information see the installation documentation.

Select

echo '{"name": "Tom"}' | dasel -r json 'name'
"Tom"

See select documentation.

Convert json to yaml

echo '{"name": "Tom"}' | dasel -r json -w yaml
name: Tom

See select documentation.

Put

echo '{"name": "Tom"}' | dasel put -r json -t string -v '[email protected]' 'email'
{
  "email": "[email protected]",
  "name": "Tom"
}

See put documentation.

Delete

echo '{
  "email": "[email protected]",
  "name": "Tom"
}' | dasel delete -r json '.email'
{
  "name": "Tom"
}

See delete documentation.

Completion

If you want to use completion from the terminal you can do the following (using zsh in this example):

Add the following to ~/.zshrc and reload your terminal.

export fpath=(~/zsh/site-functions $fpath)
mkdir -p ~/zsh/site-functions
dasel completion zsh > ~/zsh/site-functions/_dasel
compinit

Pre-Commit

Add dasel hooks to .pre-commit-config.yaml file

- repo: https://github.com/TomWright/dasel
  rev: v1.25.1
  hooks:
    - id: dasel-validate

for a native execution of dasel, or use:

• dasel-validate-docker pre-commit hook for executing dasel using the official Docker images
• dasel-validate-bin pre-commit hook for executing dasel using the official binary

Issue vs Discussion

I have enabled discussions on this repository.

I am aware there may be some confusion when deciding where you should communicate when reporting issues, asking questions or raising feature requests so this section aims to help us align on that.

Please raise an issue if:

• You find a bug.
• You have a feature request and can clearly describe your request.

Please open a discussion if:

• You have a question.
• You're not sure how to achieve something with dasel.
• You have an idea but don't quite know how you would like it to work.
• You have achieved something cool with dasel and want to show it off.
• Anything else!

Features

• Query/select data from structured data files.
• Update data in structured data files.
• Create data files.
• Supports multiple data formats/types.
• Convert between data formats/types.
• Uses a standard query/selector syntax across all data formats.
• Zero runtime dependencies.
• Available on Linux, Mac and Windows.
• Available to import and use in your own projects.
• Run via Docker.
• Faster than jq/yq.
• Pre-commit hooks.

Documentation

The official dasel docs can be found at daseldocs.tomwright.me.

Playground

You can test out dasel commands using the playground.

Source code for the playground can be found at github.com/TomWright/daselplayground.

Benchmarks

In my tests dasel has been up to 3x faster than jq and 15x faster than yq.

See the benchmark directory.

Stargazers over time

dasel's Issues

Help with TOML

Hello @TomWright,

Can you give me any help with the following TOML file? I need to create it with dasel, but I am stuck how to achieve this example below

[[servers]]
host = "127.0.0.1"
port = 389
use_ssl = false
start_tls = false
ssl_skip_verify = false
bind_dn = "cn=admin,dc=grafana,dc=org"
bind_password = 'grafana'
search_filter = "(cn=%s)"
search_base_dns = ["dc=grafana,dc=org"]

[servers.attributes]
member_of = "memberOf"
email =  "email"

Thanks again!

Replacing an object's value with new object

Thanks for creating such a great tool! I like it very well. 🤙🧡

Describe the desired outcome
My question is slightly similar to #23, what I'm trying to achieve is to simply replace the value of a specific object with a new value to be given from by the stdout pipe.

Let's consider we have an scale.yaml file:

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: foo-autoscaler
  namespace: foo
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: foo-deployment
  minReplicas: 40
  maxReplicas: 40
  targetCPUUtilizationPercentage: 70

In order to save spec object changes from each pipeline run, i should have replace spec object with current values.

Attempts

I run this command:

$ dasel put string -f scale.yml -s "spec" "$(kubectl get hpa foo-autoscaler -o yaml | dasel select -p 'yaml' '.spec')"

But I'm getting this one:

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: foo-autoscaler
  namespace: foo
spec: |-
  maxReplicas: 40
  minReplicas: 40
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: foo-deployment
  targetCPUUtilizationPercentage: 70

Please notice that, I do not want the |- (Block Chomping Indicator). But I was expecting for it to set as object. I could not find any documentation about replacing the objects in README. It will be cool if you add more examples about replacing - and maybe cross replacing, i.e.: json object to yaml object.

Thanks!

Quotes in output if string is number

Given the following YAML example:

  analog:
    type: "native"

Running dasel -n -f demo.yml analog.type gives me native. No quotes. However, if type is something like this:

  analog:
    type: "4067"

Running the same command gives me "4067". Notice the quotes in output. Is this a bug or am I missing something here?

Install dasel with scoop command-line software installer

I think dasel is interesting enough tool to be included in Scoop bucket (repository) and should be installable with this popular package manager. I already initiated PR to scoop extras and hopefully soon this going to be available.

PR is in progress here: ScoopInstaller/Extras#5220

Are you interested in an update on installation instructions once it's merged?

Assume first argument is selector if no flag is given

Is your feature request related to a problem? Please describe.
It's annoying to have to specify the -s flag in every command.

Describe the solution you'd like
If no -s flag is given, assume that the first argument is the selector.

This means that the following are equivalent:

echo '{"name": "Tom"}' | dasel select -p json -s .name
Tom

echo '{"name": "Tom"}' | dasel select -p json .name
Tom

The same should work with put commands...

echo '{"name": "Tom"}' | dasel put string -p json -s .name Frank
{"name": "Frank"}

echo '{"name": "Tom"}' | dasel select -p json .name Frank
{"name": "Frank"}

Additional context
This will remain backwards compatible since the -s flag is still in place and will take priority.

compact Output mode?

I was asking me if this is already possible but I didn't found an answer yet,

jq has a -c option which means: -c compact instead of pretty-printed output;

is that possible to add to dasel or has dasel such an option already?

Put string was working under 1.0.4, not under 1.5.1

Hello @TomWright ,

I do have this yaml:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    release: mgmt
  name: harbor-exporter
  namespace: registry
spec:
  endpoints:
  - honorLabels: true
    interval: 10s
    path: /metrics
    port: http
    scheme: http
    scrapeTimeout: 10s
  namespaceSelector:
    matchNames:
    - harbor
  selector:
    matchLabels:
      app: harbor-exporter

To change '- harbor' under matchNames I use(d):

dasel put string -f $file '.spec.namespaceSelector.matchNames[0]' 'registry'

Which gave me the correct result under 1.0.4. Under 1.5.1 the result is like this:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    release: mgmt
  name: harbor-exporter
  namespace: registry
spec:
  endpoints:
  - honorLabels: true
    interval: 10s
    path: /metrics
    port: http
    scheme: http
    scrapeTimeout: 10s
  namespaceSelector:
    matchNames:
    - harbor
    matchNames[0]: registry
  selector:
    matchLabels:
      app: harbor-exporter

Could you please take a look at this? Thanks.

dasel formats the original file in a different format

Version

Current master

Issue

I have a Cargo.toml file with the following contents:

[profile.dev]
  debug = true
  opt-level = 0

[profile.release]
  debug = false
  opt-level = 3

Whenever I use dasel to update a value inside the file, it changes the formatting of the file to a different style, in this case the above becomes:

[profile]

  [profile.dev]
    debug = true
    opt-level = 0

  [profile.release]
    debug = false
    opt-level = 3

It also adds a lot of whitespace between sections and and the top of the file

Ideally I would expect the file not to be automatically re-formatted in a different style as long as it's a valid TOML/JSON/YAML format.

Multiple file flags

Is your feature request related to a problem? Please describe.
Sometimes people need to run the same operation on multiple files.
For example: #23

Describe the solution you'd like
Allow -f, --file and -o, --out to be passed multiple times.

Dasel should then perform the operation (put or select) on all of the given files.

Describe alternatives you've considered
Comma separated file and output flags. There's no real benefit for this over accepting the flag multiple times.

Additional context
I'm not sure whether dasel should continue on error or stop processing all files if the first fails.
I'm open to feedback on this.

Add a plain flag

Is your feature request related to a problem? Please describe.
Sometimes I want to get my query results as a plain string rather than JSON encoded.

Describe the solution you'd like
A new --plain flag.

echo '{"name": "Tom"}' | dasel -p json .name
"Tom"

Becomes

echo '{"name": "Tom"}' | dasel --plain -p json .name
Tom

How do use a map value?

Hello @TomWright,

I have another question, I do have this code to create the yaml file:

echo '' | \
    dasel put string -p yaml 'apiVersion' 'cilium.io/v2' | \
    dasel put string -p yaml 'kind' 'CiliumNetworkPolicy' | \
    dasel put string -p yaml 'metadata.name' 'allow-kube-dns' | \
    dasel put string -p yaml 'spec.endpointSelector.matchLabels' {} | \
    dasel put string -p yaml 'spec.egress.[0].toEndpoints.[0].matchLabels.k8s:io\.kubernetes\.pod\.namespace' 'kube-system' | \
    dasel put string -p yaml 'spec.egress.[0].toEndpoints.[0].matchLabels.k8s-app' 'kube-dns' | \
    dasel put string -p yaml 'spec.egress.[0].toPorts.[0].ports.[0].port' '53' | \
    dasel put string -p yaml 'spec.egress.[0].toPorts.[0].ports.[0].protocol' 'UDP' 

This is the output:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-kube-dns
spec:
  egress:
  - toEndpoints:
    - matchLabels:
        k8s-app: kube-dns
        k8s:io.kubernetes.pod.namespace: kube-system
    toPorts:
    - ports:
      - port: "53"
        protocol: UDP
  endpointSelector:
    matchLabels: '{}'

But when applying this there is an error:

error: error validating "tt.yaml": error validating data: ValidationError(CiliumNetworkPolicy.spec.endpointSelector.matchLabels): invalid type for io.cilium.v2.CiliumNetworkPolicy.spec.endpointSelector.matchLabels: got "string", expected "map"; if you choose to ignore these errors, turn validation off with --validate=false

When I remove the single quotes arond the {} at matchLabels, it does work. How can I achieve this ?

  endpointSelector:
    matchLabels: {}

The docs state string, bool or int, but no other options.

Thanks again

CSV writer can't output a list of objects

I have this json:

[
  {
    "id": "ABS",
    "name": "Australian Bureau of Statistics"
  },
  {
    "id": "ECB",
    "name": "European Central Bank"
  },
  {
    "id": "ESTAT",
    "name": "Eurostat"
  },
  {
    "id": "ILO",
    "name": "International Labor Organization"
  }
]

Then I run the following and get:

$ cat test.json | dasel -r json -w csv                                                                                                                                                                (base) 
Error: could not write output: could not write to output file: could not get byte data for file: CSVParser.toBytes cannot handle type []interface {}
Usage:
  dasel select -f <file> -p <json,yaml> -s <selector> [flags]

Flags:
  -c, --compact           Compact the output by removing all pretty-printing where possible.
  -f, --file string       The file to query.
  -h, --help              help for select
      --length            Output the length of the selected value.
  -m, --multiple          Select multiple results.
  -n, --null              Output null instead of value not found errors.
  -p, --parser string     Shorthand for -r FORMAT -w FORMAT.
      --plain             Alias of -w plain
  -r, --read string       The parser to use when reading.
  -s, --selector string   The selector to use when querying the data structure.
  -w, --write string      The parser to use when writing.

Error: could not write output: could not write to output file: could not get byte data for file: CSVParser.toBytes cannot handle type []interface {}

I then tried some slight modifications. Running cat test.json | dasel -p json -w csv ".[*]" gets: Error: could not query node: could not find value: selector is not supported here: .[*]

Then, running cat test.json | dasel -p json -w csv ".[*]" -m outputs the following:

map[id:ABS name:Australian Bureau of Statistics]
map[id:ECB name:European Central Bank]
map[id:ESTAT name:Eurostat]
map[id:ILO name:International Labor Organization]

which is better, but it should be comma separated values.

Please let me know if I can provide any more information. Thanks for this great project!

Ability to return a null value when nothing returned

Is your feature request related to a problem? Please describe.
When using jq, if you reference a key that does not exist, jq will throw an error about not finding a value, and return the value null. This same behavior is not true of dasel; if setting a variable to the output of a dasel command that cannot be found, the variable is set to the error blurb.

Describe the solution you'd like
Either the default behavior, or the ability with a switch, to output a value that can be used for comparison, like null or none, or even an empty string, so it can be used cleanly in scripts.

Describe alternatives you've considered
I have thought about just piping everything to /dev/null, and using the return code for finding success, but that only works in the negative case; to get the value I want, I have to run dasel again, which I guess could work. Ideally, I could run once and be good.

Additional context
I am attempting to replace some code that is written as follows:

SCHEDULE=$(jq -r .[$i].schedule ${CONFIG} | sed 's/\*/\\*/g')
if [ "${SCHEDULE}" == "null" ]; then
	echo "Schedule Missing: $(jq -r .[$i].schedule ${CONFIG})"
	continue
fi

Is there any way to create a multidocument yaml?

Hello @TomWright ,

Is there a way to create a multidocument yaml from several yaml files?

Thanks again.

Search

Is your feature request related to a problem? Please describe.
There is no way to search documents when you don't know the structure.
Related issues: #39

Describe the solution you'd like
A new selector syntax for searching: (?:key=value).

This is a multi-select selector.

It will search the current + all child nodes using the key as the query.

If the key is . or value then compare the actual value of the node.

If the key is - or keyValue then compare the "key" of the node.

This should work in both select and put commands.

Describe alternatives you've considered
A new search command, but that complicates things when you want to select further values or update values.

Additional context
Example usage may look like:

<food>
  <tart>
    <apple color="yellow"/>
  </tart>
  <pie>
    <crust quality="flaky"/>
    <filling>
      <apple color="red"/>
    </filling>
  </pie>
  <apple color="green"/>
</food>

dasel -f in.xml -m '.food.(?:keyValue=apple)'
<apple color="yellow"/>
<apple color="red"/>
<apple color="green"/>

Homebrew releases

It would be good to use goreleaser for releases.

This will make it easy to add homebrew/linux package support.

Toml selector .p2p not working

Describe the bug
A clear and concise description of what the bug is.

I have a toml file which contains the following

[rpc]

abc = ""

[p2p]

persisent_peers = ""

I would like to select p2p but -p toml .p2p is not working
-p toml .rpc works. So I guess if it includes the number, then I can't use the selector?

To Reproduce
Steps to reproduce the behavior:

1. Go to https://dasel.tomwright.me/s/1faea436-e46e-49c0-bb44-6b30f9ed4a3c
2. Click on Run
3. See error

Include (sanitized) input files and commands to help along the way.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS] MacOS
• Version [e.g. 22] (dasel --version) latest

Additional context
Add any other context about the problem here.

How do I create a new yaml document

Hello @TomWright ,

Small question, but I am unable to figure it out, how do I create this yaml output?

metadata:
  annotations:
     node.longhorn.io/default-disks-config: '[ { "name":"fast",  "path":"/mnt/data-fast1", "allowScheduling":true, "tags":["fast"]}, { "name":"slow",  "path":"/mnt/data-slow1", "allowScheduling":true, "tags":["slow"]} ]'
  labels:
    node.longhorn.io/create-default-disk: config

Thanks!

Select output should be parsed into the correct format

Describe the bug
We currently have:

echo '{"name": "Tom"}' dasel -p json .
map[name:Tom]

We should have:

echo '{"name": "Tom"}' dasel -p json .
{
  "name": "Tom"
}

Expected behavior
The output from select should be marshalled back into the expected file format.

Additional context
This will be a breaking change in terms of the command line functionality, but it's a worthwhile fix considering the current age of dasel.

dasel complete jq replacement? Problem with maps

Describe the desired outcome
Nice tool, but I'm having a problem to achieve a particular output. Assuming the following input:

{   
  "Admin": {                                                                                                                    
    "ClientID": "eBZ5kp",         
    "ClientSecret": "rSiGsF"
  },                     
  "User": {       
    "ClientID": "Y1Btrp",
    "ClientSecret": "pHfeK-"
  }
}

I want to use the ClientIDs in a subsequent dasel search on another file with ClientID as the search item (with some shell glue). I wasn't able to achieve the desired results with dasel, which I could achieve fairly easy with jq:

jq -r '.[].ClientID'
eBZ5kp
Y1Btrp

Attempts
dasel -s '.' reproduced the input file, but I could not achieve the same result as with jq.

I went through the docs a couple of times, but I didn't saw anything that would result in the desired outcome. I typically would see errors like:

dasel select -f /tmp/clients.json -p json -s '.ClientID
Error: could not query node: could not find value: no value found for selector: .ClientID: map[......

and various combinations of ., [], [*] and .ClientID. Maybe I missed something obvious.

Using '---' as value gives an error

Hello,

With 1.13.3 I do a

echo '' | dasel put string -p yaml 'extraEnv.[0].name' 'KEYCLOAK_STATISTICS' |
dasel put string -p yaml 'extraEnv.[0].value' 'ALL' |
dasel put string -p yaml 'extraEnv.[1].name' 'KEYCLOAK_USER' |
dasel put string -p yaml 'extraEnv.[1].value' 'admin' |
dasel put string -p yaml 'extraEnv.[2].name' 'KEYCLOAK_PASSWORD' |
dasel put string -p yaml 'extraEnv.[2].value' '---keycloak-password---'

which gives a 'Error: bad flag syntax: ---keycloak-password---'.

Is there anyway I can start a string with '---' ?

Thanks.

CSV Support

Is your feature request related to a problem? Please describe.
I've received requests for CSV support.

Describe the solution you'd like
A new parser of csv.
This would parse a CSV file into an array of maps and allow usage of all the standard selectors.

Default to select command

Describe the solution you'd like
If no put or select command is given, default to select.

This will make the following equivalent:

echo '{"name": "Tom"}' | dasel select -p json -s .name
Tom

echo '{"name": "Tom"}' | dasel -p json -s .name
Tom

This shortens the commands and makes dasel more inline with the go-to tools for JSON/YAML (jq/yq).

Additional context
spf13/cobra#725
spf13/cobra#823

XML with entities like < get screwed up.

Describe the bug
Escaping in XML does not work properly (when the string is already there)

To Reproduce
Save file as es_systems.cfg

<?xml version="1.0"?>
<systemList>
  <system>
    <name>retropie</name>
    <fullname>RetroPie</fullname>
    <path>/home/fozz/RetroPie/retropiemenu</path>
    <extension>.rp .sh</extension>
    <command>sudo /home/fozz/RetroPie-Setup/retropie_packages.sh retropiemenu launch %ROM% &lt;/dev/tty &gt;/dev/tty</command>
    <platform/>
    <theme>retropie</theme>
  </system>
</systemList>

and run the command

cat es_systems.cfg | dasel put string -p xml a .systemList x

Expected behavior
The </dev/tty > should have been retained, it also should have not generated an invalid document (it put in < instead of <)

Desktop (please complete the following information):

• OS: Ubuntu 20.04
• Version: development - 971ac93

put string works with 1.0.4, but not with 1.4.0

Hello @TomWright ,

I have a problem with my code, it is working with 1.0.4, but not with 1.4.0.

This is my yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: harbor-exporter
  labels:
    app: harbor-exporter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: harbor-exporter
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: harbor-exporter
    spec:
      serviceAccountName: default
      restartPolicy: Always
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
      containers:
        - name: harbor-exporter
          image: "c4po/harbor-exporter:debug"
          imagePullPolicy: Always
          env:
            - name: HARBOR_URI
#            name of the Service for harbor-core
              value: http://harbor-core.harbor # change prefix to the name of your Helm release
            - name: HARBOR_USERNAME
              value: "admin"
            - name: HARBOR_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: harbor-core # change prefix to the name of your Helm release
                  key: HARBOR_ADMIN_PASSWORD

          securityContext:
            capabilities:
              drop:
                - SETPCAP
                - MKNOD
                - AUDIT_WRITE
                - CHOWN
                - NET_RAW
                - DAC_OVERRIDE
                - FOWNER
                - FSETID
                - KILL
                - SETGID
                - SETUID
                - NET_BIND_SERVICE
                - SYS_CHROOT
                - SETFCAP
            readOnlyRootFilesystem: true
          resources:
            limits:
              cpu: 400m
              memory: 256Mi
            requests:
              cpu: 100m
              memory: 64Mi
          ports:
            - containerPort: 9107
              name: http
          livenessProbe:
            httpGet:
              path: /-/healthy
              port: http
            initialDelaySeconds: 5
            timeoutSeconds: 5
            periodSeconds: 5
          readinessProbe:
            httpGet:
              path: /-/ready
              port: http
            initialDelaySeconds: 1
            timeoutSeconds: 5
            periodSeconds: 5

I am executng this:

dasel put string -f $file '.metadata.namespace' 'registry'
local_harbor_core_service_name=`kubectl get svc -n registry | grep harbor-core | awk '{print $1}'`
local_harbor_core_service_name will be harbor.harbor.core
dasel put string -f $file -s "spec.template.spec.containers.(name=harbor-exporter).env.(name=HARBOR_URI).value" "http://"$local_harbor_core_service_name

The value for HARBOR_URI is not changed anymore under 1.4.0 and the layout is not correct anymore.

This is the output with 1.0.4

kind: Deployment
metadata:
  labels:
    app: harbor-exporter
  name: harbor-exporter
  namespace: registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: harbor-exporter
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: harbor-exporter
    spec:
      containers:
      - env:
        - name: HARBOR_URI
          value: http://harbor-harbor-core
        - name: HARBOR_USERNAME
          value: admin
        - name: HARBOR_PASSWORD
          valueFrom:
            secretKeyRef:
              key: HARBOR_ADMIN_PASSWORD
              name: harbor-core
        image: c4po/harbor-exporter:debug
        imagePullPolicy: Always
        livenessProbe:
          httpGet:
            path: /-/healthy
            port: http
          initialDelaySeconds: 5
          periodSeconds: 5
          timeoutSeconds: 5
        name: harbor-exporter
        ports:
        - containerPort: 9107
          name: http
        readinessProbe:
          httpGet:
            path: /-/ready
            port: http
          initialDelaySeconds: 1
          periodSeconds: 5
          timeoutSeconds: 5
        resources:
          limits:
            cpu: 400m
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 64Mi
        securityContext:
          capabilities:
            drop:
            - SETPCAP
            - MKNOD
            - AUDIT_WRITE
            - CHOWN
            - NET_RAW
            - DAC_OVERRIDE
            - FOWNER
            - FSETID
            - KILL
            - SETGID
            - SETUID
            - NET_BIND_SERVICE
            - SYS_CHROOT
            - SETFCAP
          readOnlyRootFilesystem: true
      restartPolicy: Always
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
      serviceAccountName: default

Support hcl files

Is your feature request related to a problem? Please describe.

HCL is a format commonly used for hashicorp tools. It would be great to add direct support for them in dasel.

Describe the solution you'd like

Support HCL v2.

Describe alternatives you've considered

The alternative is not supporting HCL, which is currently the case :)

Additional context

I would like this so that I can add better support for Nomad in the Dokku PaaS.

Append to an array of objects not works as expected

Describe the bug
Append to an array of objects is not working as expected in YAML parser.

To Reproduce

1. Go to Append to an array of objects section
2. Try the given example:

echo 'users:
- name: Tom' | dasel put object -p yaml -s '.users[]' -t string name=Frank
users:
- name: Tom
- name: Frank

3. I got:

users:
- name: Tom
users[]:
  name: Frank

Expected behavior

users:
- name: Tom
- name: Frank

Desktop (please complete the following information):

• OS: macOS
• Version: v1.9.1

Additional context
We can add an unit test case for this scenario during fix phase. :)

Question on replacing value(s) single and multiple occurrences

Hello,

I was pointed out by someone that this tool was better to replace (multiple) values in a yaml file than using sed. I have tried some examples, but it is not clear to me how to replace certain values in https://raw.githubusercontent.com/c4po/harbor_exporter/master/kubernetes/harbor-exporter.yaml

I need to replace multiple occurrences in this file, however, I am only able to select 1. How do I add a line with the namespace value?

metadata:
  name: harbor-exporter

to

metadata:
  name: harbor-exporter
  namespace: test

Next to this, I also need to replace the value of "- harbor #......" to test and keep the remaining lines and this is failing with dasel put string -f he.yaml -s "spec.namespaceSelector.matchNames" "test"

  namespaceSelector:
    matchNames:
    - harbor # change to the namespace where you deployed Harbor

Thanks!

Select element by name

I'd like to select all elements named apple. Doesn't seem to be possible with the current dasel.

...
<food>
  <tart>
    <apple color="yellow"/>
  </tart>
  <pie>
    <crust quality="flaky"/>
    <filling>
      <apple color="red"/>
    </filling>
  </pie>
  <apple color="green"/>
</food>
...

In XPath it would be .//apple

https://stackoverflow.com/questions/15077462/xpath-to-select-all-elements-with-a-specified-name

Put Document

Is your feature request related to a problem? Please describe.
I would like to be able to put a document into a specific place within some data.

I can already put single variables/objects into data, but this requires you to define each property separately and cannot handle an entire document/nested set of objects.

Describe the solution you'd like
A new dasel put document command that will decode the value given to it, and put that into the data.

By default, the same parser as the original document will be used but you should be able to override this with a -d, --document-parser flag.

Additional context
Example usage:

echo 'foo: true
bar: 5
baz:
  qux: false
  quux: "yes"
  quuz: 7' | dasel put document --d json '.baz' '{
  "qux": false,
  "quux": "no",
  "quuz": 8
}'

Should output:

echo 'foo: true
bar: 5
baz:
  qux: false
  quux: "no"
  quuz: 8

Multi-select

Is your feature request related to a problem? Please describe.
Selectors only allow you to target a single item at a time.

E.g.

{
  "users": [
    {
      "name": "Tom",
      "enabled": true
    },
    {
      "name": "Joe",
      "enabled": false
    },
    {
      "name": "Anna",
      "enabled": true
    }
  ]
}

dasel select '.users.(enabled=true).name'
Tom

There is no way for me to find all items that match the condition.

Describe the solution you'd like
Add a -m, --multiple flag that tells dasel to return multiple matched items.

dasel select -m '.users(enabled=true).name'
Tom
Anna

It would also be nice to be able to target all items of a list too...

dasel select '.users[*].name'
Tom
Joe
Anna

In terms of put commands, it may look something like this:

dasel put bool -m '.users[*].enabled' false
{
  "users": [
    {
      "name": "Tom",
      "enabled": false
    },
    {
      "name": "Joe",
      "enabled": false
    },
    {
      "name": "Anna",
      "enabled": false
    }
  ]
}

Describe alternatives you've considered
Use some specific selector syntax instead of a new flag.

This works well for selecting all list items, but is harder to create a clear definition where dynamic selectors are used.

Additional context
This should eventually be extended to the put command too.

Expose this as a library

This is a great project. I found myself itching to use it in a Go project that requests various data (JSON, XML, CSV) from web sources, but was hesitant at the thought of writing temporary files and piping to dasels stdin.

I know that maintaining libraries can be more involved, especially regarding with versioning, API stability etc. Nevertheless, with a cursory glance at the code, it seems the readers and writers all just take and return byte slices, so it doesn't seem like it'd be too difficult to open up. I could envision this being quite useful for others.

Of course, it's your call. Thanks again!

Use - as alias of stdin/stdout

Describe the solution you'd like
Use - as an alias for stdin/stdout in the context of the --out and --file flags.

This shortens commands and enabled dasel to use a pattern used by a lot of *nix tools.

Code working with 1.13.2, not with 1.13.3

Hello @TomWright,

The code below is working with 1.13.2, but with 1.13.3 I do get an error:

Error: could not load input: could not unmarshal data: yaml: mapping values are not allowed in this context

echo '' | \
                dasel put string -p yaml [0].'job_name' 'logging-graylog' | \
                dasel put string -p yaml [0].'scrape_interval' '15s' | \
                dasel put string -p yaml [0].'basic_auth.username' admin | \
                dasel put string -p yaml [0].'basic_auth.password' 'password' | \
                dasel put string -p yaml [0].'metrics_path' '/api/plugins/org.graylog.plugins.metrics.prometheus/metrics' | \
                dasel put string -p yaml [0].'static_configs.[0].targets.[0]' 'logging.local'

Do you know why this is failing with 1.13.3 ?

Thanks!

Dynamic sub-selectors

Describe the solution you'd like
Given the following JSON document:

{
  "users": [
    {
      "name": "Tom",
      "favouriteNumbers": [1, 2, 3]
    },
    {
      "name": "Joe",
      "favouriteNumbers": [4, 5, 6]
    }
  ]
}

it would be nice to be able to perform a query like this:

dasel '.users.(favouriteNumbers.(value=2)=2).name'
Tom

Query definition in plain English:

Give me the name of the user who has a favourite number of 2.

Additional context
The new dynamic filter is essentially defining a new rootNode from the current node, and querying against it with the given selector.

Dasel fails to parse a "null" yaml file

Describe the bug

Dasel fails to parse yaml file with null as the only content.

To Reproduce

1. echo 'null' > /tmp/null.yml
2. echo '---' > /tmp/null2.yml
3. touch /tmp/null3.yml
4. dasel -f /tmp/null.yml .
5. repeat with other files
6. All cmds fail with panic: reflect: call of reflect.Value.Interface on zero Value

Expected behavior

Dasel should not fail on valid (but empty) yaml.

Desktop (please complete the following information):

• OS: Ubuntu 20.10
• Version: v1.13.2

Additional context

yamllint, yq or ytt all don't fail on the same input files.

How do I change a subvalue ?

Hello again,

I am trying to change the value of line 74 (name: harbor-core # change prefix to the name of your Helm release) of https://raw.githubusercontent.com/c4po/harbor_exporter/master/kubernetes/harbor-exporter.yaml to something else, but

dasel put string -f $file -s "spec.template.spec.containers.(name=harbor-exporter).env.(name=HARBOR_PASSWORD).valueFrom.secretKeyRef.name.value" "test"
is not correct, is dasel capable of changing this value and if so, how?

Thanks!

Support ini files

It would be great if Dasel could handle .ini files.

Cheers!

An update command for dasel could be very useful.

An update command for dasel could be very useful.

It should download the latest stable release for the current OS and replace the existing binary.

The output may look something like this:

$ dasel update
Current version: vx.x.x
Downloading latest version
Found vx.x.x
Replacing executable
New version: vx.x.x

Steps

1. Fetch latest stable version info using github API.
2. Compare to current version
3. Download latest stable version executable
4. chmod +x new version.
5. Execute dasel --version on new executable to ensure it works.
6. Replace self with new executable.

Does anyone have any thoughts?

Originally posted by @TomWright in https://github.com/TomWright/dasel/discussions/75

dasel --version shows 'development' when built from master branch

Running dasel --version returns the following when built from GitHub:

dasel version development

Which doesn't actually say which version it is (e.g. version number or git hash)

How do I do with this kind of error

Describe the desired outcome
Using dasel to select json file, but this error occur

Error: could not load input: could not unmarshal config data: invalid character '{' after top-level value

Resources
My json file content is like this
{"topic": "test", "device_info": {"device_name": "apple"}, "message": "failed"}

YAML multi-document support

Add YAML multi-document support.

installable from ASDF-VM

This is a request to support asdf-vm

I've filed an issue here to make a repo for dasel. Maybe you can comment on that issue to get maintainership of the repo once created?

I can contribute to the creation of the plugin, but it's already very very simple, and you can copy/paste from other Github-based plugins.
asdf-community/.github#11

Option to sort keys on output

Is your feature request related to a problem? Please describe.

I'd like to sort the output by keys, like jq can:

❯ jq --help | grep sort
  -S               sort keys of objects on output;

Describe the solution you'd like

A similar flag i.e. -s to sort keys.

Delete from existing array or set it to empty array

Is it possible to remove an item from an array or set an array to an empty array?

Given a foo.toml file like so:

[company]
employees = ["bob", "kate", "john"]

I can append an item with:

dasel put string -f foo.toml 'company.employees.[]' sarah

I can change a specific item with:

dasel put string -f foo.toml 'company.employees.[1]' sarah

But how would I go about deleting an item from the array?
I tried omitting the value:

dasel put string -f foo.toml 'company.employees.[1]'

But it only sets the element to an empty string.

Also how would I set an existing array to an empty array?
I tried setting it to a new object but it creates a new object instead of an array:

dasel put object -f foo.toml 'company.employees'

And there doesn't seem to be a type of 'array' only 'object', 'string', 'int' etc.

Please release using `darwin` not `macos`

this is to enable scripts that use uname to detect the platform to know which binary to download.

Help with json

Hello @TomWright,

I hope you can help me. Can I create the below json with Dasel?

{
  "Comment": "CREATE/DELETE/UPSERT a record",
    "Changes": [{
    "Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "metrics.try.my.domain",
        "Type": "SRV",
        "TTL": 300,
        "ResourceRecords": [
          { "Value": "1 10 30901 metrics-peers.try.my.domain"}
        ]
      }
    }]
}

Thanks!

Support XML data format

Is your feature request related to a problem? Please describe.
There's no way to query XML data formats.

Describe the solution you'd like
Add a new parser type of xml.

It should be auto-detected if the input filename ends with .xml.

It should be able to deal with XML strings.

Additional context
There is an xml package available here: https://golang.org/pkg/encoding/xml/

jq-like builtin function support

Is your feature request related to a problem? Please describe.
I'm always frustrated about how to get all the keys in a map, which has several nested levels. In jq, I can use the builtin function keys[] after a pipe.

Describe the solution you'd like
I hope the selector used in dasel can also support the pipe together with some handy builtin functions.

Flag to specify output format

Is your feature request related to a problem? Please describe.
It would be nice to be able to specify a desired output format.
This will allow you to easily transform data from one format to another.

Describe the solution you'd like
A new flag -n, --out-parser to specify the parser to use on output.
This should be optional. If it is not present then use the input parser.

echo '{"name":"Tom"}' | dasel -p json -n xml .
<name>Tom</name>

Describe alternatives you've considered
-n isn't a particularly declarative shorthand flag for this use-case. I'm open to suggestions on this.

Additional context
This new flag should be available on both select and put commands.

Providing a value of - should be treated the same as passing the --plain flag.