ashcrow/filetranspiler

Creates an update Ignition json file with additions from a fake root.

filetranspiler

Creates an Ignition JSON file from a fake root.

See also fcct.

Getting filetranspiler

It's recommended to use a released version of filetranspiler. See releases page for the latest releases.

Building

Container Image

Requires podman

$ make container

Running

Source

These items are required when running outside of the container.

• Python 3 (3.6+ recommended)
• PyYAML
• python3-magic
• file-magic

./filetranspile -i ignition.json -f fakeroot

Container Image

Note: When using the container don't forget to mount the host directory that houses your ignition file and fake root in to the container!

$ podman run --rm -ti --volume `pwd`:/srv:z localhost/filetranspiler:latest -i ignition.json -f fakeroot

Example

$ tree fakeroot
fakeroot/
└── etc
    ├── hostname
    ├── hostname.link -> hostname
    ├── resolve.conf
    └── sysconfig
        └── network-scripts
            ├── ifcfg-blah
            └── ifcfg-fake

3 directories, 5 files
$ ./filetranspile --help
usage: filetranspile [-h] [-i IGNITION] -f FAKE_ROOT [-o OUTPUT] [-p]
                     [--dereference-symlinks] [--format {json,yaml}]
                     [--version]

optional arguments:
  -h, --help            show this help message and exit
  -i IGNITION, --ignition IGNITION
                        Path to ignition file to use as the base
  -f FAKE_ROOT, --fake-root FAKE_ROOT
                        Path to the fake root
  -o OUTPUT, --output OUTPUT
                        Where to output the file. If empty, will print to
                        stdout
  -p, --pretty          Make the output pretty
  --dereference-symlinks
                        Write out file contents instead of making symlinks
                        NOTE: Target files must exist in the fakeroot
  --format {json,yaml}  What format of file to write out. `yaml` or `json`
                        (default)
  --version             show program's version number and exit
$ cat ignition.json 
{
  "ignition": { "version": "2.3.0" },
  "storage": {
    "files": [{
      "path": "/var/foo/bar",
      "filesystem": "root",
      "mode": 420,
      "contents": { "source": "data:,example%20file%0A" }
    }]
  }
}

$ ./filetranspile -i test/ignition.json -f test/fakeroot -p
{
    "ignition": {
        "version": "2.3.0"
    },
    "storage": {
        "files": [
            {
                "contents": {
                    "source": "data:,example%20file%0A"
                },
                "filesystem": "root",
                "mode": 420,
                "path": "/var/foo/bar"
            },
            {
                "contents": {
                    "source": "data:text/plain;charset=us-ascii;base64,c29tZXRoaW5nCg=="
                },
                "filesystem": "root",
                "mode": 436,
                "path": "/etc/hostname"
            },
            {
                "contents": {
                    "source": "data:text/plain;charset=us-ascii;base64,c2VhcmNoIDEyNy4wLjAuMQpuYW1lc2VydmVyIDEyNy4wLjAuMQo="
                },
                "filesystem": "root",
                "mode": 436,
                "path": "/etc/resolve.conf"
            },
            {
                "contents": {
                    "source": "data:text/plain;charset=us-ascii;base64,YmxhaCBibGFoIGJsYWgKMTIzNDU2Nzg5MApibGFoIGJsYWggYmxhaAo="
                },
                "filesystem": "root",
                "mode": 436,
                "path": "/etc/sysconfig/network-scripts/ifcfg-blah"
            },
            {
                "contents": {
                    "source": "data:text/plain;charset=us-ascii;base64,ZmFrZQo="
                },
                "filesystem": "root",
                "mode": 436,
                "path": "/etc/sysconfig/network-scripts/ifcfg-fake"
            }
        ],
        "links": [
            {
                "filesystem": "root",
                "hard": false,
                "path": "/etc/hostname.link",
                "target": "hostname"
            }
        ]
    }
}

filetranspiler's Issues

ModuleNotFoundError: No module named 'magic'

Using both, python on the host or the container I get the same error: "ModuleNotFoundError: No module named 'magic'"

podman run --rm -ti --volume pwd:/srv:z localhost/filetranspiler:latest -i bootstrap.ign -f bootstrap -o bootstrap0.ign
^Cls
Traceback (most recent call last):
File "/usr/bin/filetranspile", line 10, in
import magic
ModuleNotFoundError: No module named 'magic'

Release 1.1.0

Tag and release 1.1.0

Add --dereference-symlinks

Add support for dereferencing symlinks. filetranspile should write out a copy of the referenced file instead of writing a symlink when the file exists within the root and --dereference-symlinks is passed as an argument.

/cc @dustymabe

YAML module missing from image

As of the new version, the image does not have the yaml module inside of it for the Python script to run. That would likely need to be added otherwise you get errors when running the image.

Release latest

Could you do a release with latest changes please?

Thanks in advance

Traceback while using the magic package

[root@lab filetranspiler]# ./filetranspile -i ./test/ignition.json -f fake-root
Traceback (most recent call last):
File "./filetranspile", line 367, in
main()
File "./filetranspile", line 339, in main
ignition_spec = spec_cls(ignition_cfg, args)
File "./filetranspile", line 43, in init
self.mimetype = magic.open(magic.MAGIC_MIME)
AttributeError: module 'magic' has no attribute 'open'
Exception ignored in: <bound method IgnitionSpec.del of <main.SpecV2 object at 0x7f1401e93358>>
Traceback (most recent call last):
File "./filetranspile", line 47, in del
self.mimetype.close()
AttributeError: 'SpecV2' object has no attribute 'mimetype'

filetranspile vs filetranspiler

Just a nit, but I think the python script shall be renamed to filetrapiler as the project/repo :)
Thanks!

RPM package

Just FYI, I've added the filetranspiler RPM package to my copr repo https://copr.fedorainfracloud.org/coprs/eminguez/eminguez-RPMs/package/filetranspiler/

File transpiler does not replace the file if it already exists in ignition.

If a ".storage.file" with the same path is already existed in ignition config, it does not replace it, but rather creates a new block which cause duplication error.

base64 encoded output

A command line switch to have base64 encoded utf-8 output would be perfect!
Me python skills are very limited ... i tried to modify your code but the result looks not as expected ... :(

missing "filesystem" in final ignition config

Amazing work on this!

A simple file rendered with filetranspiler will look like this:
"storage": {
"files": [
{
"contents": {
"source": "data:,search%20example.com%0Anameserver%208.8.8.8%0Anameserver%204.4.4.4%0A"
},
"mode": 420,
"path": "/etc/resov.conf"
},

my system will complain about requiring the "filesystem" directive. Inserting '"filesystem": "root",' seems to fix the issue, but I'm unsure if this should be configurable in this app, or if it should default to the "root" fs.

"storage": {
    "files": [
        {
            "contents": {
                "source": "data:,search%20example.com%0Anameserver%208.8.8.8%0Anameserver%204.4.4.4%0A"
            },
            "filesystem": "root",
            "mode": 420,
            "path": "/etc/resov.conf"
        },

Update README with dependencies

Please add in the README file that the python script requires python3-yaml to be installed ( not installed by default on Debian )

$ python3 ~/Downloads/filetranspiler-master/filetranspile -i bootstrap.ign -f bootstrap -o bootstrap-static.ign
Traceback (most recent call last):
  File "/home/psergiu/Downloads/filetranspiler-master/filetranspile", line 12, in <module>
    import yaml
ImportError: No module named 'yaml'

Thank you !

SyntaxError line 135: f'link: {source_path} is not in the '

Getting a syntax error when trying to update a OpenShift bootstrap ignition file as described in: https://blog.openshift.com/openshift-4-bare-metal-install-quickstart/

$ ls -l
total 308
drwxr-x--- 2 psergiu psergiu   4096 Feb 18 10:08 auth
drwxr-xr-x 3 psergiu psergiu   4096 Feb 17 14:25 bootstrap
-rw-r----- 1 psergiu psergiu 292125 Feb 18 10:08 bootstrap.ign
-rw-r----- 1 psergiu psergiu   1833 Feb 18 10:08 master.ign
-rw-r----- 1 psergiu psergiu    104 Feb 18 10:08 metadata.json
-rw-r----- 1 psergiu psergiu   1833 Feb 18 10:08 worker.ign

$ tree bootstrap
bootstrap
└── etc
    ├── hostname
    ├── resolv.conf
    └── sysconfig
        └── network-scripts
            └── ifcfg-ens192

3 directories, 3 files

$ python3 ~/Downloads/filetranspiler-master/filetranspile -i bootstrap.ign -f bootstrap -o bootstrap-static.ign
  File "/home/psergiu/Downloads/filetranspiler-master/filetranspile", line 135
    f'link: {source_path} is not in the '
                                        ^
SyntaxError: invalid syntax

$ python2 ~/Downloads/filetranspiler-master/filetranspile -i bootstrap.ign -f bootstrap -o bootstrap-static.ign
  File "/home/psergiu/Downloads/filetranspiler-master/filetranspile", line 135
    f'link: {source_path} is not in the '
                                        ^
SyntaxError: invalid syntax