Usage?

[slimtoolkit]

Usage?

[kcq]

Thank you for the report and thank you for the sample Dockerfile!

Where exactly did you get stuck? Did the 'docker-slim build' command fail? What do you see when you run 'docker images'?

I tried to repro your use case and here's what I got (using the Dockerfile you provided):

$ docker build -t q/ds-issue-44 .

<note: this command created the initial Docker image>

$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
q/ds-issue-44 latest b33a7a958577 2 minutes ago 257MB

Now it's time to slim the image (note: skipped most of the command output)...

$ docker-slim build --show-clogs=true q/ds-issue-44
...
docker-slim: press when you are done using the container...
...
Successfully tagged q/ds-issue-44.slim:latest
INFO[0007] docker-slim: created new image: q/ds-issue-44.slim ( has data artifacts: true )

$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
q/ds-issue-44.slim latest 30219672f4b9 2 minutes ago 1.39MB
q/ds-issue-44 latest b33a7a958577 6 minutes ago 257MB

The output shows that docker-slim created a minified version of the original alpine image (that was 257MB) and this minified version is 1.39MB.

Now let's run the minified image:

$ docker run --rm -it q/ds-issue-44.slim
hello world

P.S.
Don't forget to use the latest release :)

[gcstang]

I'm not sure how your image was 257MB(maybe you're building something else) mine is only 4.15MB after creation but then once running docker slim it becomes 0MB.

`
(Same Dockerfile as above)
$ docker build -t test1 .
test1 latest 132df7fc1b65 4 seconds ago 4.15MB
$ ./docker-slim build --show-clogs=true test1

docker-slim: [build] image=test1 http-probe=false remove-file-artifacts=false image-overrides=map[] entrypoint=[] (false) cmd=[] (false) workdir='' env=[] expose=map[]
INFO[0000] docker-slim: inspecting 'fat' image metadata...
INFO[0000] docker-slim: [sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4] 'fat' image size => 4147781 (4.1 MB)
INFO[0000] docker-slim: processing 'fat' image info...
INFO[0000] docker-slim: starting instrumented 'fat' container...
INFO[0000] docker-slim: created container => b0a78b1da9e127206f49a2fde26a170a3ab0a995e29cf0b3248cbb2c943411a5
INFO[0000] docker-slim: watching container monitor...
docker-slim: press when you are done using the container...

<!!! I PUSHED ENTER HERE !!!>

INFO[0003] docker-slim: waiting for the container to finish its work...
INFO[0003] docker-slim: shutting down 'fat' container...
docker-slim: container stdout:
hello world
docker-slim: container stderr:
time="2018-02-03T21:12:56Z" level=info msg="sensor: args => []string{"/opt/dockerslim/bin/sensor"}"
time="2018-02-03T21:12:56Z" level=info msg="sensor: creating event publisher..."
time="2018-02-03T21:12:56Z" level=info msg="sensor: creating cmd server..."
time="2018-02-03T21:12:56Z" level=info msg="sensor: waiting for commands..."
time="2018-02-03T21:12:56Z" level=info msg="sensor: monitor starting..."
time="2018-02-03T21:12:56Z" level=info msg="fanmon: starting..."
time="2018-02-03T21:12:56Z" level=info msg="ptmon: starting..."
time="2018-02-03T21:12:56Z" level=info msg="ptmon: monitor is exiting... status= 0"
time="2018-02-03T21:12:56Z" level=info msg="ptmon: done =>0"
time="2018-02-03T21:12:59Z" level=info msg="sensor: waiting for monitor to finish..."
time="2018-02-03T21:12:59Z" level=info msg="fanmon: stopping..."
time="2018-02-03T21:12:59Z" level=info msg="sensor: published 'monitor.finish.completed'"
time="2018-02-03T21:12:59Z" level=info msg="sensor: done!"
docker-slim: end of container logs =============
INFO[0004] docker-slim: processing instrumented 'fat' container info...
INFO[0004] docker-slim: generating AppArmor profile...
INFO[0004] docker-slim: building 'slim' image...
INFO[0004] docker-slim: WARNING - no data artifacts
Step 1/3 : FROM scratch
--->
Step 2/3 : ENV PATH "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
---> Running in b7b2e8d18d4d
Removing intermediate container b7b2e8d18d4d
---> 685b97ad18ff
Step 3/3 : ENTRYPOINT ["echo","hello world"]
---> Running in 21631469e872
Removing intermediate container 21631469e872
---> 7c5ba6c117dc
Successfully built 7c5ba6c117dc
Successfully tagged test1.slim:latest
INFO[0004] docker-slim: created new image: test1.slim ( has data artifacts: false )

$ docker images
test1.slim latest 7c5ba6c117dc About a minute ago 0B

$ docker run --rm -it test1.slim
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: "echo": executable file not found in $PATH": unknown.
`

[kcq]

The alpine image was that big because i used the Go alpine image instead of the vanilla alpine image :-) I'll try it with a vanilla alpine too.

The "WARNING - no data artifacts" message means that something happened with the target container during the inspection phase it didn't produce any artifacts.

Are you running it on Linux or Mac OS X?

[kcq]

Tried it with the latest release (1.19) and a vanilla alpine image:

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
q/ds-issue-44.slim latest 6c9d520739cb 12 seconds ago 1.39MB
q/ds-issue-44 latest a20b34c608dc 48 seconds ago 3.99MB

$ docker run --rm -it q/ds-issue-44.slim
hello world

[gcstang]

running on a Mac

…

On Feb 4, 2018 12:45, "Kyle Quest" ***@***.***> wrote: The alpine image was that big because i used the Go alpine image instead of the vanilla alpine image :-) I'll try it with a vanilla alpine too. The "WARNING - no data artifacts" message means that something happened with the target container during the inspection phase it didn't produce any artifacts. Are you running it on Linux or Mac OS X? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#44 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADBkKapoXBXuTgFMNNPvBgJ9D2K7_Vwnks5tRfrEgaJpZM4R3U_8> .

[kcq]

Very interesting :) Sorry for the headaches! What's the Docker version you are using?

Can you also try running docker-slim with more debugging information like this:
./docker-slim --debug build --show-clogs=true test1

[gcstang]

No problem.

Mac Edge version 18.02.0-ce-rc2-mac51 (22446)

`./docker-slim --debug build --show-clogs=true test1
docker-slim: [build] image=test1 http-probe=false remove-file-artifacts=false image-overrides=map[] entrypoint=[] (false) cmd=[] (false) workdir='' env=[] expose=map[]
DEBU[0000] docker-slim: new Docker client (default) [6]
DEBU[0000] docker-slim: configured DOCKER_HOST env var
INFO[0000] docker-slim: inspecting 'fat' image metadata...
INFO[0000] docker-slim: [sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4] 'fat' image size => 4147781 (4.1 MB)
INFO[0000] docker-slim: processing 'fat' image info...
DEBU[0000]

IMAGE HISTORY =>
[]docker.ImageHistory{docker.ImageHistory{ID:"sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4", Tags:[]string{"test1:latest"}, Created:1517692328, CreatedBy:"/bin/sh -c #(nop) ENTRYPOINT ["echo" "hello world"]", Size:0, Comment:""}, docker.ImageHistory{ID:"sha256:3fd9065eaf02feaf94d68376da52541925650b81698c53c6824d92ff63f98353", Tags:[]string{"alpine:latest"}, Created:1515532258, CreatedBy:"/bin/sh -c #(nop) CMD ["/bin/sh"]", Size:0, Comment:""}, docker.ImageHistory{ID:"", Tags:[]string(nil), Created:1515532258, CreatedBy:"/bin/sh -c #(nop) ADD file:093f0723fa46f6cdbd6f7bd146448bb70ecce54254c35701feeceb956414622f in / ", Size:4147781, Comment:""}}

DEBU[0000] IMAGE INSTRUCTIONS:
DEBU[0000] # new image
DEBU[0000] ADD file:093f0723fa46f6cdbd6f7bd146448bb70ecce54254c35701feeceb956414622f in /
DEBU[0000] CMD ["/bin/sh"]
DEBU[0000] # end of image: alpine (id: sha256:3fd9065eaf02feaf94d68376da52541925650b81698c53c6824d92ff63f98353 tags: latest)
DEBU[0000]
DEBU[0000] # new image
DEBU[0000] ENTRYPOINT ["echo" "hello world"]
DEBU[0000] # end of image: test1 (id: sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4 tags: latest)
DEBU[0000]
INFO[0000] docker-slim: starting instrumented 'fat' container...
INFO[0000] docker-slim: created container => ce7c212fbd798465c998b2da3285b2f666a25bdd4366dd77298a1ddba11ea97b
DEBU[0000] container NetworkSettings.Ports => map[docker.Port][]docker.PortBinding{"65502/tcp":[]docker.PortBinding{docker.PortBinding{HostIP:"0.0.0.0", HostPort:"32768"}}, "65501/tcp":[]docker.PortBinding{docker.PortBinding{HostIP:"0.0.0.0", HostPort:"32769"}}}
DEBU[0000] cmdChannelAddr=tcp://127.0.0.1:32769 evtChannelAddr=tcp://127.0.0.1:32768
DEBU[0000] sendCmd(&{echo [hello world] [] []})
INFO[0000] docker-slim: watching container monitor...
docker-slim: press when you are done using the container...

DEBU[0014] sendCmd(&{})
DEBU[0014] 'stop' response => 'ok'
INFO[0014] docker-slim: waiting for the container to finish its work...
DEBU[0014] getEvt()
DEBU[0014] docker-slim: sensor event => 'monitor.finish.completed'
INFO[0014] docker-slim: shutting down 'fat' container...
DEBU[0014] docker-slim: getting container logs => ce7c212fbd798465c998b2da3285b2f666a25bdd4366dd77298a1ddba11ea97b
docker-slim: container stdout:
hello world
docker-slim: container stderr:
time="2018-02-07T18:09:22Z" level=info msg="sensor: args => []string{"/opt/dockerslim/bin/sensor", "-d"}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cwd => "/""
time="2018-02-07T18:09:22Z" level=debug msg="sensor: setting up channels..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: creating event publisher..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: creating cmd server..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: waiting for commands..."
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cmd server - got a command => {"name":"cmd.monitor.start","data":{"app_name":"echo","app_args":["hello world"]}}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:22Z" level=debug msg="
sensor: command => &{echo [hello world] [] []}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: 'start' command (&messages.StartMonitor{AppName:"echo", AppArgs:[]string{"hello world"}, Excludes:[]string(nil), Includes:[]string(nil)}) - starting monitor..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: monitor starting..."
time="2018-02-07T18:09:22Z" level=info msg="fanmon: starting..."
time="2018-02-07T18:09:22Z" level=info msg="ptmon: starting..."
time="2018-02-07T18:09:22Z" level=debug msg="sensor: target app started => echo []string{"hello world"}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: monitor - waiting to stop monitoring..."
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: fanRunMonitor worker starting"
time="2018-02-07T18:09:22Z" level=debug msg="sensor.startTargetApp(echo,[hello world],/)"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: fanRunMonitor worker (monitor) starting"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>20"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file open"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/bin/busybox"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file read"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/bin/busybox"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 2"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file read"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/bin/busybox"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>20"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file open"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/lib/ld-musl-x86_64.so.1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 3"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 4"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file read"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/lib/ld-musl-x86_64.so.1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 5"
time="2018-02-07T18:09:22Z" level=debug msg="sensor.startTargetApp: started target app --> PID=15"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: target PID ==> 15"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: initial process status => 1407"
time="2018-02-07T18:09:22Z" level=info msg="ptmon: monitor is exiting... status= 0"
time="2018-02-07T18:09:22Z" level=info msg="ptmon: done =>0"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: executed syscall count = 7"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: number of syscalls: 6"
time="2018-02-07T18:09:22Z" level=debug msg="[102] getuid = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[1] write = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[59] execve = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[158] arch_prctl = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[218] set_tid_address = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[10] mprotect = 2"
time="2018-02-07T18:09:25Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:25Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:28Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:28Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:30Z" level=debug msg=.
time="2018-02-07T18:09:31Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:31Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:34Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:34Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:35Z" level=debug msg=.
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - got a command => {"name":"cmd.monitor.stop"}"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:36Z" level=debug msg="
sensor: command => &{}"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: 'stop' command - stopping monitor..."
time="2018-02-07T18:09:36Z" level=info msg="sensor: waiting for monitor to finish..."
time="2018-02-07T18:09:36Z" level=debug msg="sensor: monitor - stop message..."
time="2018-02-07T18:09:36Z" level=debug msg="sensor: monitor - processing data..."
time="2018-02-07T18:09:36Z" level=info msg="fanmon: stopping..."
time="2018-02-07T18:09:36Z" level=debug msg="fanmon: sending report (processed 5 events)..."
time="2018-02-07T18:09:36Z" level=debug msg="findSymlinks - deviceId=103"
time="2018-02-07T18:09:36Z" level=debug msg="includePaths: map[]"
time="2018-02-07T18:09:36Z" level=debug msg="excludePaths: map[]"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: monitor - saving report to /opt/dockerslim/artifacts/creport.json"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: trying to publish 'monitor.finish.completed' event (attempt 1)"
time="2018-02-07T18:09:36Z" level=info msg="sensor: published 'monitor.finish.completed'"
time="2018-02-07T18:09:36Z" level=info msg="sensor: done!"
time="2018-02-07T18:09:36Z" level=debug msg="defered cleanup on shutdown..."
time="2018-02-07T18:09:36Z" level=debug msg="cleanupOnShutdown()"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - error => connection closed"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - done..."
docker-slim: end of container logs =============
INFO[0015] docker-slim: processing instrumented 'fat' container info...
INFO[0015] docker-slim: generating AppArmor profile...
DEBU[0015] docker-slim: saving seccomp profile to /Users/gcollins/Desktop/dockerslim/.images/132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4/artifacts/test1-seccomp.json
INFO[0015] docker-slim: building 'slim' image...
INFO[0015] docker-slim: WARNING - no data artifacts
Step 1/3 : FROM scratch
--->
Step 2/3 : ENV PATH "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
---> Using cache
---> 685b97ad18ff
Step 3/3 : ENTRYPOINT ["echo","hello world"]
---> Using cache
---> 7c5ba6c117dc
Successfully built 7c5ba6c117dc
Successfully tagged test1.slim:latest
INFO[0015] docker-slim: created new image: test1.slim ( has data artifacts: false )`

[gcstang]

@kcq any updates?

[kcq]

Thank you for the debug output! It's definitely helpful! Can you list the files and directories here: /Users/gcollins/Desktop/dockerslim/.images/132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4/artifacts . Is there anything there? Is there a directory called 'files'?

There should be a message saying that the master app created that 'artifacts' directory, but I don't see it in your debug messages. I also don't see any debug messages from the sensor app showing that it tried to prepare and copy any artifacts, which indicates that the sensor didn't even try to copy the artifacts.

This is a good lead. Thank you! We'll probably need an update release with more debugging information around directory and file I/O operations to surface permission related problems.

[gcstang]

Np, yw

These are the files dockerslim/.images/132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4/artifacts/
(There is directory called files)
Dockerfile Dockerfile.fat creport.json test1-apparmor-profile test1-seccomp.json

Latest Docker Version
docker-ce Version 18.02.0-ce-mac53 (22617)

[kcq]

Thanks for confirming that there's no 'files' directory there! There'll be a new version to try tomorrow. I greatly appreciate your help tracking this down!

[kcq]

@gcstang Would you be able to try the latest release ( https://github.com/docker-slim/docker-slim/releases/tag/1.24 ) to see if you still see the problem you experienced or if it's gone now? Thanks again for your report!

[gcstang]

This worked!
Thank you

[kcq]

Great to hear!