Revisit rootless podman container problems

[some-natalie]

Revisit rootless podman container problems

[some-natalie]

Hey @aarongreen85, moving discussion of this comment here, copy/pasted below.

@some-natalie we're working on a similar case and running into the same issue you described in #93. Can you shed some light on how this PR is related to that issue? Any info you have on this that you'd be willing to share we'd appreciate. :)

I don't have the bandwidth to tackle this at the moment, which is why I brain-dumped a bunch of links/logs/etc. in this issue. I'd thought #104 was related to this, but it appears not to be. If you have anything to share here, pull requests are always welcome and it'd definitely benefit the entire OpenShift community. ❤️

[shakerg]

I'll have to retest my runner, but I had podman runners working after a little tweaking Dockerfile.

[some-natalie]

Mind putting in a pull request on that tweak to kick off tests and share? ❤️

[shakerg]

it's included in my PR already :)

[shakerg]

Would you like me to create a new PR of just that Dockerfile?

[some-natalie]

🤦🏻‍♀️ thank you @shakerg - you're still the best and I'm absentminded

[some-natalie]

New error messages around how the runner agent is building the image and calling it, versus what podman wants.

Logs (link)

Run ./tests/container
Building docker image
  Dockerfile for action: '/runner/_work/kubernoodles/kubernoodles/./tests/container/Dockerfile'.
  /usr/bin/docker build -t 60e[2](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:3)26:5f9714a018e65ed72405d6d56d722dc4 -f "/runner/_work/kubernoodles/kubernoodles/./tests/container/Dockerfile" "/runner/_work/kubernoodles/kubernoodles/tests/container"
  STEP 1/5: FROM python:[3](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:4)-slim
  Resolved "python" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
  Trying to pull docker.io/library/python:3-slim...
  Getting image source signatures
  Copying blob sha256:126aaa6587b8608b62fa39019a891e98[4](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:5)60e23bad2d04fb23[5](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:6)c4dc8fae2adf3c
  Copying blob sha25[6](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:7):[7](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:8)e27c1167cc2a09e30e53c0fd2933e1[8](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:9)[9](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:10)e20b4785284e84a919fe5fd3013388b
  Copying blob sha256:0dd06b55ca686f29b26c8a7e051ce98ad571fd6cf22bba07f5cd17e98fd31e13
  Copying blob sha256:3950cece0b727ab257c49317871d00f22d2abcb77bcce19ddf86aab5c4d88e21
  Copying blob sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7
  Copying blob sha256:126aaa6587b8608b62fa39019a891e98460e23bad2d04fb235c4dc8fae2adf3c
  Copying blob sha256:7e27c1167cc2a09e30e53c0fd2933e189e20b4785284e84a919fe5fd3013388b
  Copying blob sha256:0dd06b55ca686f29b26c8a7e051ce98ad571fd6cf22bba07f5cd17e98fd31e13
  Copying blob sha256:3950cece0b727ab257c49317871d00f22d2abcb77bcce19ddf86aab5c4d88e21
  Copying blob sha256:1efc276f4ff952c055dea726cfc96ec6a4fdb8b62d9eed816bd2b788f2860ad7
  Copying config sha256:f967afcd009f2ad139[10](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:11)7ab4c75e4a3dcc7fb53c78f676ebcca1b46e2e9e878a
  Writing manifest to image destination
  Storing signatures
  STEP 2/5: COPY test.py /app/test.py
  --> 34bf5eb5aa2
  STEP 3/5: WORKDIR /app
  --> 720443492b5
  STEP 4/5: ENV PYTHONPATH /app
  --> deec467c6d6
  STEP 5/5: CMD ["/app/test.py"]
  COMMIT 60e226:5f97[14](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:15)a018e65ed72405d6d56d722dc4
  --> a51dff8940d
  Successfully tagged localhost/60e226:5f9714a018e65ed72405d6d56d722dc4
  a51dff8940dbcd08[16](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:17)9336ab4fb104681819acfbd30fddb[17](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:18)ea88ce50ae38e85
/usr/bin/docker run --name e2265f9714a0[18](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:19)e65ed72405d6d56d722dc4_792497 --label 60e226 --workdir /github/workspace --rm -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e GITHUB_STEP_SUMMARY -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/runner/_work/_temp/_github_home":"/github/home" -v "/runner/_work/_temp/_github_workflow":"/github/workflow" -v "/runner/_work/_temp/_runner_file_commands":"/github/file_commands" -v "/runner/_work/kubernoodles/kubernoodles":"/github/workspace" 60e226:5f9714a018e65ed72405d6d56d722dc4
Resolving "60e226" using unqualified-search registries (/etc/containers/registries.conf)
Trying to pull localhost/60e226:5f9714a018e65ed72405d6d56d722dc4...
time="[20](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:21)22-08-05T[21](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:22):43:21Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://localhost/60e[22](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:23)6:5f9714a018e65ed72405d6d56d722dc4: pinging container registry localhost: Get \"https://localhost/v2/\": dial tcp [::1]:443: connect: connection refused"
time="2022-08-05T21:43:22Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://localhost/60e226:5f9714a018e65ed72405d6d56d722dc4: pinging container registry localhost: Get \"https://localhost/v2/\": dial tcp [::1]:443: connect: connection refused"
time="2022-08-05T21:43:[23](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:24)Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://localhost/60e226:5f9714a018e65ed7[24](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:25)05d6d56d722dc4: pinging container registry localhost: Get \"https://localhost/v2/\": dial tcp [::1]:443: connect: connection refused"
Trying to pull registry.fedoraproject.org/60e2[26](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:27):5f9714a018e65ed72[40](https://github.com/some-natalie/kubernoodles/runs/7699330999?check_suite_focus=true#step:6:42)5d6d56d722dc4...
Trying to pull registry.access.redhat.com/60e226:5f9714a018e65ed72405d6d56d722dc4...
Trying to pull docker.io/library/60e226:5f9714a018e65ed72405d6d56d722dc4...
Trying to pull quay.io/60e226:5f9714a018e65ed72405d6d56d722dc4...

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 2 weeks.

[github-actions]

This issue was closed because it has been stalled for 2 weeks with no activity.