inti-cmnb / kicad_auto Goto Github PK
View Code? Open in Web Editor NEWDocker image for KiCad automation scripts suitable for CI/CD
License: Apache License 2.0
Docker image for KiCad automation scripts suitable for CI/CD
License: Apache License 2.0
I am working on setting kibot up under windows.
The 'kibot" script is provided further below.
I installed VcXsrv . When in interactive mode, I can run kicost
and I get the kicost
UI from the docker VM, but when running kicad
as a non-root user, I get a Segmentation fault
. When setting user and group to root
, I get the kicad UI.
I tried both dev and the release mentioned in the script.
I run make KIBOT=./kibot
where kibot is in the current directory.
I tried the kibot_variants_arduprog
demo project.
I always end up with an interactive shell, which may be my script (I am working on it).
The core issue is the Segmentation fault for a non-root user in the Docker VM.
Script (with root as user, not the current user name):
#!/bin/bash
#IMAGE=setsoft/kicad_auto:10.3-5.1.5
#IMAGE=setsoft/kicad_auto:dev
IMAGE=setsoft/kicad_auto:latest
export USER_ID=$(id -u)
export GROUP_ID=$(id -g)
# Using root on windows
USER_ID=0
group_ID=0
export WORKDIR=$(realpath "$(dirname $0)/..")
export SUBDIR=$(basename $0)
export WORKDIR=$(realpath "$(dirname $0)")
export SUBDIR=.
export DISPLAY=${DISPLAY:=0}
export HOMEDIR=$(realpath "$(dirname $0)/..")/kibot
if [ ${USER_ID} != 0 ] ; then
export TGTHOMEDIR="/home/${USER}"
else
export TGTHOMEDIR="/root"
fi
# Quote arguments
ARGS=""
for i in "$@" ; do
ARGS=$ARGS" '"${i/\'/\\\'}"'"
done
#ARGS="$@"
# In case of windows batch
# for /f "delims=[] tokens=2" %%a in ('ping -4 -n 1 %ComputerName% ^| findstr [') do set NetworkIP=%%a
# echo Network IP: %NetworkIP
IP=$(ping -4 -n 1 ${COMPUTERNAME}|head -2|tail -1|perl -p -e 's/^.*?\[(.*?)\].*/$1/;')
DISPLAY=$IP:0.0
mkdir -p $HOMEDIR
WORKDIR=$(cygpath -w "$WORKDIR")
HOMEDIR=$(cygpath -w "$HOMEDIR")
#
# passwd & shadow not propagated on windows, using root user
# --user $USER_ID:$GROUP_ID \
#ETC=$(cygpath -w "/etc")
#ETC=/etc
# --volume="${ETC}/passwd:/etc/passwd:ro" \
# --volume="${ETC}/shadow:/etc/shadow:ro" \
# --volume="${ETC}/group:/etc/group:ro" \
#
#echo $ARGS
docker run --rm -it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY \
-v $WORKDIR:${TGTHOMEDIR}/workdir \
--env NO_AT_BRIDGE=1 \
--workdir="${TGTHOMEDIR}" \
--volume="${HOMEDIR}/.config/kicad:${TGTHOMEDIR}/.config/kicad:rw" \
--volume="${HOMEDIR}/.cache/kicad:${TGTHOMEDIR}/.cache/kicad:rw" \
$IMAGE /bin/bash -c "cd workdir/$SUBDIR ; $ARGS"
If the above script is called './kicad_auto' then you can launch kicad using:
./kicad_auto kicad
It's not that complex to adapt that script to a windows bat script (quoting the arguments seems to be the hardest part).
I am getting this error, any ideas?
The CI code is here
2022-03-25T23:57:34.8950947Z WARNING:(W037) Field Datasheet of component J9 (USB_A Bottom Camera) contains extra spaces: ` ~` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8952180Z WARNING:(W037) Field Datasheet of component J10 (USB_A AUX USB) contains extra spaces: ` ~` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8953192Z WARNING:(W037) Field Datasheet of component J11 (USB_A Top Camera) contains extra spaces: ` ~` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8959092Z WARNING:(W037) Field JLCPCB of component R20 (R_Small 5.1k) contains extra spaces: ` C27834` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8960153Z WARNING:(W037) Field JLCPCB of component R22 (R_Small 5.1k) contains extra spaces: ` C27834` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8961087Z WARNING:(W037) Field JLCPCB of component R28 (R_Small 120) contains extra spaces: ` C17437` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8962600Z WARNING:(W037) Field LCSC of component R28 (R_Small 120) contains extra spaces: `C131390 ` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8964586Z WARNING:(W037) Field Mouser of component U14 (MAX3078E) contains extra spaces: `700-MAX3078EESA ` removing them. (kibot - v5_sch.py:1029)
2022-03-25T23:57:34.8971425Z - Running the ERC
2022-03-25T23:57:46.5985840Z WARNING:(W058) Missing KiCad main config file /github/home/.config/kicad/6.0/kicad_common.json (eeschema_do - misc.py:177) (kibot - kiplot.py:152)
2022-03-25T23:57:46.5986505Z WARNING:(W058) Eeschema failed to start retrying ... (eeschema_do - eeschema_do:628) (kibot - kiplot.py:152)
2022-03-25T23:57:46.5991560Z - Updating BoM in XML format
2022-03-25T23:58:04.0717002Z - 'Interactive Bill of Materials' (ibom) [ibom]
2022-03-25T23:58:04.0757152Z WARNING:(W013) `G***` component in board, but not in schematic (kibot - kiplot.py:288)
2022-03-25T23:58:04.0786186Z WARNING:(W013) `svg2mod` component in board, but not in schematic (kibot - kiplot.py:288)
2022-03-25T23:58:04.0786772Z WARNING:(W013) `kibuzzard-61025CA6` component in board, but not in schematic (kibot - kiplot.py:288)
2022-03-25T23:58:04.0787300Z WARNING:(W013) `kibuzzard-61046E92` component in board, but not in schematic (kibot - kiplot.py:288)
2022-03-25T23:58:04.0787840Z WARNING:(W013) `kibuzzard-61046EDB` component in board, but not in schematic (kibot - kiplot.py:288)
2022-03-25T23:58:06.1998185Z - 'Bill of Materials' (bom) [bom]
2022-03-25T23:58:06.2585247Z - 'HTML BOM' (html_bom) [bom]
2022-03-25T23:58:06.3132118Z - 'Print schematic (PDF)' (print_sch) [pdf_sch_print]
2022-03-25T23:58:16.5660130Z ERROR:Eeschema created an error dialog, expect a fail (eeschema_do - eeschema_do:66) (kibot - kiplot.py:149)
2022-03-25T23:58:16.5661026Z ERROR:Eeschema failed to start or reported a fatal error (eeschema_do - eeschema_do:633) (kibot - kiplot.py:149)
2022-03-25T23:58:16.5661514Z ERROR:eeschema_do returned 10 (kibot - out_pdf_sch_print.py:73)
2022-03-25T23:58:16.7098333Z ##[error]Process completed with exit code 12.
Something has changed in the last 3 days (time since last successful build using KiBot in GitHub Actions) that has broken ibom generation:
DEBUG:Doing Interactive BoM (kibot - out_ibom.py:141)
DEBUG:Running: ['generate_interactive_bom.py', '/github/workspace/gpwdev/gpwdev.kicad_pcb', '--dest-dir', '/github/workspace/output/gpwdev', '--no-browser', '--bom-view', 'left-right', '--checkboxes', 'Sourced,Placed', '--extra-data-file', '/github/workspace/gpwdev/gpwdev.xml', '--extra-fields', 'MPN,SPN', '--include-nets', '--include-tracks', '--layer-view', 'FB', '--name-format', 'ibom', '--sort-order', 'C,R,L,D,U,Y,X,F,SW,A,~,HS,CNN,J,P,NT,MH'] (kibot - out_ibom.py:176)
ERROR:Failed to create BoM, error 1 (kibot - out_ibom.py:184)
DEBUG:Output from command: Loading /github/workspace/gpwdev/gpwdev.kicad_pcb
Traceback (most recent call last):
File "/usr/bin/generate_interactive_bom.py", line 71, in <module>
ibom.main(parser, config, logger)
File "/usr/lib/python3/dist-packages/InteractiveHtmlBom/core/ibom.py", line 331, in main
pcbdata, components = parser.parse()
File "/usr/lib/python3/dist-packages/InteractiveHtmlBom/ecad/kicad.py", line 716, in parse
"fabrication": self.parse_drawings_on_layers(
File "/usr/lib/python3/dist-packages/InteractiveHtmlBom/ecad/kicad.py", line 360, in parse_drawings_on_layers
for drawing in self.parse_drawing(d[1]):
File "/usr/lib/python3/dist-packages/InteractiveHtmlBom/ecad/kicad.py", line 325, in parse_drawing
result.append(self.parse_dimension(d))
File "/usr/lib/python3/dist-packages/InteractiveHtmlBom/ecad/kicad.py", line 296, in parse_dimension
for s in d.GetShapes():
TypeError: 'SwigPyObject' object is not iterable
(kibot - out_ibom.py:186)
There have been no changes to the gpwdev.kicad_pcb
file since the last successful build.
Currently I use kicad8_auto:latest
and my GitLab pipeline fails very early because the docker image seems to be broken.
I install into the docker some specific tools and get the following result:
$ apt install something_else
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Reading package lists...
Building dependency tree...
Reading state information...
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
kidiff : Depends: kiauto but it is not installable
Recommends: scour but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
When running kicad-cli version
using ghcr.io/inti-cmnb/kicad7_auto:latest
the version reported is 7.0.8
and the version reported by setsoft/kicad_auto:nightly
is even older (<9 months as it doesn't have kicad-cli
available).
A recent commit to KiCad 7 added the ability to import Specctra SES files in headless mode through Python, which enables automated Freerouting workflows. It would be nice to have fresh images to play with.
Running the following command:
export SUBDIR=kicad-ci-test-spora/pcb_main export WORKDIR=test-repos docker run --rm -it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -v $(pwd)/$WORKDIR:/home/$USER/workdir --user $USER_ID:$GROUP_ID --env NO_AT_BRIDGE=1 --workdir="/home/$USER" --volume="/etc/group:/etc/group:ro" --volume="/home/$USER/.config/kicad:/home/$USER/.config/kicad:rw" --volume="/home/$USER/.cache/kicad:/home/$USER/.cache/kicad:rw" --volume="/etc/passwd:/etc/passwd:ro" --volume="/etc/shadow:/etc/shadow:ro" setsoft/kicad_auto:latest /bin/bash -c "cd workdir/$SUBDIR; kiplot"
Produces:
ERROR:Output from command:
../src/common/stdpbase.cpp(62): assert "traits" failed in Get(): create wxApp before calling this
ERROR:Time-out waiting for eeschema, giving up (eeschema_do - eeschema_do:156)
(kibot.kibot.kiplot - kiplot.py:146)
ERROR:eeschema_do returned 10 (kibot.kibot.out_pdf_sch_print - out_pdf_sch_print.py:56)
I pulled the kicad_auto image and I am running https://github.com/INTI-CMNB/kicad-ci-test-spora
Environment: Debian 10 with XFCE4 desktop on Digital Ocean
We're holding off a little while longer before beginning our migration to KiCAD 6.
Which version of this package should I point to for a CI runner? The GitLab example seems to point to the KiCAD 6 image:
image:
name: setsoft/kicad_auto:dev_k6
And throws some errors that appear to be associated with Kicad 6.0 config files.
Downloading artifacts
00:01
Running before_script and script
00:08
$ [ -f quad_uart.sch ] && kibot -e quad_uart.sch -b quad_uart.kicad_pcb -c config.kibot.yaml print_sch
WARNING:(W008) Unable to find KiCad configuration file (/root/.config/kicad/6.0/kicad_common.json) (kibot - config.py:144)
WARNING:(W010) Unable to find KiCad 3D modules (kibot - config.py:300)
ERROR:At line 1 of `/usr/share/kicad/symbols/Regulator_Linear.kicad_sym`: Missing library signature (kibot - kiplot.py:264)
ERROR:Line content: `(kicad_symbol_lib (version 20211014) (generator kicad_symbol_editor)` (kibot - kiplot.py:265)
Running after_script
Uploading artifacts for failed job
Uploading artifacts...
WARNING: Schematic_*.pdf: no matching files
ERROR: No files to upload
ERROR: Job failed: command terminated with exit code 1
kikit from kicad7_auto: latest complains as below:
Gtk-Message: 21:30:19.301: Failed to load module "canberra-gtk-module"
The same version of kikit (1.3.0-7) installed on the host processes the same set of files without any problem.
Can you add arm64 architecture image?
This would allow for easy deployment on Raspberry Pi boards an other ARM-based computers which begin to gain popularity.
The container doesn't seem to include /usr/bin/zip. This makes it difficult to create custom Zip files of the build outputs in a pipeline.
(In my case I wanted to rename some of the outputs, but only when building a release against a pushed tag, and them zip them).
Would you be OK with a PR to add it?
Running the following command:
eeschema_do run_erc <MY_SCHEMATIC> <MY_REPORT_FOLDER>
generates:
Segmentation fault (core dumped)
Running:
eeschema_do -h
works as expected
Running:
eeschema_do run_erc
generates the missing parameters error as expected.
The docker run command is:
$ docker run -t -d -u 1001:1001 -w /var/lib/jenkins/workspace/<MY_GITHUB_REPO> -v /var/lib/jenkins/workspace/<MY_GITHUB_REPO>:/var/lib/jenkins/workspace/<MY_GITHUB_REPO>:rw,z -v /var/lib/jenkins/workspace/<MY_GITHUB_REPO>@tmp:/var/lib/jenkins/workspace/<MY_GITHUB_REPO>@tmp:rw,z -e setsoft/kicad_auto:latest cat
(Repo info has been sanitized)
I am using Jenkins 2.303.2 with a declarative Jenkinsfile Pipeline, running docker on Ubuntu 20.04.
Hi
im using this setup here: https://github.com/swissbyte/kicadtest
and im getting this error message: ERROR:Time-out waiting for eeschema, giving up
do you have an idea why this could happen?
thank you ๐
in docker image kicad7_auto_full:dab7a024aecb (latest at the time of writing this issue) is used plugin InteractiveHtmlBom version 2.7.0. This version has broken handling of kicad DNP field (configuration entry dnp_field: 'kicad_dnp'
has no effect). This was fixed in later versions, it is working in v2.8.1. If I manually copy new version of plugin inside dunning docker image, configuration entry dnp_field: 'kicad_dnp'
is working as expected.
Could you put new version of InteractiveHtmlBom to the docker image?
So I created this simple Dockerfile:
FROM ghcr.io/inti-cmnb/kicad7_auto:1.6.3-2_k7.0.9_d12.1
WORKDIR /
Then executed docker build . -t test_builder
and finally docker run --rm -v ${PWD}:/project test_builder kibot --quick-start --dry
but the last command errors out:
Traceback (most recent call last):
File "/usr/bin/kibot", line 33, in <module>
sys.exit(load_entry_point('kibot==1.6.3', 'console_scripts', 'kibot')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/kibot/__main__.py", line 470, in main
generate_examples(args.start, args.dry, args.type)
File "/usr/lib/python3/dist-packages/kibot/kiplot.py", line 1048, in generate_examples
for f in _walk(start_dir, 6):
File "/usr/lib/python3/dist-packages/kibot/kiplot.py", line 981, in _walk
yield from _walk(entry.path, depth)
File "/usr/lib/python3/dist-packages/kibot/kiplot.py", line 981, in _walk
yield from _walk(entry.path, depth)
File "/usr/lib/python3/dist-packages/kibot/kiplot.py", line 981, in _walk
yield from _walk(entry.path, depth)
[Previous line repeated 2 more times]
File "/usr/lib/python3/dist-packages/kibot/kiplot.py", line 980, in _walk
if entry.is_dir() and depth > 0:
^^^^^^^^^^^^^^
PermissionError: [Errno 1] Operation not permitted: './proc/self/fd/5/map_files/400000-41f000'
In the latest kicad_auto
release, which is 1.6.4, the main KiCad version and the library versions are not the same.
$ docker run -it ghcr.io/inti-cmnb/kicad7_auto:1.6.4 /bin/bash
root@51b1fee6c322:/# apt list kicad*
Listing... Done
kicad-footprints/now 7.0.9-0~ubuntu23.04.1 all [installed,local]
kicad-git-filters/now 1.0.1-1 all [installed,local]
kicad-symbols/now 7.0.9-0~ubuntu23.04.1 all [installed,local]
kicad-templates/now 7.0.9-0~ubuntu23.04.1 all [installed,local]
kicad/now 7.0.10~ubuntu23.04.1 amd64 [installed,local]
root@51b1fee6c322:/#
I would expect the library version to be the same as KiCad (7.0.10) and not 7.0.9
In my case, this causes incorrect ERC warnings that don't appear when ERC is run manually in KiCad.
e.g. WARNING:(lib_symbol_issues) Symbol 'Buzzer' has been modified in library 'Device'
This is not an issue at all, you can close this when you read it.
I found this, and since you told me that you are aware of the waste of the space in container images. Maybe this can be helpful for you too.
https://www.ardanlabs.com/blog/2020/02/docker-images-part1-reducing-image-size.html
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.