onioniot/openwrt-sdk-wrapper

Easily build packages using the OpenWRT SDK

OpenWRT SDK Wrapper

Supporting scripts and config to easily build OpenWRT packages using the OpenWRT SDK

System Requirements

We recommend using Ubuntu 22.04 Linux to build this repo. See the OpenWRT Build System Setup instructions for details on what packages are required.

How to use this Repo

By default, this repo is setup to build OpenWRT packages (based on source code in a Git repo) for deployment.

1. Run bash onion_buildenv setup_sdk to download and setup the OpenWRT SDK
2. Two options here:
3. Run bash onion_buildenv build_all_packages to build all packages listed in the SDK_PACKAGES variable in profile
4. Run bash onion_buildenv build_packages <pkg1> <pkg2> ... <pkgN> to build the specified packages
5. Find compiled package ipk files at openwrt-sdk/bin/packages/mipsel_24kc/onion

Details

The bash onion_buildenv setup_sdk command:

• Will download and setup the OpenWRT SDK
• Add the custom feeds specified as env variable PACKAGE_FEEDS in a build profile to the SDK

The bash onion_buildenv build_packages <pkg1> <pkg2> ... <pkgN> command:

• Will compile the packages in the <pkg1> <pkg2> ... <pkgN> arguments

The bash onion_buildenv build_all_packages command:

• Will compile all the packages specified as env variable SDK_PACKAGES in a build profile to the SDK
• Will build a package index signed with the keys in the keys/ directory (learn more about package signing here and here)
• All output will be in the openwrt-sdk/bin/packages/mipsel_24kc/onion directory

---

Using this Repo for Development

By default, the packages are built from source code in git repo specified as env variable PACKAGE_FEEDS in a build profile to SDK.

While your packages are still under development and you are iterating on the package source code, it will be more straight-forward to work from a local copy of the package source instead.

To work from a local copy of your source code:

1. Clone your package source code repo to your development machine
2. Update env variable PACKAGE_FEEDS in a profile to the local source. See profiles/README.md for details.
3. Make any changes to the package source code
4. Run bash onion_buildenv setup_sdk to download and setup the OpenWRT SDK
5. Run bash onion_buildenv build_packages <pkg1> <pkg2> ... <pkgN> to build the desired packages
6. Find your compiled packages in the openwrt-sdk/bin/packages/mipsel_24kc/<feed>/ directory
7. Test compiled packages on the device. If more code changes are required, start again from step 3.

Example

If we wanted to build the packages from the Onion OpenWRT-Packages repo for development, the steps would be as follows:

1. clone your package source code repo to your development machine
   • Clone the repo: git clone https://github.com/OnionIoT/OpenWRT-Packages.git note the path to the cloned directory, in our case it's /home/ubuntu/OpenWRT-Packages
   • Checkout desired branch if required
2. Update env variable PACKAGE_FEEDS in a profile:
   • set a value: PACKAGE_FEEDS="src-link custom /home/ubuntu/OpenWRT-Packages", since /home/ubuntu/OpenWRT-Packages is the absolute path to the cloned directory from step 1
3. Make any changes to the package source files in /home/ubuntu/OpenWRT-Packages
4. Run bash onion_buildenv setup_sdk to download and setup the OpenWRT SDK
5. Run bash onion_buildenv build_packages to build the desired packages
6. Find the compiled packages in the openwrt-sdk/bin/packages/mipsel_24kc/custom/ directory

openwrt-sdk-wrapper's Issues

prepare_sdk function called multiple times during setup_sdk

Description

After updates in PR #8, it looks like the prepare_sdk function will be called multiple times when onion_buildenv is called with the setup_sdk argument

Steps to Reproduce

Running bash onion_buildenv setup_sdk

When the setup_sdk function is run

openwrt-sdk-wrapper/onion_buildenv

Lines 118 to 124 in 65ea6c1

	setup_sdk() {
	download_sdk || return 1
	verify_sdk || return 1
	extract_sdk || return 1
	update_package_feeds || return 1
	prepare_sdk || return 1
	}

It will call the update_package_feeds function and the prepare_sdk function.

However, the update_package_feeds function will ALSO call prepare_sdk (on line 108):

openwrt-sdk-wrapper/onion_buildenv

Lines 100 to 111 in 65ea6c1

	update_package_feeds() {
	## add packages feeds
	[ -z "$PACKAGE_FEEDS" ] && return 0
	echo -e "${PACKAGE_FEEDS}" > $TARGET_SDK_DIR/feeds.conf.default
	cat $TARGET_SDK_DIR/feeds.conf.default.org >> $TARGET_SDK_DIR/feeds.conf.default
	prepare_sdk || return 1
	return 0
	}

⚠️ Resulting in prepare_sdk being run twice.

Running bash onion_buildenv update_sdk -> No Issue

The update_sdk doesn't have this issue:

https://github.com/OnionIoT/openwrt-sdk-wrapper/blob/65ea6c1c3f72278e15ff64ea1fec9b819348910c/onion_buildenv#L113C15-L116

Proposed Resolution

We should avoid running prepare_sdk twice since the feeds update and installation operations take quite a bit of time

Possible solutions:

1. Remove prepare_sdk call from update_package_feeds, and add prepare_sdk call to update_sdk
2. Remote prepare_sdk from setup_sdk

Will leave this up to you @jempatel

Bug: Build Script should exit with non-zero if package compilation fails

Describe the bug

The CD system for this repo was reporting build succeeded even though the packages failed compilation.
Upon further inspection, it looks like the build script returns 0 no matter what happens to the package compilation

(This is related to OnionIoT/openwrt-imagebuilder-wrapper#4)

To Reproduce

1. Setup this repo: bash onion_buildenv setup_sdk
2. Make some changes to make the package compilation fail (the current CD system uses ubuntu 20.04 which causes the failure)
3. Try to compile the packages: bash onion_buildenv build_all_packages
4. Observe errors in compilation output
5. Run echo $? and observe 0

Expected behavior

Expecting the onion_buildenv script to return 1 if any of the packages cannot compile correctly.

More context

Attaching full CD logs

Relevant parts:

Error compiling packages

make[2]: Leaving directory '/codebuild/output/src642488073/src/openwrt-sdk/package/toolchain'
--
787 | time: package/toolchain/compile#0.28#0.07#0.30
788 | ERROR: package/toolchain failed to build.
789 | make[1]: *** [package/Makefile:116: package/toolchain/compile] Error 1
790 | make[1]: Leaving directory '/codebuild/output/src642488073/src/openwrt-sdk'
791 | make: *** [/codebuild/output/src642488073/src/openwrt-sdk/include/toplevel.mk:223: package/omega2-base/compile] Error 2
792 | make: Leaving directory '/codebuild/output/src642488073/src/openwrt-sdk'
793 | \n>> ERROR: Compile of package omega2-base returned code 2
794 | \n>> PACKAGE COMPILATION ERROR!\n

But the next CD phase includes a check that the build succeeded, and it passes:

 make[1] package/index
make: Leaving directory '/codebuild/output/src642488073/src/openwrt-sdk'

[Container] 2023/04/28 17:45:59 Phase complete: BUILD State: SUCCEEDED
[Container] 2023/04/28 17:45:59 Phase context status code:  Message: 
[Container] 2023/04/28 17:45:59 Entering phase POST_BUILD