Yocto Build fails for wolfssh and wolfmqtt

[wolfssl]

Yocto Build fails for wolfssh and wolfmqtt

[kaleb-himes]

Hi @prateek-khatri,

Thank you for reaching out to wolfSSL. Can you tell us a bit about who this effort is for and what you are working on? If you do not wish to disclose publicly feel free to reach out to us at [email protected] for a more private venue.

I'll have one of the wolfSSH developers review your notes and post some feedback shortly.

Regards,

K

[prateek-khatri]

We are designing an MQTT client for streaming some logs. At this point we are evaluating performance and ease of use for MQTT Client libraries. If WolfSSL and WolfMqtt meets our needs we may buy the commercial license. I am afraid I won't be able to disclose more details at this point.

[kaleb-himes]

@prateek-khatri,

Not a problem, thanks for sharing what you can, it helps us to understand the nature of the inquiry and we appreciate what details you can provide.

Do you have a specific device or linux distro you are targeting at the moment? If not I have a test script I could send over that builds for qemux86. I just ran it to see if there was something changed that broke the build steps but everything checks out. If it helps we specifically target yocto-2.6 of the git://git.yoctoproject.org/poky project. (We haven't yet tested with newer versions yet).

Regards,

K

[prateek-khatri]

Hi Kaleb,

I am targetting the AM65x EVM from TI. It is using the processor SDK for Linux provided by TI.
The build configuration can also be seen in the original post:
Build Configuration: BB_VERSION = "1.36.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "ubuntu-18.04" TARGET_SYS = "aarch64-linux" MACHINE = "am65xx-evm" DISTRO = "arago" DISTRO_VERSION = "2018.10" TUNE_FEATURES = "aarch64" TARGET_FPU = ""

[kaleb-himes]

@prateek-khatri,

Thanks, sorry I overlooked it. Here are the build specs I'm using, quite different as you can see:

===== Compiling wolfSSH Recipe ==========================================
WARNING: Host distribution "ubuntu-18.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |############################################################################| Time: 0:00:00
Loaded 1276 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.40.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "i586-poky-linux"
MACHINE              = "qemux86"
DISTRO               = "poky"
DISTRO_VERSION       = "2.6"
TUNE_FEATURES        = "m32 i586"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "my-yocto-2.6:84eecb017ef92ef36b4df730908828e54aeff85c"
meta-wolfssl         = "master:f4842e35cf7d69563d0f5c985cd2a7638ba69c00"

Could you try building with distro version 2.6 and let us know if you see the same errors as you are getting with 2018.10?

[prateek-khatri]

The build environment is provided by TI as processor SDK. I am not sure if I can change the distro version.

[kaleb-himes]

@prateek-khatri,

Is there a guide or instructions for building that one of our engineers could use to try and reproduce the issue on our side?

Regards,

K

[prateek-khatri]

http://software-dl.ti.com/processor-sdk-linux/esd/docs/05_03_00_07/linux/Overview_Building_the_SDK.html

[kaleb-himes]

Thanks! I'll try to reproduce the issue and let you know if we can find a solution for it.

[prateek-khatri]

Thanks Kaleb, let me know if you have more questions along the way.

Please make note for the MACHINE variable I am using for bitbake:

MACHINE=am65xx-evm bitbake wolfssh

[kaleb-himes]

NOTE 1) Selected ./oe-layertool-setup.sh -f configs/processor-sdk/processor-sdk-05.03.00.07-config.txt from guide (Not sure which version you used) I got stuck on this step though and testing on a VM it ran out of memory :-(

Moved over to a physical machine with more memory and trying again.

[prateek-khatri]

Hi Kaleb,

Yes that is the version I used. You would need around 200 GB hard drive space for this build.

Thanks for going through the trouble of building the whole thing, I really appreciate it.

Prateek

[kaleb-himes]

@prateek-khatri,

No problem, I got the toolchain nearly built but it's close to the end of the day here so I won't be able to test wolfSSL till the beginning or middle of next week (My Mondays and Tuesdays are pretty booked at the moment) Would an update by next Wed or Thursday be OK for your timeline?

• KH

[prateek-khatri]

Hi Kaleb,

I was able to build wolfssl, however SSH and MQTT were giving the issue.

Yes, Wed or Thursday would work.

Thank you so much!

[prateek-khatri]

@kaleb-himes
Hi Kaleb,
Any update regarding this?

[kaleb-himes]

@prateek-khatri ,

I am so very sorry no, I had some priorities come through that tied up my week. Thank you so much for the reminder and bringing this back to the top of my work queue. I just logged into the machine and the step $ ./oe-layertool-setup.sh -f configs/processor-sdk/processor-sdk-05.03.00.07-config.txt did finish successfully so I am ready to perform next steps. Will post an update ASAP.

• KH

[kaleb-himes]

UPDATE:

I can reproduce the successful build of wolfSSL and am now working to reproduce the failure of wolfSSH bitbake build.

(Very slow build process sorry for the delay again)

[prateek-khatri]

Hi Kaleb,

Yes the SSH and MQTT were the ones failing.

Thanks,
Prateek

[kaleb-himes]

Update: Still working late on my Friday trying to sort this out, Can easily reproduce the errors but the TISDK is downloading wolfSSL-3.12.0 a much older version when I build SSH and I am unable to determine where that is coming from when our bitbake file specifies to use 4.0.0. I suspect that TI is distributing an older version of wolfSSL with this SDK.

[kaleb-himes]

In the older version of wolfSSL (3.12.0) we had not yet updated to WC_SHA, we were still using just SHA. The wolfSSH product assumes the newer defines WC_SHA so it is failing to build with the older wolfSSL library.

[prateek-khatri]

Hi Kaleb, Thanks for testing it out on Friday. I am updating my SDK to 5.1 and will know on Monday if that fixes it. However, I download the layer and built the recipe from WolfSSL repo. Thank you, Prateek

…

On Fri, Jun 21, 2019, 7:43 PM Kaleb Himes ***@***.***> wrote: Update: Still working late on my Friday trying to sort this out, Can easily reproduce the errors but the TISDK is download wolfSSL-3.12.0 a much older version when I build SSH and I am unable to determine where that is coming from when our bitbake file specifies to use 4.0.0. I suspect that TI is distributing an older version of wolfSSL with this SDK. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9?email_source=notifications&email_token=ADUNHJKLB5SKKCPWTI4HJZLP3WGTRA5CNFSM4HYLSP22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYJ6JOY#issuecomment-504620219>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADUNHJNAIW25UMOEXHBXJN3P3WGTRANCNFSM4HYLSP2Q> .

[kaleb-himes]

I noticed it when I saw this warning:

WARNING: wolfssl-3.12.0-r0 do_configure: QA Issue: wolfssl: configure was passed unrecognised options: --enable-ssh --enable-cryptodev [unknown-configure-option]

wolfssl-3.12.0 doesn't even know about wolfSSH, we hadn't added the config for it yet. I suspect we are going to have to contact TI support and have them put wolfSSL-4.0.0 into their SDK to get this resolved since meta-wolfssl is resolving to wolfssl-3.12.0 when used with their SDK.

[kaleb-himes]

Update: Trying two other things:

1. Can we replace the meta-wolfSSL ourselves in the SDK.
2. If not 1, going to see if using a different name will by-pass the wolfssl-3.12 in the SDK

Will post results when testing complete.

[kaleb-himes]

Update:

1. May work.

• Replaced sources/meta-openembedded/meta-networking/recipes-connectivity/wolfssl/wolfssl-3.12.0.bb with wolfssl-4.0.0.bb (@cconlon sent an update to meta-openembedded to get this updated)

• add --enable-singlethreaded to wolfssl%.bb.append in meta-wolfssh directory, working to get SINGLE_THREADED set in CFLAGS in wolfssh-1.3.0.bb, ran out of time to test before end-of-day

[kaleb-himes]

@prateek-khatri

Just wanted to let you know I’m out of the office till Monday but will resume trying to identify a resolution to the noted issue as soon as possible.