Linux/AM3358: OpenSSL update in Yocto

Hi,

Applied the patch you suggested, but am still getting the same error.

[PATCH]

$ git diff bitbake/lib/bb/utils.py

diff --git a/sources/bitbake/lib/bb/utils.py b/sources/bitbake/lib/bb/utils.py

index 3544bbe..a6fe93b 100644

--- a/sources/bitbake/lib/bb/utils.py

+++ b/sources/bitbake/lib/bb/utils.py

@@ -967,6 +967,26 @@ def contains_any(variable, checkvalues, truevalue, falsevalue, d):

        return truevalue

    return falsevalue

+def filter(variable, checkvalues, d):

+    """Return all words in the variable that are present in the checkvalues.

+     Arguments:

+     variable -- the variable name. This will be fetched and expanded (using

+    d.getVar(variable)) and then split into a set().

+     checkvalues -- if this is a string it is split on whitespace into a set(),

+    otherwise coerced directly into a set().

+     d -- the data store.

+    """

+    val = d.getVar(variable)

+    if not val:

+        return ''

+    val = set(val.split())

+    if isinstance(checkvalues, str):

+        checkvalues = set(checkvalues.split())

+    else:

+        checkvalues = set(checkvalues)

+    return ' '.join(sorted(checkvalues & val))

+

+

def cpu_count():

    return multiprocessing.cpu_count()

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[ERROR]

$ bitbake openssl

NOTE: Started PRServer with DBfile: /home/einfochips/project/tms/SEM3_UCM/build/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 45467, PID: 17905

Loading cache: 100% |####################################################################################################################################| ETA:  00:00:00

Loaded 2859 entries from dependency cache.

WARNING: /home/einfochips/project/tms/SEM3_UCM/sources/oe-core/meta/recipes-connectivity/openssl/openssl_1.0.2o.bb: Exception during build_dependencies for do_install-:--:--

WARNING: /home/einfochips/project/tms/SEM3_UCM/sources/oe-core/meta/recipes-connectivity/openssl/openssl_1.0.2o.bb: Error during finalise of /home/einfochips/project/tms/SEM3_UCM/sources/oe-core/meta/recipes-connectivity/openssl/openssl_1.0.2o.bb

ERROR: ExpansionError during parsing /home/einfochips/project/tms/SEM3_UCM/sources/oe-core/meta/recipes-connectivity/openssl/openssl_1.0.2o.bb: Failure expanding variable do_install, expression was # Create /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image//usr to fix parallel issues

mkdir -p /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image//usr/

oe_runmake INSTALL_PREFIX="/home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image" MANDIR="/usr/share/man" install

oe_libinstall -so libcrypto /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib

oe_libinstall -so libssl /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib

install -d /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/include

cp --dereference -R include/openssl /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/include

install -Dm 0755 /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/openssl-c_rehash.sh /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/bin/c_rehash

sed -i -e 's,/etc/openssl,/etc/ssl,g' /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/bin/c_rehash

oe_multilib_header openssl/opensslconf.h

if [ "${@bb.utils.filter('PACKAGECONFIG', 'perl', d)}" ]; then

sed -i -e '1s,.*,#!/usr/bin/env perl,' /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/misc/CA.pl

sed -i -e '1s,.*,#!/usr/bin/env perl,' /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/misc/tsget

else

rm -f /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/misc/CA.pl /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/misc/tsget

fi

# Create SSL structure

install -d /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/etc/ssl/

mv /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/openssl.cnf \

  /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/certs \

  /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/private \

  \

  /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/etc/ssl/

ln -sf /etc/ssl/certs /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/certs

ln -sf /etc/ssl/private /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/private

ln -sf /etc/ssl/openssl.cnf /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/lib/ssl/openssl.cnf

# Rename man pages to prefix openssl10-*

for f in `find /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/share/man -type f`; do

   mv $f $(dirname $f)/openssl10-$(basename $f)

done

for f in `find /home/einfochips/project/tms/SEM3_UCM/build/arago-tmp-external-linaro-toolchain/work/armv7ahf-neon-linux-gnueabi/openssl/1.0.2o-r0.arago0/image/usr/share/man -type l`; do

   ln_f=`readlink $f`

   rm -f $f

   ln -s openssl10-$ln_f $(dirname $f)/openssl10-$(basename $f)

done

which triggered exception TypeError: getVar() takes at least 3 arguments (2 given)

Summary: There were 2 WARNING messages shown.

Summary: There was 1 ERROR message shown, returning a non-zero exit code.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[openssl directory and its content is copied from Rocko version to Krogoth version]

NOTE: We have removed the openssl  directory of Krogoth version and then replaced the whole directory and its content with openssl directory from the Rocko version.

$ tree openssl/
openssl/
├── openssl
│   ├── 0001-Take-linking-flags-from-LDFLAGS-env-var.patch
│   ├── openssl-c_rehash.sh
│   └── run-ptest
├── openssl-1.0.2o
│   ├── 0001-Fix-build-with-clang-using-external-assembler.patch
│   ├── 0001-openssl-force-soft-link-to-avoid-rare-race.patch
│   ├── configure-musl-target.patch
│   ├── configure-targets.patch
│   ├── debian
│   │   ├── ca.patch
│   │   ├── c_rehash-compat.patch
│   │   ├── debian-targets.patch
│   │   ├── man-dir.patch
│   │   ├── man-section.patch
│   │   ├── no-rpath.patch
│   │   ├── no-symbolic.patch
│   │   └── pic.patch
│   ├── debian1.0.2
│   │   ├── block_digicert_malaysia.patch
│   │   ├── block_diginotar.patch
│   │   ├── soname.patch
│   │   └── version-script.patch
│   ├── engines-install-in-libdir-ssl.patch
│   ├── find.pl
│   ├── Makefiles-ptest.patch
│   ├── oe-ldflags.patch
│   ├── openssl-c_rehash.sh
│   ├── openssl-fix-des.pod-error.patch
│   ├── openssl_fix_for_x32.patch
│   ├── openssl-util-perlpath.pl-cwd.patch
│   ├── parallel.patch
│   ├── ptest-deps.patch
│   ├── ptest_makefile_deps.patch
│   ├── reproducible-cflags.patch
│   ├── reproducible-mkbuildinf.patch
│   ├── run-ptest
│   ├── shared-libs.patch
│   └── Use-SHA256-not-MD5-as-default-digest.patch
├── openssl_1.0.2o.bb
├── openssl10.inc
└── openssl_1.1.0h.bb

Regards,

Sunny.

Oh, I see the difference in the error now. Thanks your answers solved the issue.

Am a bit naive in using Yocto. To recapitulate, I followed the steps mentioned below to integrate openssl-1.0.2o from Rocko to Krogoth. Let me know if anything else needs to be taken care of. 

1. Deleted openssl directory currently existing in the Krogoth source under "<source>/oe-core/meta/recipes-connectivity" folder.
2. Copied openssl directory from Rocko to Krogoth under "<source>/oe-core/meta/recipes-connectivity" folder.
3. Added openssl_%.bbappend file under "./meta-arago/meta-arago-distro/recipes-connectivity/openssl/" directory of Krogoth source.
4. Added manpages.bbclass and relative_symlinks.bbclass from Rocko to Krogoth under "<source>/oe-core/meta/classes/" folder.
5. Patch the Krogoth source with this patch.
6. Changed val = d.getVar(variable) to val = d.getVar(variable, True) in the patched code as described in step 5.
7. Patch the Krogoth source with this patch to overcome error "Exception: AttributeError: 'module' object has no attribute 'replace_absolute_symlinks'"
8. Changed d.getVar('D') to d.getVar('D', True) in the relative_symlinks.bbclass under "<source>/oe-core/meta/classes/" folder.
9. Removed openssl_1.1.0h.bb in Krogoth source from "<source>/oe-core/meta/recipes-connectivity/openssl/" folder
10. Removed openssl_1.0.2h.bbappend in Krogoth source from "<source>/meta-arago/meta-arago-distro/recipes-connectivity/openssl/" folder.
11. Applied the following commands and it compiled successfully.

$ bitbake openssl -c cleansstate

NOTE: Started PRServer with DBfile: /home/einfochips/project/tms/SEM3_UCM/build/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 35043, PID: 19786
Loading cache: 100% |####################################################################################################################################| ETA: 00:00:00
Loaded 2865 entries from dependency cache.
Parsing recipes: 100% |##################################################################################################################################| Time: 00:00:00
Parsing of 2263 .bb files complete (2262 cached, 1 parsed). 2859 targets, 474 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION = "1.30.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-16.04"
TARGET_SYS = "arm-linux-gnueabi"
MACHINE = "am335x-evm"
DISTRO = "arago"
DISTRO_VERSION = "2016.12"
TUNE_FEATURES = "arm armv7a vfp thumb neon callconvention-hard"
TARGET_FPU = "hard"
meta-arago-distro
meta-arago-extras
meta-qt5
meta-networking
meta-ruby
meta-python
meta-oe
meta-ti
meta-linaro-toolchain
meta-optee
meta
meta-sem3 = "24VDC_dev:517ecbb4d9c5c872b6db726337a780c950ebf92f"

NOTE: Preparing RunQueue
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory

$ bitbake openssl

NOTE: Started PRServer with DBfile: /home/einfochips/project/tms/SEM3_UCM/build/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 35309, PID: 7178
Loading cache: 100% |####################################################################################################################################| ETA: 00:00:00
Loaded 2865 entries from dependency cache.
Parsing recipes: 100% |##################################################################################################################################| Time: 00:00:00
Parsing of 2263 .bb files complete (2261 cached, 2 parsed). 2859 targets, 474 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION = "1.30.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-16.04"
TARGET_SYS = "arm-linux-gnueabi"
MACHINE = "am335x-evm"
DISTRO = "arago"
DISTRO_VERSION = "2016.12"
TUNE_FEATURES = "arm armv7a vfp thumb neon callconvention-hard"
TARGET_FPU = "hard"
meta-arago-distro
meta-arago-extras
meta-qt5
meta-networking
meta-ruby
meta-python
meta-oe
meta-ti
meta-linaro-toolchain
meta-optee
meta
meta-sem3 = "24VDC_dev:517ecbb4d9c5c872b6db726337a780c950ebf92f"

NOTE: Preparing RunQueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: external-linaro-toolchain-2016.11-r0.arago32 do_package: QA Issue: external-linaro-toolchain: Files/directories were installed but not shipped in any package:
/lib/libgfortran.spec
/lib/libsanitizer.spec
/usr/sbin
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
external-linaro-toolchain: 3 installed and not shipped files. [installed-vs-shipped]
NOTE: Tasks Summary: Attempted 431 tasks of which 389 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory

Summary: There was 1 WARNING message shown.

Regards,

Sunny.