Cross compile for my target board

[shubham101096]

Hi

I need the pycryptodome/lxml package for the following architecture of my target board. The target board does not have internet access so I cannot directly install pycryptodome/lxml on it. I need a method to compile it on my ubuntu 64 bit pc and then transfer it on my board. Following describes the toolchain I am using and the architecture of the target board. I cant find a .whl file or anything else for this architecture. Can I solve this problem using crossenv and how?

PPC_e5500_CC_BASE_DIR := /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux
ifeq (${TARGET_CPU}, PPC_e5500)
ifeq (${TARGET_CPU_BIT},)
$(info TARGET_CPU_BIT must be 32BIT/64BIT)
$(warning TARGET_CPU_BIT not specified, assuming 32BIT)
TARGET_CPU_BIT = 32BIT
endif
TARGET_ARCH = powerpc
COMPILER_TYPE = gcc
ifeq (${TARGET_CPU_BIT}, 32BIT)
#TARGET_CPU_BIT = 32BIT
CROSS_COMPILE_PATH = ${PPC_e5500_CC_BASE_DIR}/usr/bin/powerpc-fsl-linux:${PPC_e5500_CC_BASE_DIR}/usr/bin
CROSS_COMPILE = powerpc-fsl-linux-
HOST = powerpc-fsl-linux
CC_SYSROOT = ${PPC_e5500_CC_BASE_DIR}/../ppce5500-fsl-linux
COMMON_CFLAGS += -m32 -mhard-float -mcpu=e5500
COMMON_CFLAGS += --sysroot=${CC_SYSROOT}

Regards
Shubham Mishra

[benfogle]

Building a wheel for an embedded platform is an intended use case for crossenv. While I haven't tested this on a powerpc target yet, if you have successfully compiled Python for this board, then crossenv should work.

[shubham101096]

I installed the python-docx package using crossenv. The names of the .so files have "cpython-36m-powerpc-linux-gnu" written in them because I cross compiled them. For eg: the name of _raw_ecb.so is written as _raw_ecb.cpython-36m-powerpc-linux-gnu.so . So will it cause problem because I think code would try to find _raw_ecb.so and not "_raw_ecb.cpython-36m-powerpc-linux-gnu.so". Will I have to remove "cpython-36m-powerpc-linux-gnu" from every .so file name ?

[benfogle]

No, I don't think you'll need to rename anything, and those file names are a good sign that everything is working.

What your seeing is a PEP3149 ABI-tagged .so file, which was introduced in Python 3.2. Python will produce the long name by default and prefers that on import, but it will still fall back to the shorter version (_raw_ecb.so) if it can't find the long one.

[shubham101096]

I am trying to cross compile lxml package for my board which has following coniguration:

PPC_e5500_CC_BASE_DIR := /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux
TARGET_CPU_BIT = 32BIT
TARGET_ARCH = powerpc
COMPILER_TYPE = gcc
CROSS_COMPILE_PATH = ${PPC_e5500_CC_BASE_DIR}/usr/bin/powerpc-fsl-linux:${PPC_e5500_CC_BASE_DIR}/usr/bin
CROSS_COMPILE = powerpc-fsl-linux-
HOST = powerpc-fsl-linux
CC_SYSROOT = ${PPC_e5500_CC_BASE_DIR}/../ppce5500-fsl-linux
COMMON_CFLAGS += -m32 -mhard-float -mcpu=e5500
COMMON_CFLAGS += --sysroot=${CC_SYSROOT}

My pc has ubuntu 16.04 64bit OS. I am using the crossenv package for this purpose (https://pypi.org/project/crossenv/). But I am getting the following error when installing lxml.

(cross) p@OptiPlex-5050:~/Desktop$ pip install lxml
Collecting lxml
Using cached https://files.pythonhosted.org/packages/39/2b/0a66d5436f237aff76b91e68b4d8c041d145ad0a2cdeefe2c42f76ba2857/lxml-4.5.0.tar.gz
Installing collected packages: lxml
Running setup.py install for lxml ... error
Complete output from command /home/prateek/Desktop/v/cross/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-nr9hop30/lxml/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-qq5739wi-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/prateek/Desktop/v/cross/include/site/python3.6/lxml:

---

• Crossenv has leaked into another Python interpreter!
• You should probably file a bug report.
• Version 3.6.9 (default, Nov 7 2019, 10:44:02)
  [GCC 8.3.0]
• Executable /home/prateek/Desktop/v/cross/bin/python3

---

Building lxml version 4.5.0.
Building without Cython.
Using build configuration of libxslt 1.1.29
running install
running build
running build_py
creating build
creating build/lib.l--x-3.6
creating build/lib.l--x-3.6/lxml
copying src/lxml/sax.py -> build/lib.l--x-3.6/lxml
copying src/lxml/pyclasslookup.py -> build/lib.l--x-3.6/lxml
copying src/lxml/doctestcompare.py -> build/lib.l--x-3.6/lxml
copying src/lxml/_elementpath.py -> build/lib.l--x-3.6/lxml
copying src/lxml/init.py -> build/lib.l--x-3.6/lxml
copying src/lxml/usedoctest.py -> build/lib.l--x-3.6/lxml
copying src/lxml/cssselect.py -> build/lib.l--x-3.6/lxml
copying src/lxml/builder.py -> build/lib.l--x-3.6/lxml
copying src/lxml/ElementInclude.py -> build/lib.l--x-3.6/lxml
creating build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/init.py -> build/lib.l--x-3.6/lxml/includes
creating build/lib.l--x-3.6/lxml/html
copying src/lxml/html/defs.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/formfill.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/_html5builder.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/init.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/_diffcommand.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/clean.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/html5parser.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/soupparser.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/diff.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/usedoctest.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/builder.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/_setmixin.py -> build/lib.l--x-3.6/lxml/html
copying src/lxml/html/ElementSoup.py -> build/lib.l--x-3.6/lxml/html
creating build/lib.l--x-3.6/lxml/isoschematron
copying src/lxml/isoschematron/init.py -> build/lib.l--x-3.6/lxml/isoschematron
copying src/lxml/etree.h -> build/lib.l--x-3.6/lxml
copying src/lxml/etree_api.h -> build/lib.l--x-3.6/lxml
copying src/lxml/lxml.etree.h -> build/lib.l--x-3.6/lxml
copying src/lxml/lxml.etree_api.h -> build/lib.l--x-3.6/lxml
copying src/lxml/includes/dtdvalid.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/xinclude.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/init.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/htmlparser.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/xmlerror.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/xmlschema.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/xslt.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/xmlparser.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/etreepublic.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/c14n.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/uri.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/schematron.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/tree.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/config.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/xpath.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/relaxng.pxd -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/lxml-version.h -> build/lib.l--x-3.6/lxml/includes
copying src/lxml/includes/etree_defs.h -> build/lib.l--x-3.6/lxml/includes
creating build/lib.l--x-3.6/lxml/isoschematron/resources
creating build/lib.l--x-3.6/lxml/isoschematron/resources/rng
copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.l--x-3.6/lxml/isoschematron/resources/rng
creating build/lib.l--x-3.6/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl
creating build/lib.l--x-3.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.l--x-3.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build/temp.l--x-3.6
creating build/temp.l--x-3.6/src
creating build/temp.l--x-3.6/src/lxml
powerpc-fsl-linux-gcc -m32 -mhard-float -mcpu=e5500 --sysroot=/opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux -g -DBOARD_epih -DPPC_e5500 -DCPLD_8BIT -DCPLD_LOCALBUS -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc -Isrc/lxml/includes -I/home/prateek/Desktop/v/cross/include -I/home/prateek/workspace/GPON/targets/epih/fs/usr/local/include/python3.6m -c src/lxml/etree.c -o build/temp.l--x-3.6/src/lxml/etree.o -w
In file included from src/lxml/etree.c:692:0:
src/lxml/includes/etree_defs.h:23:32: fatal error: libxslt/xsltconfig.h: No such file or directory
#include "libxslt/xsltconfig.h"
^
compilation terminated.
Compile failed: command 'powerpc-fsl-linux-gcc' failed with exit status 1
creating tmp
cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /tmp/xmlXPathInitwnv0m7j2.c -o tmp/xmlXPathInitwnv0m7j2.o
In file included from /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/include/endian.h:36:0,
from /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/include/bits/waitstatus.h:64,
from /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/include/stdlib.h:42,
from /usr/include/libxml2/libxml/SAX.h:16,
from /usr/include/libxml2/libxml/globals.h:20,
from /usr/include/libxml2/libxml/xmlIO.h:117,
from /usr/include/libxml2/libxml/parser.h:811,
from /usr/include/libxml2/libxml/xmlerror.h:10,
from /usr/include/libxml2/libxml/xpath.h:26,
from /tmp/xmlXPathInitwnv0m7j2.c:1:
/opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/include/bits/endian.h:33:4: warning: #warning Cannot determine current byte order, assuming big-endian. [-Wcpp]

warning Cannot determine current byte order, assuming big-endian.

^~~~~~~
/tmp/xmlXPathInitwnv0m7j2.c:2:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main (int argc, char **argv) {
^~~~
cc tmp/xmlXPathInitwnv0m7j2.o -lxml2 -o a.out
/usr/bin/ld: /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/lib/../lib/Scrt1.o: Relocations in generic ELF (EM: 20)
/usr/bin/ld: /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/lib/../lib/Scrt1.o: Relocations in generic ELF (EM: 20)
/usr/bin/ld: /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/lib/../lib/Scrt1.o: Relocations in generic ELF (EM: 20)
/usr/bin/ld: /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/lib/../lib/Scrt1.o: Relocations in generic ELF (EM: 20)
/usr/bin/ld: /opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/lib/../lib/Scrt1.o: Relocations in generic ELF (EM: 20)
/opt/fsl-qoriq/2.0/sysroots/x86_64-fslsdk-linux/../ppce5500-fsl-linux/usr/lib/../lib/Scrt1.o: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status

---

Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?

---

error: command 'powerpc-fsl-linux-gcc' failed with exit status 1

---

Command "/home/prateek/Desktop/v/cross/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-nr9hop30/lxml/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-qq5739wi-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/prateek/Desktop/v/cross/include/site/python3.6/lxml" failed with error code 1 in /tmp/pip-build-nr9hop30/lxml/
You are using pip version 9.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Then I tried installing libxml2 and got the following error:

(cross) p@OptiPlex-5050:~/Desktop$ pip install libxml2
Collecting libxml2
Could not find a version that satisfies the requirement libxml2 (from versions: )
No matching distribution found for libxml2
You are using pip version 9.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

If possible, please help me in resolving this issue. Thank you.

[benfogle]

Libxml2 is a C library that the lxml uses, along with libxslt. Lxml requires both to build and to run. You will need to do the following:

1. Download libxml2 and libxslt from ftp://xmlsoft.org/libxml2/
2. Cross compile both for your system and install them as appropriate. I'm afraid I can't provide much guidance here except to say that you will need the header files to build, and the resulting .so files will need to be somewhere on your firmware where the linker can find them.
3. Let lxml know where to find them. If you installed them to sysroot, maybe you don't have to do anything. Otherwise set the environment variables CFLAGS and LDFLAGS as needed. You can do this by passing --env when building crossenv, or by simply running export CFLAGS=... after activating the environment.
4. Pip install should work. If it doesn't you'll need to examine the output (that will look much like what you posted above) to figure out the error. The important line is the one that begins with powerpc-fsl-linux-gcc -m32 -mhard-float .... Make sure that it has the right -I and -L flags to find libxml2 and libxslt.