Installing lxml on Mac OSX (10.11) inside a virtualenv with pip
The lxml library is a dependency for many popular Python projects such as Scrapy and Nikola. If you are a Mac OSX user, it's highly likely that you have encountered the following error when trying to install lxml or its dependants with pip:
In file included from src/lxml/lxml.etree.c:346: $WORKON_HOME/venv_name/build/lxml/src/lxml/includes/etree_defs.h:9:10: fatal error: 'libxml/xmlversion.h' file not found #include "libxml/xmlversion.h" ^ 1 error generated. error: command 'clang' failed with exit status 1
For a long time, the easy fix for this was to simply set the env var
which makes the setup.py for lxml download and build the latest versions of libxml2 and libxslt instead of using the system version, as suggested by the official installation instructions and this StackOverflow answer:
$ STATIC_DEPS=true pip install lxml
This was all fine until 20 Nov 2015, when version 2.9.3 of libxml2 was released. When installing with the above command, you will encounter an exception similar to this:
Exception: Command "make -j6" returned code 512
with the compilation error somewhere above in the traceback message:
xmlIO.c:1450:52: error: use of undeclared identifier 'LZMA_OK' ret = (__libxml2_xzclose((xzFile) context) == LZMA_OK ) ? 0 : -1; ^ CC uri.lo 1 error generated. make: *** [xmlIO.lo] Error 1
This seems to be an error introduced in version 2.9.3. For now, a reasonable workaround is to install lxml as before, but explicitly specify the version of lxml to use (version 2.9.2). In another StackOverflow answer, they manually download lxml and run setup.py with flags --static-deps, --libxml2-version and libxslt-version.
Note that we can still use pip (only version >= 7.0) and propagate the options to setup.py with --install-option:
$ pip install lxml --install-option="--static-deps=true" \ --install-option="--libxml2-version=2.9.2"
Alternatively, if you must us an older version of pip that does not support passing options, the setup.py for lxml is implemented to support passing command line options as environment variables (see setupinfo.py). This is why we were able to simply set STATIC_DEPS=true before.
In the same manner, we can just set LIBXML2_VERSION=2.9.2:
$ STATIC_DEPS=true LIBXML2_VERSION=2.9.2 pip install lxml Collecting lxml Using cached lxml-3.5.0.tar.gz Building wheels for collected packages: lxml Running setup.py bdist_wheel for lxml Stored in directory: [...] Successfully built lxml Installing collected packages: lxml Successfully installed lxml-3.5.0
Now lxml should be successfully installed:
$ python -c 'import lxml' $ python -c 'from lxml import etree'
CommentsComments powered by Disqus