Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
  • Loading branch information
s-m-e committed Sep 30, 2021
1 parent cd0aa58 commit 2154ddc
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions QEP.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ Distributing Python packages is currently a non-trivial problem, unfortunately.

For the purpose of this discussion, Python packages can be divided into two categories: Packages without binary components and Packages with binary components. The first category is relatively easy to package and distribute across all relevant platforms. The second category is both problematic but also much more interesting. One of Python's strengths is its ability to play nice with all sorts of other programming languages, e.g. C, C++ and Fortran. It has therefore become natural to integrate components written in other programming languages into Python packages. A clear majority of contemporary scientific and geospatial Python packages are following this approach and greatly benefit from it. It allows all sorts of interesting applications, such as accessing established numerical libraries, actual thread-based parallelism (something the [CPython interpreter](https://en.wikipedia.org/wiki/CPython) itself [is not capable of](https://wiki.python.org/moin/GlobalInterpreterLock) at the moment) and [access to GPUs for general-purpose computing](https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units). The downside is that both packaging *and installing* Python packages of this kind usually requires rather significant toolchains and sufficient background knowledge, among other issues. In the Linux ecosystem, the mentioned tool chains (as well as the mentioned background knowledge) are rather common, so it is safe to say that many (but not all) Linux users will not have massive problems installing common scientific and geospatial Python packages. On OS X, the overall picture is somewhat similar. However, on Windows, things are known to be highly problematic.

The most widely used distribution channel for Python packages happens to be the [Python Package Index](https://pypi.org/). As of May 2020, it holds roughly 1/4 million packages. Python packages can be installed from the index through [`pip`](https://pip.pypa.io/en/stable/), a fully featured [package manager](https://en.wikipedia.org/wiki/Package_manager). Both `pip` and the index can handle two types of package formats: (1) "source distributions", i.e. tarballs containing the raw source code and installation / build scripts, and (2) "[wheels](https://www.python.org/dev/peps/pep-0427/)", i.e. pre-build Python packages. While wheels are becoming more common for popular Python packages thanks to the efforts of projects like [manylinux](https://github.com/pypa/manylinux), they are still not as widely adapted (and as easy to build) as they should be. They are also rather rare for Windows, which leaves Windows users in the awkward position of having to build, i.e. compile, binary components within Python packages themselves.
The most widely used distribution channel for Python packages happens to be the [Python Package Index](https://pypi.org/). As of May 2020, it holds roughly 1/4 million packages. Python packages can be installed from the index through [`pip`](https://pip.pypa.io/en/stable/), a fully featured [package manager](https://en.wikipedia.org/wiki/Package_manager). Both `pip` and the index can handle two types of package formats: (1) "source distributions", i.e. tarballs containing the raw source code and installation / build scripts, and (2) "[wheels](https://www.python.org/dev/peps/pep-0427/)", i.e. pre-built Python packages. While wheels are becoming more common for popular Python packages thanks to the efforts of projects like [manylinux](https://github.com/pypa/manylinux), they are still not as widely adapted (and as easy to build) as they should be. They are also rather rare for Windows, which leaves Windows users in the awkward position of having to build, i.e. compile, binary components within Python packages themselves.

All of the described problems - from complicated to use setuptools to the lack of pre-build wheels for Windows - let to the advent of [Python distributions](https://wiki.python.org/moin/PythonDistributions). Very similar to Linux distributions, Python distributions usually bundle pre-build Python packages with relevant toolchains and their own package managers. As of 2019, [Anaconda](https://anaconda.org/) is probably [the most widely used Python distribution](https://www.jetbrains.com/lp/python-developers-survey-2019/#PythonVersions). Its package manager is named [`conda`](https://github.com/conda/conda). Currently, Anaconda is the best if not the only option of installing a lot of relevant scientific and geospatial Python packages on Windows. Anaconda is also offered for Linux and OS X. Compared to `pip`, `conda` can manage not only Python packages but all sorts of other tools and dependencies, which makes it rather similar to [`apt`](https://en.wikipedia.org/wiki/APT_(software)), [`yum`](https://en.wikipedia.org/wiki/Yum_(software)) or [`zypper`](https://en.wikipedia.org/wiki/ZYpp). Ignoring features specific to Python, Anaconda is therefore very similar to a Linux distribution. It ships an entire stack of tools and libraries all the way down to [libc](https://en.wikipedia.org/wiki/C_standard_library) but, other than Linux distributions, no operating system kernel. Compared to Linux distributions, Anaconda also ships a much larger and significantly younger selection of Python packages. While Anaconda is technically both a company and a commercial product, the package manager itself, the package format specification and the package repository specification are open source. The [conda-forge project](https://conda-forge.org/) is probably the most widely known pure open source non-profit user of those specifications and offers independent, Anaconda-compatible [packages](https://anaconda.org/conda-forge) and a [distribution installer](https://github.com/conda-forge/miniforge). Beside conda-forge, there are also other [re-implemenations](https://github.com/QuantStack/mamba) of the `conda` package manager itself based on its specification. The conda package specification is better documented than setuptools and wheel, somewhat easier to use for complex packages and offers a much higher degree of flexibility.
All of the described problems - from complicated to use setuptools to the lack of pre-built wheels for Windows - let to the advent of [Python distributions](https://wiki.python.org/moin/PythonDistributions). Very similar to Linux distributions, Python distributions usually bundle pre-built Python packages with relevant toolchains and their own package managers. As of 2019, [Anaconda](https://anaconda.org/) is probably [the most widely used Python distribution](https://www.jetbrains.com/lp/python-developers-survey-2019/#PythonVersions). Its package manager is named [`conda`](https://github.com/conda/conda). Currently, Anaconda is the best if not the only option of installing a lot of relevant scientific and geospatial Python packages on Windows. Anaconda is also offered for Linux and OS X. Compared to `pip`, `conda` can manage not only Python packages but all sorts of other tools and dependencies, which makes it rather similar to [`apt`](https://en.wikipedia.org/wiki/APT_(software)), [`yum`](https://en.wikipedia.org/wiki/Yum_(software)) or [`zypper`](https://en.wikipedia.org/wiki/ZYpp). Ignoring features specific to Python, Anaconda is therefore very similar to a Linux distribution. It ships an entire stack of tools and libraries all the way down to [libc](https://en.wikipedia.org/wiki/C_standard_library) but, other than Linux distributions, no operating system kernel. Compared to Linux distributions, Anaconda also ships a much larger and significantly younger selection of Python packages. While Anaconda is technically both a company and a commercial product, the package manager itself, the package format specification and the package repository specification are open source. The [conda-forge project](https://conda-forge.org/) is probably the most widely known pure open source non-profit user of those specifications and offers independent, Anaconda-compatible [packages](https://anaconda.org/conda-forge) and a [distribution installer](https://github.com/conda-forge/miniforge). Beside conda-forge, there are also other [re-implemenations](https://github.com/QuantStack/mamba) of the `conda` package manager itself based on its specification. The conda package specification is better documented than setuptools and wheel, somewhat easier to use for complex packages and offers a much higher degree of flexibility.

## How does QGIS fit into the bigger picture?

Expand Down

0 comments on commit 2154ddc

Please sign in to comment.