.. _install:

Installation
============

Stetl up to and including version 1.3 only runs with Python 2 (2.7+).
Starting with Stetl v2.0 only Python 3 (3.6+) will be supported.
You may want to read :ref:`py3upgrade` when upgrading from a Stetl pre-v2 version.

Easiest is to first install the Stetl-dependencies (see below) and then
install and maintain Stetl on your system as a Python package (`pip` is preferred). ::

    (sudo) pip install stetl
    or
    easy_install stetl

Alternatively you can download Stetl from
Github: by cloning (preferred) or downloading: https://github.com/geopython/stetl/archive/master.zip
and then install locally  ::

	(sudo) python setup.py install

Try the examples first. This should work on Linuxes and Mac OSX.

Windows installation may be more involved depending on your local Python setup. Platform-specific
installations below.

You may also want to download the complete .tar.gz distro from PyPi:
https://pypi.python.org/pypi/Stetl . This includes the examples and tests.

**Docker**

Since version 1.0.9 Stetl also can be installed and run via `Docker <http://docker.com>`_. See
:ref:`install_docker` below.

**Debian/Ubuntu**

Thanks to Bas Couwenberg, work is performed to provide Stetl as Debian packages on both Debian and Ubuntu, see details:
https://packages.debian.org/search?keywords=stetl (Debian) and
https://launchpad.net/ubuntu/+source/python-stetl (Ubuntu, Xenial and later).
Stetl is split into 2 packages ``python-stetl``, the Python framework and ``stetl`` the command line utility.
NB the versions of these packages may be older than when installing Stetl via `pip` from PyPi
or directly from GitHub. Always check this first.

Dependencies
------------

Stetl depends on the following Python packages:

* GDAL (>=v2.4) bindings for Python
* psycopg2 (PostgreSQL client)
* lxml >=4.4.2
* Jinja2 templating
* Deprecated

``GDAL`` Python binding requires the native GDAL/OGR libs and tools to be installed.

``lxml`` http://lxml.de/installation.html requires the native (C) libraries:

* libxslt (required by lxml)
* libxml2 with Python bindings (required by lxml)

When using the ``Jinja2`` templating filter, ``Jinja2TemplatingFilter``, see http://jinja.pocoo.org:

* Python ``Jinja2`` package

``Deprecated`` is used to indicated deprecated functions and classes.

Platform-specific guidelines for dependencies follow next.

Linux
~~~~~

For Debian-based distro's like Ubuntu and Debian itself, most packages should be able to be installed via apt-get.

Tip: to get latest versions of GDAL and other Open Source geospatial software, best is
to add the `UbuntuGIS Repository <https://wiki.ubuntu.com/UbuntuGIS>`_.
Below a setup that works in Ubuntu 16.04 Xenial using Debian/Ubuntu packages. In some cases you may
choose to install the same packages via `pip3` to have more recent versions like for `lxml`.

More Linux Tips. See also:

* the install commands for Debian in the [Dockerfile](https://github.com/geopython/stetl/blob/master/Dockerfile).
* the install commands for Ubuntu in the [Travis file](https://github.com/geopython/stetl/blob/master/.travis.yml).

- Python dependencies: ::

	apt-get install python3-setuptools
	apt-get install python3-dev
	apt-get install python3-pip
	pip3 install --upgrade pip
	
- ``libxml2/libxslt`` libs are usually already installed. Together with Python ``lxml``, the total install for ``lxml`` is: ::

	apt-get install python3-libxml2
	apt-get install python-libxslt1
	apt-get install libxml2-dev libxslt1-dev lib32z1-dev
	apt-get install python3-lxml

- ``GDAL`` (http://gdal.org) version 2+ with Python bindings: ::

	# Add UbuntuGIS repo to get latest GDAL, at least v2 on Ubuntu 16.04, Xenial.
	add-apt-repository ppa:ubuntugis/ubuntugis-unstable
	apt-get update
	apt-get install gdal-bin
	gdalinfo --version
	# should show something like: GDAL 2.4.0, released 2019/03/04

	apt-get install python3-gdal

- the PostgreSQL client library for Python ``psycopg2``: ::

	apt-get install python3-psycopg2

- for ``Jinja2``: ::

	apt-get install python3-jinja2


Mac OSX
~~~~~~~

Dependencies can best be installed via `Homebrew <http://brew.sh/>`_.

Tip: sometimes installing GDAL Python bindings can be tricky as the
installed GDAL binaries must be compatible. To install the right version you may use: ::

	pip install GDAL==`gdalinfo --version | cut -d' ' -f2 | cut -d',' -f1`

Windows
~~~~~~~

Best is to install GDAL and python using the OSGeo4W Installer from http://trac.osgeo.org/osgeo4w.

* Download and run the OSGeo4W Installer
* Choose ``Advanced Install``
* On the ``Select Packages`` page expand ``Commandline_Utilities`` and Select from the list ``gdal`` and ``python``
* (``psycopg2``??)
* Install ``easy_install`` to allow you to install ``lxml``
* Download the ``ez_setup.py`` script
* Open the OSGeo4W Shell (Start > Programs > OSGeo4W > OSGeo4W > OSGeo4W Shell)
* Change to the folder that you downloaded ``ez_setup.py`` to (if you downloaded to C:\Temp then run cd C:\Temp)
* Install ``easy_install`` by running python ``ez_setup.py``
* To install ``lxml`` with easy_install run ``easy_install lxml``

Only Psycopg2 needs explicit installation. Many install via: http://www.stickpeople.com/projects/python/win-psycopg.
Once the above has been installed you should have everything required to run Stetl.

Alternatively you may use Portable GIS. Still you will need to manually install psycopg2.
See http://www.archaeogeek.com/portable-gis.html for details.

Test Installation
-----------------

If you installed via Python 'pip' you can check if you run the latest version ::

    stetl -h

You should get meaningful output like ::

	2013-09-16 18:25:12,093 util INFO running with lxml.etree, good!
	2013-09-16 18:25:12,100 util INFO running with cStringIO, fabulous!
	2013-09-16 18:25:12,122 main INFO Stetl version = 1.0.3
	usage: stetl [-h] -c  CONFIG_FILE [-s  CONFIG_SECTION] [-a  CONFIG_ARGS]

Especially check the Stetl version number. You can also use the `-v` or `--version` option for stetl.

Try running the examples when running with a downloaded distro. ::

	cd examples/basics
	./runall.sh

Look for any error messages in your output.

Run Unit Tests
--------------

You can run unit tests to completely verify your installation. First install some extra packages: ::

	pip install -r requirements-dev.txt

Then run the tests using `nose2`. ::

	nose2

.. _install_docker:

Install with Docker
-------------------

The fastest way to use Stetl is via `Docker <http://docker.com>`_. The Stetl Docker Image is lightweight,
compressed just over 100MB, based on a Debian "slim" Docker Image.

Your environment needs to be
setup to use Docker and probably you want to use some tooling like `Vagrant <https://www.vagrantup.com/>`_. The author uses
a combination of VirtualBox with Ubuntu and Vagrant on Mac OSX to run Docker, but this
is a bit out of scope here.

Assuming you have a working Docker environment, there are two ways to install Stetl with Docker:

* build a Docker image yourself using the Dockerfile in https://github.com/geopython/stetl/blob/master/Dockerfile
* use a prebuilt public Stetl Docker image from Docker Hub: https://hub.docker.com/r/geopython/stetl

When rebuilding you can add build arguments for your environment, defaults:  ::

	ARG TIMEZONE="Europe/Amsterdam"
	ARG LOCALE="en_US.UTF-8"
	ARG ADD_PYTHON_DEB_PACKAGES=""
	ARG ADD_PYTHON_PIP_PACKAGES=""

For example building with extra Python packages, building your local Docker Image: ::

	docker build --build-arg ADD_PYTHON_DEB_PACKAGES="python-requests python-tz" -t geopython/stetl:local .
	docker build --build-arg ADD_PYTHON_PIP_PACKAGES="scikit-learn==0.18 influxdb" -t geopython/stetl:local .

Or you may extend the Stetl Dockerfile with your own Dockerfile.

For running Stetl using Docker see  :ref:`run_docker`.
