Concepts and How Donation Store Works

Introduction

This document is intended for people who want to get a better understanding of how Donation Store works so they can customise it more but also be more confident in managing in on the server its running on. It is broken down into a few categories that describe most of the components involved.

The App

The main Donation Store application (the actual web app), is a Django based web application written in Python. It follows the standard MVC (or MVT in Django's case) design pattern that Django enforces. When deployed on your server, it runs in what is called a virtual environment.

A virtual environment can be thought of as a container for Python applications. They have their own set of dependencies which are installed only in that virtual environment. When Donation Store runs, it runs in it's virtual environment. That's why if you may not see dependencies that Donation Store requires if you check for them, if your outside the virtual environment. Virtual Environments are intended to separate the apps from other Python apps on the server so they all run in their own isolated environment with their own dependencies. It's mostly based on dependencies.

Dependencies and pip

As Donation Store is a Python application, it uses pip to install and manages its Python based dependencies. Like other languages, Python has it's own de facto standard package/dependency manager. For Node and JS apps its NPM or Yarn, for PHP apps its Composer, for Ruby its RubyGems and for Python its pip. While these aren't the only package managers for these languages, these are the most prevalent.

Like most package managers, pip operates in a similar way. It has a list of dependencies that are installed in the virtual environment. You can install new dependencies or remove existing dependencies. You can change where pip downloads it's packages from, however by default, it uses PyPI (the Python Package Index). If you want search the packages you can install with pip, you can visit the PyPI website

If you have read the installation guide for Donation Store, you will have noticed you have installed pip along with Python before starting the install.

Please don't uninstall any dependencies that Donation Store requires. To find an up to date list of the dependencies, you can check the requirements.txt file that was downloaded to the directory you installed Donation Store in

If you ever need to install dependencies using pip, please note that your virtual environment must be activated. This means that your installing the packages into that specific virtual environment, not globally. You can activate the virtual environment and install an example package using

source /home/donationstore/env/bin/activate
pip3 install mysqlclient

You will know that the virtual environment is activated, as the command line will be proceeded by (env).

Gunicorn and Nginx

In order for Donation Store to run, it requires a Web Server (Nginx) and a WSGI Server (Gunicorn).

When a request comes to your Donation Store webstore, it will come on either port 80 (HTTP) or preferably port 443 (HTTPS). Nginx will pick up on this request and start to figure out what it means. If the request is for static files, like CSS, HTML or Java Script files, Nginx will fetch that file and then send it back to the user. This means that the application itself doesn't get called on these requests, increasing performance. If however the request isn't for a static file, after Nginx figures that out, it will pass it to Gunicorn.

Gunicorn (Green Unicorn), is a WSGI (Web Server Gateway Interface) server that allows for server requests to be made to Python code (in Donation Store). So when Gunicorn passes the request to Donation Store, it executes whatever the request is asking and then sends a result back to Gunicorn, which is passed back to Nginx and back to the user who sent the request.

You don't need to know anything about these two to run and use Donation Store, but if you plan on managing it like you would with your server network, it's good to know how they work. For more information about Nginx, visit the Nginx Website and for more information about Gunicorn, visit the Gunicorn website..

Celery and RabbitMQ

Celery and RabbitMQ are used by Donation Store for its asynchronous task queue. Whenever Donation Store is asked to perform a task that might take a long time (like sending an email), it hands the task off the Celery to handle it. Celery runs in a separate process away from Donation Store. This allows Donation Store to remain responsive when these sort of tasks are being executed. Celery is a Python library that is installed by the Donation Store setup script into the virtual environment.

RabbitMQ is the messaging queue used to allow for Donation Store to pass tasks off to Celery. Unlike Celery, RabbitMQ is a standalone application and is installed by Donation Store onto the server itself.

While you don't need to know anything about these two services to use Donation Store, you can read more about RabbitMQ on their website and Celery on their website.

Both Celery and RabbitMQ are controlled and managed with Donation Store using Daemons. More on this in the next section.

Donation Store Daemons

Daemons are processes that are on their own, detached from a terminal window and run on their own. Originally with Donation Store you would have to run a command to start it, but then if you needed to stop it and you had closed that window, you would have to find the process by ID and then kill it manually and run the command again.

To fix this problem and also to ensure that all of Donation Store's dependent services are started, stopped and restarted when Donation Store is, Donation Store runs as a daemon by default.

Donation Store uses Systemd to create its services which can then be controlled by using start stop and restart commands. There are three main service files. This section will show each one and explain what each does.

Celery Service /etc/systemd/system/donationstore-celery.service

[Unit]
Description=Celery Service
PartOf=donationstore.service
After=donationstore.service

[Service]
User=donationstore
Group=www-data
WorkingDirectory=/home/donationstore/env
ExecStart=/home/donationstore/env/bin/celery -A controlpanel.tasks worker -l info

[Install]
WantedBy=donationstore.service

The Celery service is a child service to the main Donation Store service. It is used to manage Celery (as described above) and to ensure that when Donation Store is restart, it too is restarted. If you need to configure anything to do with the Celery service, you can change the ExecStart line, as it defines the command to run to actually start the service.

App Service /etc/systemd/system/donationstore-app.service

[Unit]
Description=donationstore daemon
PartOf=donationstore.service
After=donationstore.service

[Service]
User=donationstore
Group=www-data
WorkingDirectory=/home/donationstore/env
ExecStart=/home/donationstore/env/bin/gunicorn DonationStoreV2.wsgi:application --bind IP:8000 --error-logfile /var/log/donationstore/error.log --access-logfile /var/log/donationstore/access.log

[Install]
WantedBy=donationstore.service

The app service is used to manage the actual Donation Store application itself. It runs Gunicorn (as explained above), binds it to the machines IP address and directs the error logs to go to one file and the access logs to go to another.

Main Service /etc/systemd/system/donationstore.service

[Unit]
Description=donationstore daemon

[Service]
Type=oneshot
User=donationstore
ExecStart=/bin/true
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

The main service is used to link both the app and Celery services together so if one if the entire application is stopped, started or restarted, it manages all dependent services. It is probably best to not edit this service, but instead edit the child services or add your own that are part of this main service.

If you do intend on updating any of the services, you will have to perform a Daemon reload before the changes will come into affect. After doing that, you can then control your services as usual and the updates will be reflected

systemctl daemon-reload