debshots/doc/README.Installation on Dokku.md

7.2 KiB

Installing debshots on Dokku

What is Dokku?

Dokku is an open-source software that helps deploy web applications. Once installed and set up you essentially do a git push and Dokku will do its magic. It works similar to Heroku (a paid service) and even uses Heroku's buildpacks for that purpose. It can handle common web frameworks like Ruby-on-Rails. It has been used for screenshots.debian.net in production for several years.

Some of the features that make it a good choice:

  • healthchecks (will refuse to deploy if your new version is failing)
  • seamless deployment (no downtime while switching to the new version)
  • provides persistent volumes (where screenshots are stored)
  • provides a PostgreSQL database and tells the application how to access it through an automatic environment variable
  • handles Let's Encrypt certificates automatically
  • deploying a new application version is done simply by git push

Install Dokku

Grab a (virtual) server with at least 3 GB of RAM and 20 GB of SSD space. The application is currently run on a Hetzner server in Germany that costs less than 5€ per month.

Follow the installation instructions on https://dokku.com/

Install the Let's Encrypt plugin:

dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git

Set the email address as a contact for Let's Encrypt:

dokku letsencrypt:set screenshots-beta.debian.net email haas@debian.org

Create a new blank Dokku application:

dokku apps:create screenshots-beta.debian.net

Create a new blank PostgreSQL database and link it to the application:

dokku postgres:create screenshots_beta
dokku postgres:link screenshots_beta screenshots-beta.debian.net

Create a directory that will hold the screenshots' files:

mkdir -p /srv/www/screenshots-beta.debian.net/shrine

Mount it into the application's path:

dokku storage:mount screenshots-beta.debian.net /srv/www/screenshots-beta.debian.net/shrine:/app/public/shrine

By default the Nginx (that Dokku provides) will only allow uploads up to 1 MB. You may want to increase that limit:

dokku nginx:set screenshots-beta.debian.net client-max-body-size 10m
dokku proxy:build-config screenshots-beta.debian.net

If you want to add a message that gets printed on all pages you can set an environment variable for that purpose:

dokku config:set screenshots-beta.debian.net DEBSHOTS\_ALERT\_MESSAGE='This is a staging site. Uploads will not persist. Testing only.'

dokku config:set --no-restart screenshots-beta.debian.net SALSA_OAUTH_SECRET='c4c137fcbbcb5d9c6ff98a76bbf45bceee77ebe13645198359a0fa38cd8fb1d0' dokku config:set --no-restart screenshots-beta.debian.net SALSA_OAUTH_KEY='644440cd5595b3b7aea6e959ab1488e7026c2b5cf24faea8f9af5ce914377d52'

Make sure that you set the environment variables as described in README.SSO.md before you start the Rails application. Otherwise most single sign-on providers cannot work. You also need to get the https://salsa.debian.org/debsso-team/debsso/raw/master/update-debsso-ca script and run it in /etc/nginx to update the certficate and CRL used by the Debian single-sign-on provider.

The production configuration uses Nginx that decrypts HTTPS and either serves static assets (images, Javascript, CSS) itself or requests for dynamic content to the Rails application. You will need a SSL certificate for Nginx. Just get one from LetsEncrypt using the certbot tool.

Finally create an admin user for moderation:

Start a Rails console:

rails c -e production

Create a new user record:

user = User.create(realname: 'Christoph Haas', password: 'foobartest', email: 'email@christoph-haas.de')
user.save!

Cron jobs

Parse Nginx access logs to count the number of visits of each package:

RAILS_ENV=production bundle exec rake debshots:accesslog2visits[/var/log/nginx/cache-access.log.1]

As a cron job:

41 9 * * * cd $HOME/debshots ; RAILS_ENV=production /home/debshots/.rbenv/shims/bundle exec rake debshots:accesslog2visits[/var/log/nginx/cache-access.log.1]

Get the latest information on packages from the Debian repositories.

bundle exec rails debshots:update_from_deb_repos RAILS_ENV=production
bundle exec rails debshots:update_longdescription_from_deb_repos RAILS_ENV=production

Authentication

This web application tries to avoid to create yet another directory of user accounts. So it leverages other common web sites like StackExchange, Debian SSO (using browser/client certificates) or GitHub as single-sign-on services. It is still possible to create local user accounts for administrators but registration is disabled.

Anonymous uploads are possible but they have to be moderated by an administrator.

Authorization (not yet implemented)

A user can have one of these access levels.

Anonymous

Without authentication all user actions are moderated. Such users can upload screenshots or report existing/published screenshots as inappropriate. However a user with level 'admin' will have to approve that action.

Authenticated by single-sign-login (SSO)

Users can use different authentication services to login - like Github or StackExchange. Such users internally get an account with a random password created but they will keep logging in via SSO.

The first upload will have to be moderated. Subsequent uploads will be approved automatically.

Authenticated by Debian signle-sign-login (SSO)

If a user authenticates using a Debian SSO client certificate they will be able to upload screenshots without moderation. (Anohter idea was to restrict them to uploading only screenshots for the packages they maintain. But that may be reliable and is not implemented at this time.)

Administrator

Such a user has an internal user account. The database record has the 'admin' field set to 1. Admins can freely upload, moderate and delete screenshots. They can also read the log files.

Nginx during development

During development it is advised to run Nginx to proxy ports 3000 (HTTP) and 3001 (HTTPS) to the rails application.

cd nginx
nginx -c `pwd`/nginx.conf

The SSL certitificate is just a self-signed certificate with the common name "localhost". It was generated using:

openssl req -x509 -newkey 4096 -nodes -keyout localhost.key -out localhost.crt -days 3650

Run "rails s" in another window. That will use the built-in Ruby 'Puma' application server that can also be used in production. It requires less configuration than Phusion Passenger but may be slower in regard to performance and multi-threading / multi-core support.

If you get "400 Bad Request / The SSL certificate error" in the browser then please check the error log (/tmp/error*). Once cause is that the CRL of the Debian SSO has expired and needs to be updated.

Tidy up the screenshot data:

bundle exec rake debshots:remove_broken_screenshots

bundle exec rake debshots:remove_duplicate_images

Frequent update of Debian SSO certificate

Run /etc/nginx/update-sso-ca regularly to update the CA certificate and the CRL. The CRL (certificate revocation list) is only valid for three days. So make sure you have a daily cron job for the updates. Do not forget to restart Nginx afterwards.

Otherwise you will end up with this error message:

400 Bad Request
The SSL certificate error
nginx