Added SSL notes to mail/git/web server build log.

[website_subgeniuskitty.com] / data / notes / mail_web_git_server.md

# Overview #

These notes cover the creation of a server, hosted with Linode, running Debian
Linux, offering the following services:

  - Mail server

      - SMTP via postfix

      - IMAP via dovecot

      - Realtime blacklists for SPAM rejection

      - MySQL for virtual domain and user management.

  - Web server

      - Apache as central HTTP server with multiple vhosts.

      - [CMless](http://git.subgeniuskitty.com/cmless/.git) as CGI for content management

      - ACME for automated SSL certificate management

  - Git server

      - SSH-based, authenticated read-write access to all git repositories

      - Anonymous read-only access to a subset of git repositories via:

          - Customized [gitweb](http://git.subgeniuskitty.com/gitweb-sgk/.git) for
            GUI git browsing with syntax highlighting, diffs, etc

          - Git-daemon for cloning repositories via the `git://` protocol

These notes are a high-level checklist for my reference rather than a
step-by-step installation guide for the public. That means they make no attempt
to explain all options at each step, rather that they mention only the options
I use on my servers. It also means they use my domains, my file system paths,
etc in the examples.


# TODO List #

  - Take a snapshot on Linode's backup service once the basic services are
    operational.

  - Migrate mail server. Delete old linode vserver after downloading a disk image.

  - Finish this documentation.

  - Improve CSS on gitweb, especially for displaying READMEs.

  - Add some form of web logfile viewing.


# Basic Configuration #


## General Information ##

**Name:** SGK-Main-2020

**OS:** Debian 10

**Creation Date:** 2020-11-01

**Filesystem Points of Interest:**

  - `/srv/apache_vhosts`: Contains websites hosted by Apache2. See
    `/etc/apache2/sites_available` for vhost configurations.

  - `/srv/git`: Master location for bare git repositories. All are private.

  - `/srv/gitweb_cache`: Contains checked out copies of git repositories from
    `/srv/git`, publicly visible via `gitweb` and cloneable via `git-daemon`.


## Preparation ##

Setup DNS entries for `subgeniuskitty.com` and `logicavalanche.com` through
Linode. Remember to do IPv4 and IPv6 entries for the bare domain, `www`,
`mail`, and `git`.

Create a new Debian 10 on Linode and update the system with `apt-get update &&
apt-get upgrade`.

Add a user via `adduser ataylor` and following the prompts. Edit
`/etc/ssh/sshd_config` to set `PermitRootLogin: no` and restart SSH. For both
the `ataylor` user and `root`, add the line `set mouse=` to `~/.vimrc` in order
to disable mouse support in `vim`, allowing normal mark-and-paste in the
terminal.

Install useful packages:

    apt-get install net-tools screen bzip2 zip


# Web Server #


## HTTP: Apache2 ##

Install Apache2.

    apt-get install apache2

If not already defined elsewhere, add a `ServerName 127.0.0.1` entry to the
bottom of `/etc/apache2/apache2.conf`, or whatever is appropriate.

Since we use `/srv` instead of `/var/www`, edit `/etc/apache2/apache2.conf` to
comment out the `<Directory ...>` entry for `/var/www` and replace it with
this:

    <Directory /srv/>
            Options Indexes FollowSymLinks
            AllowOverride None
            Require all granted
    </Directory>

Make and edit the file `/srv/apache_vhosts/default/index.html` with (rewritten)
contents:

    <p>Invalid VHost</p>
    <p>Contact user @at@ domain .dot. com</p>

Ensure everything under `/srv/apache_vhosts`, including that directory itself,
is owned recursively by `www-data:www-data`.

Edit `/etc/apache2/sites-available/000-default.conf` and
`/etc/apache2/sites-available/000-default.conf`, changing references for the
default sites from `/var/www/...` to `/srv/apache_vhosts/...` as necessary.

Reload Apache2 with `systemctl reload apache2` and check status with `systemctl
status apache2`.


### SSL ###

Install certbot and generate a key for its use.

    apt-get install certbot
    openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Create `/etc/apache2/conf-available/ssl-params.conf` with the following
contents.

    SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite          ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
    SSLHonorCipherOrder     off
    SSLSessionTickets       off
    
    SSLUseStapling On
    SSLStaplingCache "shmcb:logs/ssl_stapling(32768)"
    
    Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    Header always set X-Frame-Options SAMEORIGIN
    Header always set X-Content-Type-Options nosniff
    
    SSLOpenSSLConfCmd DHParameters "/etc/ssl/certs/dhparam.pem"

Enable the new configuration and required mods, then restart Apache2.

    a2enconf ssl-params
    a2enmod ssl
    a2enmod headers
    systemctl restart apache2

Retrieve an initial certificate with the following command, modified to match
the desired webroot and server names.

<http://subgeniuskitty.com> and <http://logicavalanche.com>:

    certbot certonly --agree-tos --email webmaster@subgeniuskitty.com --webroot -w /srv/apache_vhosts/subgeniuskitty.com/site/data/ -d subgeniuskitty.com -d www.subgeniuskitty.com

<http://archive.subgeniuskitty.com> and <http://git.subgeniuskitty.com>:

    certbot certonly --agree-tos --email webmaster@subgeniuskitty.com --webroot -w /srv/apache_vhosts/archive.subgeniuskitty.com/ -d archive.subgeniuskitty.com

Edit `/etc/apache2/sites-available/subgeniuskitty.com`, adding the following
`VirtualHost` definition that mostly copies the non-SSL entry.

    <VirtualHost *:443>
            SSLEngine on
            SSLCertificateFile /etc/letsencrypt/live/subgeniuskitty.com/fullchain.pem
            SSLCertificateKeyFile /etc/letsencrypt/live/subgeniuskitty.com/privkey.pem
   
            ...copy of vhost definition for host *:80... 
    </VirtualHost>

Edit `/etc/cron.d/certbot` and append `--renew-hook "systemctl reload apache2"`
to the certbot invokation.

Test with `certbot renew --dry-run`.

Repeat the process for any other sites hosted on this server.

Backup the `/etc/letsencrypt` folder off-server periodically.


## Basic Website ##

Using <http://archive.subgeniuskitty.com> as an example of a basic website,
create an Apache2 vhost configuration file at
`/etc/apache2/sites-available/archive.subgeniuskitty.com.conf`.

    <VirtualHost *:80>
            DocumentRoot "/srv/apache_vhosts/archive.subgeniuskitty.com"
            ServerName archive.subgeniuskitty.com
            ServerAdmin webmaster@subgeniuskitty.com
            ErrorLog /var/log/apache2/error_log.archive.subgeniuskitty.com
            CustomLog /var/log/apache2/access_log.archive.subgeniuskitty.com combined
            <Directory "/srv/apache_vhosts/archive.subgeniuskitty.com">
                    Options +FollowSymLinks
                    AllowOverride None
                    Require all granted
            </Directory>
            <Directory "/srv/apache_vhosts/archive.subgeniuskitty.com/sites">
                    Options +FollowSymLinks +Indexes
                    AllowOverride None
                    Require all granted
            </Directory>
    </VirtualHost>

Make the directory `/srv/apache_vhosts/archive.subgeniuskitty.com`, move your
data into it, and ensure everything is owned by `www-data:www-data`. 

Enable the vhost with `a2ensite archive.subgeniuskitty.com` and `systemctl
reload apache2`.


## CMless Website ##

Enable `mod_rewrite` and either `mod_cgi` or `mod_cgid` as appropriate with
these commands.

    a2enmod rewrite
    a2enmod cgid

Install `discount` to convert Markdown to HTML.

    apt-get install discount

Create `/etc/apache2/sites-available/subgeniuskitty.com.conf`.

    <VirtualHost *:80>
            DocumentRoot "/srv/apache_vhosts/subgeniuskitty.com"
            ServerName subgeniuskitty.com
            ServerAlias www.subgeniuskitty.com
            ServerAdmin webmaster@subgeniuskitty.com
            ErrorLog /var/log/apache2/error_log.subgeniuskitty.com
            CustomLog /var/log/apache2/access_log.subgeniuskitty.com combined
            AddHandler cgi-script .py
            <Directory "/srv/apache_vhosts/subgeniuskitty.com">
                    Options -ExecCGI -Indexes
                    AllowOverride None
                    Require all granted
            </Directory>
            <Directory "/srv/apache_vhosts/subgeniuskitty.com/bin">
                    Options ExecCGI
                    AllowOverride None
                    Require all granted
            </Directory>
            RewriteEngine On
            RewriteRule (.*) /srv/apache_vhosts/subgeniuskitty.com/site/data/$1
            RewriteCond %{REQUEST_FILENAME} !-f
            RewriteRule .* /srv/apache_vhosts/subgeniuskitty.com/bin/cmless.py
    </VirtualHost>

Enable the site with `a2ensite subgeniuskitty.com`.

Clone a copy of [CMless](http://git.subgeniuskitty.com/cmless/.git) into
`/srv/apache_vhosts/subgeniuskitty.com` and ensure everything is owned by
`www-data:www-data`.

Clone a copy of [the website
data](http://git.subgeniuskitty.com/website_subgeniuskitty.com/.git) into
`/srv/apache_vhosts/subgeniuskitty.com/site`. Verify it is owned by
`ataylor:ataylor` (but still readable by all) so we can update the site
remotely with a simple script like this:

    #!/usr/local/bin/bash
    #
    # Usage: No cmdline arguments
    # Update content of www.subgeniuskitty.com to the latest version.
    
    ssh ataylor@git.subgeniuskitty.com "cd /srv/apache_vhosts/subgeniuskitty.com/site && git pull"

Reload Apache's configuration with `systemctl reload apache2` and test access
to the website.

Repeat this process for <http://logicavalanche.com>.


# Git Server #

The git server provides read-write access to a private collection of bare
repositories located at `/srv/git` via SSH. It also provides read-only access
to a public collection of normal repositories located at `/srv/gitweb_cache`
via <http://git.subgeniuskitty.com/repo_name> through gitweb and via
[git://git.subgeniuskitty.com/repo_name](git://git.subgeniuskitty.com/repo_name)
through `git-daemon`. 


## Read/Write: SSH ##

Install git with `apt-get install git`.

On my workstation, generate an SSH key with `ssh-keygen -t rsa`.

On the server, as user `ataylor`:

    mkdir ~/.ssh
    chmod 700 ~/.ssh
    touch ~/.ssh/authorized_keys
    chmod 600 ~/.ssh/authorized_keys

Then `cat` the public SSH key from the workstation to the server, appending it
onto `~/.ssh/authorized_keys`.

Verify ability to login using new certificate.

Create a directory `/srv/git` owned by `ataylor:ataylor`. This will hold bare
git repositories and act as the central private store for SGK git repos.

From the workstation, we can create a new bare repository on the server. For
example, packed up in a simple script:

    #!/usr/local/bin/bash
    #
    # Usage: sgkgit-new-repo project_name
    # Setup a repository on the SGK git server.
    
    if [ "$#" -ne 1 ]; then
            echo "Must specify repo name as only parameter."
            exit 2
    fi
    
    ssh ataylor@git.subgeniuskitty.com "git init --bare /srv/git/$@"

We then set the remote of an existing repository to the new bare repository.
Again as a script:

    #!/usr/local/bin/bash
    #
    # Usage: sgkgit-set-origin project_name
    # Sets remote to the correct path for 'project_name' on the SGK git server.
    
    if [ "$#" -ne 1 ]; then
            echo "Must specify repo name as only parameter."
            exit 2
    fi
    
    git remote remove origin
    git remote add origin ataylor@git.subgeniuskitty.com:/srv/git/$@
    echo "Remember to make the first push with \"git push --set-upstream origin master\"."

After the first push to the bare repository on the server, simply `git push`
and `git pull` as normal.

You can also list the repositories currently on the server with simple SSH commands.

    #!/usr/local/bin/bash
    #
    # Usage: No cmdline arguments.
    # List all remote repos available on SGK git server.
    
    ssh ataylor@git.subgeniuskitty.com "ls -lt /srv/git/"

Clone one of these repositories, with remote correspondingly pre-set.

    #!/usr/local/bin/bash
    #
    # Usage: sgkgit-checkout project_name
    # Clone a local copy of repo "project_name" from SGK git server.
    
    if [ "$#" -ne 1 ]; then
            echo "Must specify repo name as only parameter."
            exit 2
    fi
    
    git clone ataylor@git.subgeniuskitty.com:/srv/git/$@


## Read-Only: Gitweb ##

Enable `mod_rewrite` and either `mod_cgi` or `mod_cgid` as appropriate with
these commands.

    a2enmod rewrite
    a2enmod cgid

Install `discount` to convert Markdown to HTML, `highlight` for syntax
highlighting, and `gitweb` to pull in any dependencies.

    apt-get install discount highlight gitweb

Create `/etc/apache2/sites-available/git.subgeniuskitty.com.conf`.

    <VirtualHost *:80>
            ServerName git.subgeniuskitty.com
            ServerAdmin webmaster@subgeniuskitty.com
    
            DocumentRoot "/srv/apache_vhosts/git.subgeniuskitty.com"
    
            ErrorLog /var/log/apache2/error_log.git.subgeniuskitty.com
            CustomLog /var/log/apache2/access_log.git.subgeniuskitty.com combined
    
            <Directory "/srv/apache_vhosts/git.subgeniuskitty.com">
                    Options +FollowSymLinks +ExecCGI
                    AllowOverride None
                    Require all granted
                    AddHandler cgi-script .cgi
                    DirectoryIndex gitweb.cgi
                    RewriteEngine On
                    RewriteCond %{REQUEST_FILENAME} !-f
                    RewriteCond %{REQUEST_FILENAME} !-d
                    RewriteRule ^.* /gitweb.cgi/$0 [L,PT]
            </Directory>
    </VirtualHost>

Enable the site with `a2ensite git.subgeniuskitty.com`.

Clone a copy of [the SGK gitweb fork](http://git.subgeniuskitty.com/gitweb-sgk/.git)
into `/srv/apache_vhosts/git.subgeniuskitty.com` and ensure everything is owned
by `ataylor:ataylor` (but world-readable!) so we can update via SSH with a script like this.

    #!/usr/local/bin/bash
    #
    # Usage: No cmdline arguments
    # Update git.subgeniuskitty.com to the latest version of forked gitweb repo.
    
    ssh ataylor@git.subgeniuskitty.com "cd /srv/apache_vhosts/git.subgeniuskitty.com && git pull"

This fork makes a few changes to gitweb, displaying READMEs by default, etc.
Read the gitweb project's `README.md` for more details.

Create a gitweb config file at `/etc/gitweb.conf`. Note that we are adding a
`clone url` to the toolbar with the address of our `git-daemon` server.
Remember to set that up.

    $site_name = "git.subgeniuskitty.com";
    @git_base_url_list = ("git://git.subgeniuskitty.com");
    $projectroot = "/srv/gitweb_cache";
    $git_temp = "/tmp";
    
    @stylesheets = ("static/gitweb.css");
    $javascript = "static/gitweb.js";
    $logo = "static/sgk-logo.png";
    $favicon = "static/git-favicon.png";
    
    # git-diff-tree(1) options to use for generated patches
    @diff_opts = ();
    
    # Enable PATH_INFO so the server can produce URLs of the
    # form: http://git.hokietux.net/project.git/xxx/xxx
    # This allows for pretty URLs *within* the Git repository,
    # also needs the Apache rewrite rules for full effect.
    $feature{'pathinfo'}{'default'} = [1];
    
    # HTML text to include as home page header.
    $home_text = "indextext.html";
    
    # Add a toolbar option with the 'git clone url'.
    $feature{'actions'}{'default'} = [('clone url', 'git://git.subgeniuskitty.com/%n', 'summary')];
    
    # Category name is read from .git/category, in the same manner as .git/description.
    $projects_list_group_categories = 1;
    $project_list_default_category = "misc";
    
    # Needed for displaying README files.
    $prevent_xss = 0;

    # Enable syntax highlighting.    
    $feature{'highlight'}{'default'} = [1];

    ################################################################################
    
    # Enable blame, pickaxe search, snapshop, search, and grep
    # support, but still allow individual projects to turn them off.
    # These are features that users can use to interact with your Git trees. They
    # consume some CPU whenever a user uses them, so you can turn them off if you
    # need to.  Note that the 'override' option means that you can override the
    # setting on a per-repository basis.
    $feature{'blame'}{'default'} = [1];
    $feature{'blame'}{'override'} = [1];
    
    $feature{'pickaxe'}{'default'} = [1];
    $feature{'pickaxe'}{'override'} = [1];
    
    $feature{'snapshot'}{'default'} = [1];
    $feature{'snapshot'}{'override'} = [1];
    
    $feature{'search'}{'default'} = [1];
    
    $feature{'grep'}{'default'} = [1];
    $feature{'grep'}{'override'} = [1];

Create the directory `/srv/gitweb_cache`. It should be readable by the web
server (`www-data`), but owned by `ataylor:ataylor` so that simple scripts like
the following can make repositories public/private.

    #!/usr/local/bin/bash
    #
    # Usage: sgkgit-make-public project_name
    # Make a repo accessible through gitweb and git-daemon.
    
    if [ "$#" -ne 1 ]; then
            echo "Must specify repo name as only parameter."
            exit 2
    fi
    
    read -p "Enter a category name: " category
    read -p "Enter a description: " description
    
    echo "You entered:"
    printf "\tCategory: $category \n"
    printf "\tDescription: $description \n"
    read -p "Confirm (y/n)?: " confirm
    
    if [ "$confirm" == "y" ]; then
            ssh ataylor@git.subgeniuskitty.com "cd /srv/gitweb_cache && rm -rf $@ && git clone /srv/git/$@"
            ssh ataylor@git.subgeniuskitty.com "cd /srv/git && \
                    printf '#!/usr/bin/bash\ncd /srv/gitweb_cache/$@\ngit --git-dir=.git pull\n' > \
                    /srv/git/$@/hooks/post-update && chmod +x /srv/git/$@/hooks/post-update"
            ssh ataylor@git.subgeniuskitty.com "echo \"$category\" > /srv/gitweb_cache/$@/.git/category"
            ssh ataylor@git.subgeniuskitty.com "echo \"$description\" > /srv/gitweb_cache/$@/.git/description"
    fi

In addition to cloning the git repo into `/srv/gitweb_cache` from the private,
SSH-only collection of repositories in `/srv/git`, this also creates a
`post-update` hook in the bare repo in `/srv/git` to ensure that gitweb's cache
is updated every time a push is made via SSH to the private repo.

This script also sets the `.git/description` and `.git/category` files needed
by gitweb for displays like the project summary page. These are not under
version control and may be directly edited to change what is displayed in
gitweb.

We can remove the public cache, making the repo private-only as shown in this
script.

    #!/usr/local/bin/bash
    #
    # Usage: sgkgit-make-private project_name
    # Make a repo inaccessible through gitweb and git-daemon.
    
    if [ "$#" -ne 1 ]; then
            echo "Must specify repo name as only parameter."
            exit 2
    fi
    
    ssh ataylor@git.subgeniuskitty.com "rm -rf /srv/gitweb_cache/$@ && rm /srv/git/$@/hooks/post-update"

Any repos made private/public via this method are immediately reflected in
`gitweb` and `git-daemon`.


## Read-Only: Git Daemon ##

Since `git` was already installed, simply create a new service profile in
`/etc/systemd/system/git-daemon.service`.

    [Unit]
    Description=Start Git Daemon
    
    [Service]
    ExecStart=/usr/bin/git daemon --export-all --reuseaddr --base-path=/srv/gitweb_cache/ /srv/gitweb_cache/
    
    Restart=always
    RestartSec=500ms
    
    StandardOutput=syslog
    StandardError=syslog
    SyslogIdentifier=git-daemon
    
    User=ataylor
    Group=ataylor
    
    [Install]
    WantedBy=multi-user.target

Then start the service with `systemctl daemon-reload` and `systemctl start
git-daemon`. Verify functionality before enabling daemon-autostart with
`systemctl enable git-daemon`.

The `--export-all` flag tells `git-daemon` to export all repositories located
at the specified path, regardless of the presence (or lack) of the file
`.git/git-daemon-export-ok` in the individual repository. We do this because
only public repos are checked out to this folder. All private repos are kept
entirely elsewhere.

The directory `/srv/gitweb_cache` should have already been created when
installing gitweb. If not, go read those instructions.

A repository located at `/srv/gitweb_cache/repo_name` may be cloned through
`git-daemon` at
[git://git.subgeniuskitty.com/repo_name](git://git.subgeniuskitty.com/repo_name)
with `git clone`.


# Mail Server #

TODO

TODO

TODO

TODO

TODO

TODO

TODO

TODO