Merge branch 'main' of github.com:romkey/pdxhackerspace-hackstack

Author: romkey

README.md

@@ -2,6 +2,20 @@
 
 This repo configures a set of services that PDX Hackerspace uses to provide services for its members and for infrastructure, administration and yes, fun and entertainment.
 
+Hackstack uses Docker (with compose) to 
+
+Hackstack's priorities are:
+- consistency - all applications follow similar conventions in how
+  they are configured, managed, store data and are backed up
+- ease of management - Hackstack is meant to be manageable by users
+  with basic Linux knowledge and without advanced Docker or
+  virtualization environment experience
+- low overhead - Hackstack is intended to minimize overhead when
+  possible - it should be possible to operate many Hackstack services
+  on small computers like a Raspberry Pi
+- ease of recovery - Hackstack is designed to make disaster recovery
+  easy when possible
+
 - [Philosphy and Design](docs/design.md)
 - [Examples](docs/examples.md)
 - [Installation](docs/installation.md)
@@ -12,5 +26,8 @@ This repo configures a set of services that PDX Hackerspace uses to provide serv
 
 ## Contributing
 
+Please see the [Contributing](docs/contributing.md) guide.
+
 ## License
 
+Hackstack is released under the MIT License.

docs/contributing.md

@@ -6,9 +6,31 @@ We don't intend it as any kind of ubiquitous or universal guide to server softwa
 
 Hackstack applications all follow a set of conventions that dictate how they store data, publish network ports, use databases, are configured and backed up, and interact with one another. By following conventions the system stays manageable and organized; we know where everything lives and how it works, instead of being a haphazard collection of Docker and compose files that all do everything differently.
 
+## tldr;
+
+1. All applications have their own Docker Compose files
+
+2. Each application lives in a directory in `/opt/docker`
+
+3. Each application's volumes are filesystem binds that are stored in a directory named for the application `/opt/lib`, `/opt/logs` or `/opt/run`
+
+4. Unless absolutely necessary, applications do not use host networking and do not publish ports. Whenever possible, applications are exposed via `nginx-proxy-manager`
+
+5. Applications are isolated on Docker networks that connect functional groups
+
+6. Unless absolutely necessary, applications use the shared instances of Postgresql and Mariadb and do not spin up their own instances of them. However, applications that require Redis do use their own instance of that.
+
+7. Environment variables are stored in `.env`, not in `docker-compose.yml`. Applications should provide a `.env.example` with secrets redacted.
+
+8. The environment variable `BACKUP_DATABASE_URLS` in a `.env` file will allow dumps of live databases to be backed up automatically.
+
+9. Applications will be automatically updated by Watchtower. For locally built applications or applications that shouldn't be updated, add the label `com.centurylinklabs.watchtower.enable=false` to the compose file.
+
+10. Applications that follow these conventions will have their files and databases backed up automatically.
+
 ## Docker Compose
 
-All services are managed by Docker Compose. The service should include a docker-compose.yml file in a directory at the root of the repository.
+All services are managed by Docker Compose. The service should include a `docker-compose.yml` file in a directory at the root of the repository.
 
 As a rule we try to allow users to customize the installation as much as possible without modifying the compose file. Obviously this isn't always possible but we strongly prefer doing so when possible. This allows the user a change to potentially be able to simply pull new versions of the hackstack to gain new applications.
 
@@ -17,7 +39,7 @@ As a rule we try to allow users to customize the installation as much as possibl
 Docker Compose can specify external directories or Docker volumes to be mounted inside the container. We use a specific organization for volumes (`APPNAME` is the name of the application):
 
 - any configuration data will be stored in a directory called `config` in the directory containing the `docker-compose` file
-- any runtime data, state information, sqlite3 databases - anything that maintains the state of the application and which must be preserved across runs - will be stored in `../../lib/APPNAME`. Applications which mingle configuration and run-time data in a way that makes it difficult to isolate them may store configuration data in this directory as well.
+- any runtime data, state information, Sqlite3 databases - anything that maintains the state of the application and which must be preserved across runs - will be stored in `../../lib/APPNAME`. Applications which mingle configuration and run-time data in a way that makes it difficult to isolate them may store configuration data in this directory as well.
 - log files go in `../../log/APPNAME`
 - transient runtime files, like caches or FIFOs, go in `../../run/APPNAME`
 

docs/design.md

@@ -1,5 +1,54 @@
 # Philosophy
 
+Hackstack's priorities are:
+- consistency
+- ease of maintenance
+- ease of disaster recovery
+
+Hackstack is designed specifically for a hacker/makerspace type of
+environment and recognizes that the needs and resources of such an
+environment are very different from those of a business selling
+web-based services.
+
+Hacker/Makerspaces are generally smallish, low resourced
+organizations. They:
+- have memberships (users) which more likely number in the hundreds
+  rather than hundreds of  thousands to millions
+- offer services which may reasonably be only accessible on-site
+- most likely do not have the resources to buy modern server hardware
+  (and in fact may be running on scavenged or reclaimed hardware)
+- most likely do not have the resources to have an on call tech
+  support or IT staff but rather a few volunteers that keep things
+  chugging along
+
+### Anti-goals
+
+Recognizing these conditions helps us understand some
+anti-goals. These spaces do not need services that:
+- scale
+- migrate easily or transparently between nodes
+and they do not need the overhead of running and maintaining systems
+that offer those as primary features.
+
+### Hardware
+
+Because Hacker/Makerspaces often use older hardware they are subject
+to a higher likelihood of hardware failure than businesses would
+generally anticipate while having fewer resources to manage
+failures. This dictates that overall design favor ease of disaster
+recovery and keeping resource requirements minimal.
+
+### Consistency
+
+There is little consistency to how applications are offered online
+using Docker compose files. Naming, port allocation, filesystem use
+all vary tremendously. From a maintenance perspective having to figure
+out how each individual application is set up can be a collosal
+headache when they're all different. Therefore Hackstack provides a
+set of guidelines as to how applications will be installed, similar to
+how Linux has done for decades. Once a maintainer is familiar with
+them they can easily inspect and manage any Hackstack application.
+
 ## Underlying System
 
 Keep the underlying system hosting the servers as clean and unpolluted

docs/installation.md

@@ -1,4 +1,35 @@
 ## Installation
 
-First, install [Docker Engine](), [Docker CLI]() and [Docker Compose]() on your system.
+Hackstack is designed to require minimal additional software installed
+on a stock Linux server. Whenever possible dependencies are isolated
+within Docker containers. PDX Hackerspace runs Hackstack on a Debian
+installation. Unless you absolutely require a GUI we recommend saving
+the resources that it would take and just running a "server"-stye
+system with no graphic user interface.
 
+First, install [Docker CLI](https://docs.docker.com/engine/install/)
+and if needed Docker Compose on your system.
+
+Then make sure `git` is installed. On Debian-style systems you can do
+this with:
+```
+sudo apt update
+sudo apt install git
+```
+
+Then clone the Hackstack repo, preferably into `/opt/docker`:
+```
+git clone https://github.com/romkey/pdxhackerspace-hackstack
+```
+
+At that point you can begin spinning up services. Start with
+foundational services like `dnsmasq`, `postgresql` and `mariadb` if
+you need them. For each service you'll want to edit its `.env` file if
+there is one and then bring up its Docker container. For instance, for
+`calibre-web` you would:
+```
+cd /opt/docker/calibre
+cp .env.example .env
+vi .env
+docker compose up -d
+```