https://raw.githubusercontent.com/stakkr-org/stakkr/master/docs/stakkr-logo.png

Overview

https://scrutinizer-ci.com/g/stakkr-org/stakkr/badges/quality-score.png?b=master https://api.travis-ci.com/stakkr-org/stakkr.svg?branch=master https://img.shields.io/pypi/l/stakkr.svg

Stakkr is a a development tool, based on docker-compose, to easily use a stack of services. For example, if you need a LAMP environment for your developments, then stakkr is the right tool.

Via a very simple configuration file you can setup the required services and let stakkr link and start everything for you.

Why stakkr and not docker-compose ? * To avoid searching for the right image, * To stop fighting with permissions, * To do lot of stuff without reading the full docker documentation, * Also because you need to share your configuration with your team, so you want a portable tool, * Finally, you want a very simple tool

What does that do exactly ?

If you have heard of Docker, you know that when you need to build a full environment with multiple services that are linked, you either have to do everything manually or use docker-compose. The second solution is the best but it implies that you need, for each environment, to change your parameters, choose your images, learn the docker-compose command line tool, etc … In brief, it’s not very flexible and hard to learn.

Stakkr will help you, via a very simple configuration file and a predefined list of services (that can be extended with aliases and custom services) to build a complete environment. Plus, to control it in command line. It makes use of docker easy.

Last, but not the least, it’s highly configurable and each service mounts a volume to have a persistence of data. You can even, if you want, add more directives on some services (change the php.ini for example and choose your versions (PHP 5.6 or 7.1 or 7.3 or anything else).

Examples

You can combine services as you want to have :

  • A Dev LAMP stack (Nginx + Postgres 13 + PHP 8.0 with xdebug and xhprof) … and if suddenly you want to test your code with PHP 7.0, change it in stakkr.yml, restart, it’s done !

  • Or Apache 2.2 + PHP 5.6 + MySQL 5.5 for a legacy environment

  • Or a ready made symfony stack (with the project initialized!)

  • etc…

Installation

Docker

You must have Docker installed on your computer. Pick the right version for your OS from https://docs.docker.com/get-docker/

Warning

Also, to use docker for Linux as a normal user, you need to add your user to the docker group (see the documentation)

Stakkr

Stakkr is installable via pip, system-wide (or for a specific user). It’s clean and detects automatically the config file (stakkr.yml) presence in a directory or parent directory to execute commands. stakkr.yml acts like docker-compose.yml with super power (plus, it is super simple !).

It means that a directory with a stakkr.yml is a complete stack.

Installation and configuration

To install stakkr, you need python 3 and docker.

  • For Ubuntu, install python3 with

sudo apt -y install python3-pip python3-setuptools python3-virtualenv python3-wheel. You can download Docker from : https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/

with brew install python3. Then docker from https://docs.docker.com/docker-for-mac/install/. **Don’t forget to allow docker to access some of your directories (where stakkr is installed, such as /usr/local/lib/python3.8/site-packages and where you put your projects).

from https://docs.docker.com/docker-for-windows/install/

Then :

$ sudo python3 -m pip --no-cache-dir install stakkr

###
# If you want a beta / rc version append "--pre" at the end
# If you don't want to install it system wide, you can remove "sudo" and append "--user" at the end
# but to make it work, you'll need to change your $PATH variable
# That option is recommended when you know how to do and especially for MacOs
# For windows just run : python -m pip --no-cache-dir install stakkr
###

$ mkdir my_project
$ cd my_project

##
# RECIPE is optional but better to start from an existing one !
# Examples (see below for the current list) :
#   stakkr-init wordpress
#   stakkr-init symfony
##

$ stakkr-init {RECIPE}

stakkr-init copies some templates and creates base directories to work.

Recipes

Currently, the following recipes are available (see https://github.com/stakkr-org/stakkr/tree/master/stakkr/static/recipes) :

  • LAMP : PHP (latest) + Apache (2.4) + MySQL (5.7) + stakkr composer to use composer and stakkr mysql to use mysql.

  • LEMP : PHP (latest) + Nginx (latest alpine) + MySQL (5.7) + stakkr composer to use composer and stakkr mysql to use mysql.

  • LEPP : PHP (latest) + Nginx (latest alpine) + PostgreSQL (latest) + stakkr composer to use composer.

  • Symfony : PHP (7.2) + Nginx (latest alpine) + stakkr composer to use composer + Symfony Framework pre-installed.

  • Wordpress : PHP (7.2) + Apache (2.4) + MySQL (5.7) + stakkr wp to use wp-cli + Wordpress pre-installed.

Use stakkr from a docker dind (Docker-In-Docker) image

You can else use the ready-to-go Docker Image edyan/stakkr to test the tool.

Be careful that uid and gid of stakkr user into the container won’t be the same than for your user. The volume will contain files with other permissions.

$ mkdir ~/my_project
$ docker run -p 80:80 -p 443:443 -v ~/my_project:/home/stakkr/app -d --privileged --rm --name stakkr-dev stakkr/stakkr
$ docker exec -ti stakkr-dev ash
$ chown -R stakkr:stakkr /home/stakkr
$ su - stakkr
# Create a symfony project from a recipe
$ cd ~/app
$ stakkr-init symfony

Now open http://nginx.app.localhost from your browser.

Development

Setup your env

To develop stakkr, you have to create a virtual environment :

$ git clone git@github.com:stakkr-org/stakkr.git stakkr
$ cd stakkr
$ python3 -m venv venv_stakkr
$ source venv_stakkr/bin/activate
# For Windows use "venv_stakkr\Scripts\activate"

Then install stakkr and its dependencies :

$ python -m pip install --upgrade pip setuptools wheel
$ python -m pip install -e .
$ python -m pip install -r requirements-dev.txt
$ stakkr-init
Run Tests
$ py.test
Generate that doc
$ cd docs
$ sphinx-autobuild . _build_html
Try stakkr from a docker in docker container

The code below starts a dind container and init a symfony app :

# From the host
$ docker run -d -v /var/run/docker.sock:/var/run/docker.sock --privileged --rm --name stakkr-test docker:dind
$ docker exec -ti stakkr-test ash

# From the container
# Install packages required by stakkr + w3m as a local browser
$ apk add curl git python3 w3m

# Install stakkr
$ python3 -m pip install --upgrade https://github.com/stakkr-org/stakkr/archive/master.zip
# Stakkr should always be started as another user than root
$ addgroup stakkr
$ adduser -s /bin/ash -D -S -G stakkr stakkr
$ addgroup stakkr root
$ su - stakkr

# Create a symfony project from a recipe
$ mkdir ~/app && cd ~/app
$ stakkr-init symfony
# The following command should returns the default symfony page
$ w3m http://nginx.app.localhost

# Go further
$ mkdir ~/wp && cd ~/wp
$ stakkr-init wordpress
# The following command should returns wordpress home
$ w3m http://apache.wp.localhost
Test your local cloned stakkr from a container

The code below starts a dind container, mount stakkr then install it and init a symfony app. You need to be inside the stakkr cloned repository for that :

# From the host
$ docker run -d --privileged -v $(pwd):/stakkr-src --rm --name stakkr-test docker:dind
$ docker exec -ti stakkr-test ash

# From the container
# Install packages required by stakkr + w3m as a local browser
$ apk add --no-cache python3 alpine-sdk curl git openssl-dev python3-dev w3m libffi-dev

# Install stakkr
$ python3 -m pip install --upgrade /stakkr-src

# Then do what you want ...

Configuration

If you used a recipe, simply edit the stakkr.yml file manually to change a service version or set any parameter. Else, copy the file stakkr.yml.tpl to stakkr.yml and set the right configuration parameters you need.

Configuration is validated. Read carefully the message in case of error.

Main configuration parameters should be defined in the services section.

Services

You can define a list of services you want to have. Each service consists of a yml file in the services/ directory of the source code. Each container (“Service”) will have a hostname which is the … service name. To reach, for example, the elasticsearch server from a web application use elasticsearch. To connect to mysql it’s mysql.

Example of a LAMP stack :

services:
  adminer:
    enabled: true
  mysql:
    enabled: true
    version: 5.7
    ram: 1024M
    root_password: root
  apache:
    enabled: true
  php:
    enabled: true
    version: latest
    ram: 1024M
    blocked_ports: [25, 465, 587]

To have a complete list of services, launch :

$ stakkr services

The parameters are pretty generic, but some services could define new parameters such as databases for passwords :

services:
  any_service:
    # Enable it or not. Default false
    enabled: false
    # Version on docker hub
    version: latest
    # Limited as much as possible to keep computer resources usage
    ram: 512M
    # Displayed after stakkr has started
    service_name: Portainer (Docker Webadmin)
    # Same than above
    service_url: http://{}
    # Port to block for outgoing connexions. Requires :
    # - "cap_add: [NET_ADMIN, NET_RAW]" in compose file
    # - iptables on the container
    blocked_ports: []

HTTPS

Stakkr is now full https with an self-signed certificate. If you don’t want to accept the certificate everytime, you can ask chrome to accept all localhost certificates by calling chrome://flags/#allow-insecure-localhost as a URL.

Aliases

To enter a container you can use the stakkr console command. Nevertheless, to avoid doing :

stakkr console php
cd app
composer install

You can set the following alias in the stakkr.yml file :

services:
...

aliases:
  composer:
    description: Run a PHP composer command
    exec:
      - container: php
        user: www-data
        args: [php, /home/www-data/bin/composer]

And then :

cd app
stakkr composer install

An alias is a dictionary with :

  • A key that is the command name (composer above)

  • A description displayed when you run stakkr

  • An exec list with all commands to run when stakkr {alias} is invoked.
    • container is the container name

    • user the user to run the command

    • args a dictionnary with the command cut in pieces (that’s required).

Network and changes in general

You can define your own network in compose.ini by setting a subnet. It’s optional, and it’s probably better to let it like that.

Warning

If you change that, run docker system prune -f -a --volumes which removes orphans images, stopped container, etc …

As we use traefik as a reverse proxy, no need to expose any ports or to access containers directly via their IP.

Also, if you change any parameter such as an environment variable run a stakkr restart --recreate to make sure that you start from a clean environment.

Special case of Elasticsearch

ElasticSearch needs a few manual commands to start from the version 5.x. Before starting stakkr, do the following :

$ mkdir data/elasticsearch
$ sudo chown -R 1000:1000 data/elasticsearch
$ sudo sysctl -w vm.max_map_count=262144

Special case of xhgui service

To be able to profile your script, add the service xhgui and read the `documentation`_

Other useful parameters

Project name (will be used as container’s prefix). It should be different for each project.

environment: dev # Environment variables sent to containers

proxy: # traefik
  enabled: true # By default it's enabled
  domain: localhost # append domain. Example : http://apache.my_project.localhost
  http_port: 80 # Http Port to expose
  https_port: 443 # Https Port to expose

project_name: '' # detected automatically, usually the main directory name

subnet: '' # if you really need to override the default network

uid: # if you really need to set a specific uid for files, current user by default
gid: # same for gid, current user's group by default

Files location

Public Files

  • All files served by the web server are located into www/

Services Data

  • MySQL data is into data/mysql

  • Mongo data is into data/mongo

  • ElasticSearch data is into data/elasticsearch

  • Redis data is into data/redis

Logs

  • Logs for Apache and PHP are located into logs/

  • Logs for MySQL are located into data/mysql/ (slow and error).

Configuration

  • If you need to override the PHP configuration you can put a file in conf/php-fpm-override with a .conf extension. The format is the fpm configuration files one. Example: php_value[memory_limit] = 127M.

  • If you need to override the mysql configuration you can put a file in conf/mysql-override with a .cnf extension.

Add binaries

You can add binaries (such as phpunit) that will automatically be available from the PATH by putting it to home/www-data/bin/

Important

You can use home/www-data to put everyhting you need to keep: your shell parameters in .bashrc, your ssh keys/config into .ssh, etc.

Usage

Get Help

To get a list of commands do stakkr --help and to get help for a specific command : stakkr start --help

Important

Help doesn’t work yet with command aliases.

CLI Reference

Docker Commands

stakkr-compose

Wrapper for docker-compose

stakkr-compose [OPTIONS] [COMMAND]...

Options

-c, --config-file <config_file>

Set stakkr config file location (default stakkr.yml)

Arguments

COMMAND

Optional argument(s)

Stakkr Commands

stakkr

Main CLI Tool that easily create / maintain a stack of services, for example for web development.

Read the configuration file and setup the required services by linking and managing everything for you.

stakkr [OPTIONS] COMMAND [ARGS]...

Options

--version

Show the version and exit.

-c, --config <config>

Set the configuration filename (stakkr.yml by default)

-d, --debug, --no-debug
-v, --verbose
console

Enter a container to perform direct actions such as install packages, run commands, etc.

stakkr console [OPTIONS] CONTAINER

Options

-u, --user <user>

User’s name. Valid choices : www-data or root

-t, --tty, --no-tty

Use a TTY

Arguments

CONTAINER

Required argument

exec

Execute a command into a container.

Examples:

  • stakkr -v exec mysql mysqldump -p'$MYSQL_ROOT_PASSWORD' mydb > /tmp/backup.sql

  • stakkr exec php php -v : Execute the php binary in the php container with option -v

  • stakkr exec apache service apache2 restart

stakkr exec [OPTIONS] CONTAINER COMMAND...

Options

-u, --user <user>

User’s name. Be careful, each container have its own users.

-t, --tty, --no-tty

Use a TTY

-w, --workdir <workdir>

Working directory

Arguments

CONTAINER

Required argument

COMMAND

Required argument(s)

restart

Restart all (or a single as CONTAINER) container(s)

stakkr restart [OPTIONS] [CONTAINER]

Options

-p, --pull

Force a pull of the latest images versions

-r, --recreate

Recreate all containers

-P, --proxy, --no-proxy

Restart the proxy

Arguments

CONTAINER

Optional argument

services

List available services available for stakkr.yml (with info if the service is enabled)

stakkr services [OPTIONS]
services-add

Download a pack of services from github (see github) containing services. PACKAGE is the git url or package name. NAME is the directory name to clone the repo.

stakkr services-add [OPTIONS] PACKAGE [NAME]

Arguments

PACKAGE

Required argument

NAME

Optional argument

services-update

Update all services packs in services/

stakkr services-update [OPTIONS]
start

Start all (or a single as CONTAINER) container(s) defined in compose.ini

stakkr start [OPTIONS] [CONTAINER]

Options

-p, --pull

Force a pull of the latest images versions

-r, --recreate

Recreate all containers

-P, --proxy, --no-proxy

Start proxy

Arguments

CONTAINER

Optional argument

status

Display a list of running containers

stakkr status [OPTIONS]
stop

Stop all (or a single as CONTAINER) container(s)

stakkr stop [OPTIONS] [CONTAINER]

Options

-P, --proxy, --no-proxy

Stop the proxy

Arguments

CONTAINER

Optional argument

Stakkr Init

stakkr-init

Initialize for the first time stakkr by copying templates and directory structure

stakkr-init [OPTIONS] [RECIPE]

Options

-f, --force

Force recreate directories structure

Arguments

RECIPE

Optional argument

Custom Services

Overview

If you need a specific service that is not included in stakkr by default, you can create your own package and add it to the services/ directory.

Write a Package

A stakkr package adds to stakkr a set of new services. For example, when you add services via the stakkr services-add command, it adds a directory into services/ that contain one or more services.

Each service respects the docker-compose standard, plus a few customizations.

Some rules:

  • A package comes with its config validation.

  • Each yaml file defining a service must be named with the same name than the service

  • The service will be available in stakkr.yml once defined

  • A configuration parameter such as :

memcached:
  ram: 1024M

generates an environment variable with a name like DOCKER_MEMCACHED_RAM. That variable is usable in the service definition (docker-compose file).

Example

Let’s make a new nginx service.

1/ We need to define the config_schema.yml that will validate the service : See https://json-schema.org

services/nginx2/config_schema.yml
---

"$schema": http://json-schema.org/draft-04/schema#
type: object
properties:
  services:
    type: object
    additionalProperties: false
    properties:
      nginx2:
        type: object
        additionalProperties: false
        properties:
          enabled: { type: boolean }
          version: { type: [string, number] }
          ram: { type: string }
          service_name: { type: string }
          service_url: { type: string }
        required: [enabled, version, ram, service_name, service_url]

2/ Then the config_default.yml with the default values, some are required :

services/nginx2/config_default.yml
---

services:
  nginx2:
    enabled: false # Required and set to false by default
    version: latest
    ram: 256M
    service_name: Nginx (Web Server) # Required for stakkr status message
    service_url: http://{} (works also in https) # Required for stakkr status message

3/ Then the service itself in a docker-compose/ subdirectory :

services/nginx2/docker-composer/nginx2.yml
version: "3.8"

services:
    nginx2:
        image: edyan/nginx:${DOCKER_NGINX2_VERSION}
        mem_limit: ${DOCKER_NGINX2_RAM}
        container_name: ${COMPOSE_PROJECT_NAME}_nginx2
        hostname: ${COMPOSE_PROJECT_NAME}_nginx2
        networks: [stakkr]
        labels:
            - traefik.frontend.rule=Host:nginx2.${COMPOSE_PROJECT_NAME}.${PROXY_DOMAIN}

4/ Finally, check that it’s available and add it to stakkr.yml :

stakkr services

Output should be like :

...
- mysql ()
- nginx2 ()
- php ()
...

Now in stakkr.yml

stakkr.yml
services:
  nginx2:
    enabled: true
    ram: 1024M

Restart:

$ stakkr restart --recreate
$ stakkr status

To run a command, use the standard exec wrapper or create an alias:

$ stakkr exec nginx2 cat /etc/passwd

Build your service instead of using an existing image

When you need to build your own image and use it in stakkr, you just need to add a Dockerfile, like below, then run stakkr-compose build each time you need to build it. Once built, a simple stakkr start is enough to start it.

Example again with nginx2 :

1/ Create the services/nginx2/docker-compose/Dockerfile.nginx2 file :

FROM edyan/nginx:latest
# etc...

2/ Change the services/nginx2/docker-composer/nginx2.yml file :

services/nginx2/docker-composer/nginx2.yml
version: "3.8"

services:
  nginx2:
    build:
      context: ${COMPOSE_BASE_DIR}/services/nginx2/docker-compose
      dockerfile: Dockerfile.nginx2
    mem_limit: ${DOCKER_NGINX2_RAM}
    container_name: ${COMPOSE_PROJECT_NAME}_nginx2
    hostname: ${COMPOSE_PROJECT_NAME}_nginx2
    networks: [stakkr]
    labels:
      traefik.enable: "true"
      traefik.http.services.nginx2.loadbalancer.server.port: 80
      traefik.http.routers.nginx2.tls: "true"
      traefik.http.routers.nginx2.rule: "Host(`nginx2.${COMPOSE_PROJECT_NAME}.${PROXY_DOMAIN}`)"

Stakkr’s code structure

Stakkr works with a few modules / classes:

Module stakkr.actions

Stakkr main controller. Used by the CLI to do all its actions.

class stakkr.actions.StakkrActions(ctx: dict)[source]

Main class that does actions asked in the cli.

console(container: str, user: str, is_tty: bool)[source]

Enter a container. Stakkr will try to guess the right shell.

exec_cmd(container: str, user: str, args: tuple, is_tty: bool, workdir: Optional[str] = None)[source]

Run a command from outside to any container. Wrapped into /bin/sh.

get_config()[source]

Read and validate config from config file

get_services_urls()[source]

Once started, displays a message with a list of running containers.

get_url(service_url: str, service: str)[source]

Build URL to be displayed.

init_project()[source]

Initializing the project by reading config and setting some properties of the object

start(container: str, pull: bool, recreate: bool, proxy: bool)[source]

If not started, start the containers defined in config.

status()[source]

Return a nice table with the list of started containers.

stop(container: str, proxy: bool)[source]

If started, stop the containers defined in config. Else throw an error.

Module stakkr.aliases

Aliases management

stakkr.aliases.get_aliases()[source]

Get aliases from config file

stakkr.aliases.get_config_from_argv(argv: list)[source]

Search for a config file option in command line

Module stakkr.command

Command Wrapper.

A command wrapper to get a live output displayed.

stakkr.command.launch_cmd_displays_output(cmd: list, print_msg: bool = True, print_err: bool = True, err_to_out: bool = False)[source]

Launch a command and displays conditionally messages and / or errors.

stakkr.command.verbose(display: bool, message: str)[source]

Display a message if verbose is On.

Module stakkr.configreader

Simple Config Reader.

class stakkr.configreader.Config(config_file: str)[source]

Parser of Stakkr.

Set default values and validate stakkr.yml with specs

display_errors()[source]

Display errors in STDERR.

read()[source]

Parse the configs and validate it.

It could be either local or from a local services (first local then packages by alphabetical order).

Module stakkr.docker_actions

Docker functions to get info about containers.

stakkr.docker_actions.add_container_to_network(container: str, network: str)[source]

Attach a container to a network.

stakkr.docker_actions.block_ct_ports(service: str, ports: list, project_name: str)tuple[source]

Run iptables commands to block a list of port on a specific container.

stakkr.docker_actions.check_cts_are_running(project_name: str)[source]

Throw an error if cts are not running.

stakkr.docker_actions.container_running(container: str)[source]

Return True if the container is running else False.

stakkr.docker_actions.create_network(network: str)[source]

Create a Network.

stakkr.docker_actions.get_api_client()[source]

Return the API client or initialize it.

stakkr.docker_actions.get_client()[source]

Return the client or initialize it.

stakkr.docker_actions.get_ct_item(compose_name: str, item_name: str)[source]

Get a value from a container, such as name or IP.

stakkr.docker_actions.get_ct_name(container: str)[source]

Return the system name of a container, generated by docker-compose.

stakkr.docker_actions.get_network_name(project_name: str)[source]

Find the full network name.

stakkr.docker_actions.get_running_containers(project_name: str)tuple[source]

Get the number of running containers and theirs details for the current stakkr instance.

stakkr.docker_actions.get_running_containers_names(project_name: str)list[source]

Get a list of compose names of running containers for the current stakkr instance.

stakkr.docker_actions.get_subnet(project_name: str)[source]

Find the subnet of the current project.

stakkr.docker_actions.get_switch_ip()[source]

Find the main docker daemon IP to add routes.

stakkr.docker_actions.guess_shell(container: str)str[source]

By searching for binaries, guess what could be the primary shell available.

stakkr.docker_actions.network_exists(network: str)[source]

Return True if a network exists in docker, else False.

Module stakkr.file_utils

Files Utils to find dir, files, etc.

Such as : static files locations or directories location

stakkr.file_utils.find_project_dir()[source]

Determine the project base dir, by searching a stakkr.yml file

stakkr.file_utils.get_dir(directory: str)[source]

Detect if stakkr is a package or a clone and gives the right path for a directory.

stakkr.file_utils.get_file(directory: str, filename: str)[source]

Detect if stakkr is a package or a clone and gives the right path for a file.

stakkr.file_utils.get_lib_basedir()[source]

Return the base directory of stakkr, where all files are, to read services and config.

Module stakkr.proxy

Manage public proxy to expose containers.

class stakkr.proxy.Proxy(http_port: int = 80, https_port: int = 443, ct_name: str = 'proxy_stakkr', version: str = 'latest')[source]

Main class that does actions asked by the cli.

start(stakkr_network: Optional[str] = None)[source]

Start stakkr proxy if stopped.

stop()[source]

Stop stakkr proxy.

Module stakkr.services

Manage Stakkr Packages (extra packages)

stakkr.services.install(services_dir: str, package: str, name: str)[source]

Install a specific service by cloning a repo

stakkr.services.update_all(services_dir: str)[source]

Update all services by pulling

stakkr.services.update_package(path: str)[source]

Update a single service withgit pull

Indices and tables