====== Docker Development Workflow ======
Taking project vfoss as example, we describe workflow when working with the project.
Summary of steps:
- Create local image
- Download and Install additional software to base image: **Dockerfile.s2i.base**
- Create s2i workspace for: Dockerfile, src folder in the image (/opt/app-root).
- Modify Dockerfile, install packages, update s2i files, etc.
- Build local image: docker build s2i_base_img
- **Result: s2i_base_img**
- Run s2i to push source code to the base image and produce local image.
- s2i build source_folder s2i_base_img **my_image**
- (Optional) Rerun S2I to add source code to image with scripts: **s2i build ...**
- (Optional) other methods: commit changes to of running image to the local image.
- Development Dockerfile / Compose.yml
- Upload image
- Tag local image: **docker tag ...**
- docker tag image_id docker_hub_id/repo_image_name:version
- Upload: **docker push ...***
- docker push docker_hub_id/repo_image_name
- Production Dockerfile / Compose.yml
Create 1st image: Stock image --docker build Dockerfile.s2i--> image:s2i --s2i build--> image:latest
Development: image:latest --docker build Dockerfile.dev--> image:dev --> run + edit code
* https://blog.openshift.com/create-s2i-builder-image/
* https://github.com/openshift/source-to-image
===== - Local Images and Source =====
==== - Building Base Image with Dockerfile.s2i.base ====
The purpose of this step is to create an image with minimal packages required to run our application (django-cms). We generate an s2i workspace and update the generated Dockerfile (Dockerfile.s2i.base).
To generate s2i workspace refer to TBD: Create s2i workspace for: Dockerfile, src folder in the image (/opt/app-root).
Modify Dockerfile, install packages, update s2i files, etc. Sample Dockerfile.s2i.base below.
We can now build an s2i image using the Dockerfile:
docker build -t vfoss_img_web:s2i -f Dockerfile.s2i.base . | or make build (Makefile)
=== - Dockerfile.s2i.base ===
# This image provides a Python 2.7 environment you can use to run your Python
# applications.
### 2 Steps
# 1. Make (pre)build image: docker build -t vfoss_img_web:s2i -f Dockerfile.s2i.base . | or make build (Makefile)
# 2. build s2i command
# s2i build --loglevel=4 vfoss_src vfoss_img_web:s2i vfoss_img_web:dev
# s2i build --loglevel=4 vfoss_org vfoss_img_web:s2i vfoss_img_web:dev
# change src file and run more s2i build to create final image.
# 3. Commit vfoss_img_web:latest/release
# 4. Use uploaded image for production with production Dockerfile
FROM centos/s2i-base-centos7
MAINTAINER Thuy Dang
EXPOSE 8080
ENV PYTHON_VERSION=2.7 \
PATH=$HOME/.local/bin/:$PATH
LABEL io.k8s.description="Platform for building and running Python 2.7 applications" \
io.k8s.display-name="Python 2.7" \
io.openshift.expose-services="8080:http" \
io.openshift.tags="builder,python,python27,rh-python27"
USER root
RUN yum install -y centos-release-scl && \
INSTALL_PKGS="libjpeg-turbo libjpeg-turbo-devel python27 python27-python-devel python27-python-setuptools python27-python-pip nss_wrapper httpd httpd-devel atlas-devel gcc-gfortran gettext postgresql-libs nmap-ncat " && \
yum install -y --setopt=tsflags=nodocs --enablerepo=centosplus $INSTALL_PKGS && \
rpm -V $INSTALL_PKGS && \
# Some additional packages
#pip install virtualenvwapper && \
#echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc && \
# Remove centos-logos (httpd dependency, ~20MB of graphics) to keep image
# size smaller.
#rpm -e --nodeps centos-logos && \
yum clean all -y
# Each language image can have 'contrib' a directory with extra files needed to
# run and build the applications.
COPY ./contrib/ /opt/app-root
# Copy the S2I scripts from the specific language image to $STI_SCRIPTS_PATH.
#COPY ./s2i/bin/ $STI_SCRIPTS_PATH
#USER root
#COPY ./s2i/bin/* /usr/libexec/s2i/
LABEL io.openshift.s2i.scripts-url=image:///usr/libexec/s2i
COPY ./vfoss_org/.s2i/bin/ /usr/libexec/s2i
#RUN rm -rf /opt/app-root/*
# Copy the S2I scripts from the specific language image to other location and update.
#USER 1001
#RUN mkdir -p /opt/app-root/s2i/bin
#COPY ./s2i/bin /opt/app-root/s2i/
#LABEL io.openshift.s2i.scripts-url=image:///opt/app-root/s2i/bin
# App specifics
#RUN mkdir -p /opt/app-root/logs
# In order to drop the root user, we have to make some directories world
# writable as OpenShift default security model is to run the container under
# random UID.
RUN chown -R 1001:0 /opt/app-root && chmod -R ug+rwx /opt/app-root
USER 1001
# Set the default CMD to print the usage of the language image.
CMD /usr/libexec/s21/usage
==== - Produce Image With Source Code ====
Run s2i to push source code to the base image and produce local image.
s2i build source_folder s2i_base_img **my_image**
s2i build --loglevel=4 vfoss_src vfoss_img_web:s2i vfoss_img_web:latest
Before tagging the local image and publish it, we may changes source code and rerun s2i build multiple times.
- (Optional) Rerun S2I to add source code to image with scripts: **s2i build ...**
- (Optional) other methods: commit changes to of running image to the local image.
==== - Developing / Changing Source ====
The generated image is kept minimal for production. Changing source code with the image is inconvenient e.g., when doing web development. Moreover, while developing, dev profile / settings may be more desirable. Consequently we create a docker image for developing using Dockerfile.development below. We use generated image (vfoss_image_web:lates) and add convenient packages, ENV variables for the development.
Creating development image:
docker build -t vfoss_img_web:dev -f Dockerfile.development . (the dot)
We can run bash shell to directly changes source code on the development image. Here we mount source and logs folders so we can edit and debug using host machine's development tools.
sudo docker run -it -u root -v $(pwd)/vfoss_org:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfoss_img_web:dev bash
docker run -it -u root -v $(pwd)/vfoss_org:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfoss_img_web:dev bash
We can also use a docker-compose file to quickly start container for development.
docker-compose -f docker-compose-dev.yml run --service-ports vfoss_web bash
=== - docker-compose-dev.yml ===
### Compose file to work with source
# 0. create network for containers
# 1. start db container
# 2. start app container from production image with mounted app_src, logs,
version: "2"
services:
vfoss_web:
container_name: vfoss_web
#image: vsrccom/vfoss_img_web
image: vfoss_img_web:dev
build:
context: ./
dockerfile: Dockerfile.development
networks:
- vfoss_bridge
ports:
#- "host-->container"
- "9002:8080"
volumes:
# Path on the host, relative to the Compose file, abs path
- ./vfoss_org:/opt/app-root/src
- ./logs:/opt/app-root/logs
# - ./api:/var/www/app
#env_file: ../../vfoss_web/vfoss/.s2i/environment
environment:
- DJANGO_SETTINGS_MODULE=vfoss_org.settings.dev
#- DB_ENGINE=django.db.backends.postgresql_psycopg2
#- DB_NAME=vfossdb
#- DB_USER=vfossdbuser
##DB_PASSWORD=
#- DB_HOST=db
#- DB_Port=5432
#tty: true
#depends_on:
#- vfoss_db
#command: /var/www/app/start.sh
#user: $UID
user: root
command: /bin/bash
# vfoss_db:
# container_name: vfoss_db
# image: vfoss_img_postgres:9.5
# #build: ../../vfoss_db
# volumes:
# - "vfoss_db_data:/var/lib/postgresql/data"
# networks:
# - vfoss_bridge
# ports:
# - "5002:5432"
# privileged: true
# #tty: true
# # DB_VARS in setup.sh entry point
# #env_file: .env
# #environment:
# # - POSTGRES_USER=vfossdbuser
# # - POSTGRES_PASSWORD=
# # - POSTGRES_DB=vfossdb
#
#volumes:
# vfoss_db_data:
# #must declare volumes
networks:
vfoss_bridge:
#driver: overlay
## usage: docker-compose up,
#
## before: docker-compose build (svc_name)
# Compose build only runs Dockerfile. Image with s2i can not be built with compose.
# Compose can be use to run (up) container, however. Note set image-name to s2i built image.
=== - Dockerfile.development ===
# This image is built with s2i and ready to use.
### 2 Steps
# 1. Make (pre)build image: docker build -t vfoss_img_web_build | or make build (Makefile)
# 2. build s2i command
# s2i build --loglevel=4 vfoss_src vfoss_img_web_build vfoss_img_web_final
# s2i build --loglevel=4 vfoss vfoss_img_web_build vfoss_img_web_final
# change src file and run more s2i build to create final image.
# 3. Commit vfoss_img_web_final
# 4. Use uploaded image for production with production Dockerfile
FROM vfoss_img_web:latest
#FROM centos/python-27-centos7
MAINTAINER vsrc.com
EXPOSE 8080
## SYSTEM ENV
#
ENV PYTHON_VERSION=2.7 \
PATH=$HOME/.local/bin/:$PATH
LABEL io.k8s.description="Platform for building and running Python 2.7 applications" \
io.k8s.display-name="Python 2.7" \
io.openshift.expose-services="8080:http" \
io.openshift.tags="builder,python,python27,rh-python27"
USER root
RUN yum install -y centos-release-scl && \
INSTALL_PKGS="libjpeg-turbo libjpeg-turbo-devel python27-python-pip nss_wrapper atlas-devel gcc-gfortran gettext postgresql-libs nmap-ncat vim-enhanced net-tools" && \
yum install -y --setopt=tsflags=nodocs --enablerepo=centosplus $INSTALL_PKGS && \
rpm -V $INSTALL_PKGS && \
# Some additional packages
#pip install virtualenvwapper && \
#echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc && \
# Remove centos-logos (httpd dependency, ~20MB of graphics) to keep image
# size smaller.
#rpm -e --nodeps centos-logos && \
yum clean all -y
## WORKING DIR
# Each language image can have 'contrib' a directory with extra files needed to
# run and build the applications.
COPY ./contrib/ /opt/app-root
RUN mkdir -p /opt/app-root/logs
RUN chown -R 1001:0 /opt/app-root && chmod -R ug+rwx /opt/app-root
## PYTHON PKG
ENV PIP_REQUIRE_VIRTUALENV false
# Install at build time
ADD ./vfoss_org/requirements/development.txt /tmp/requirements.txt
ADD ./vfoss_org/requirements/base.txt /tmp/base.txt
RUN /opt/rh/python27/root/usr/bin/pip install -r /tmp/requirements.txt
#RUN pip install -r /tmp/requirements.txt
## START APP
USER 1001
ENV DJANGO_SETTINGS_MODULE=vfoss_org.settings.dev
WORKDIR /opt/app-root/src
## run dev env. Must be put in entry-point.sh
#RUN python createsu.py
#CMD python manage.py runserver 0.0.0.0:8080
#
# sudo docker run -it -u root -v $(pwd)/vfoss_org:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfoss_img_web:dev bash
#
===== - Create Production Image =====
When source code is updated we need to create new production images.
Build image from s2i base:
s2i build --loglevel=4 vfoss_org vfoss_img_web:s2i visrccom/vfoss_img_web
Build image from last version (**Creating larger image**):
s2i build --loglevel=4 vfoss_org vfoss_img_web:latest visrccom/vfoss_img_web
===== - Test Production Image =====
Cleanup:
docker-compose -f eco/backend/docker-compose.yml rm --all
Django static files might not be updated.
Run production container locally:
docker-compose -f eco/backend/docker-compose.yml up -d
# OR
docker-compose -f ./eco/backend/docker-compose.yml start vfoss_db
Starting vfoss_db ... done
docker-compose -f ./eco/backend/docker-compose.yml start vfoss_web
# Wait a while...
docker rm -f vfoss_db
sudo umount /var/lib/docker... IT WORKS despite ERROR MSG
# to get a bash shell in the container
docker exec -it /bin/bash
docker exec -ti -u root container_name bash
docker-compose -f ./eco/backend/docker-compose.yml exec vfoss_db bash
docker ps -a
docker exec -it mad_goldstine bash
psql -U dbuser -d anthenaodb
root@f08c71411a8d:/# gosu postgres psql
psql (9.5.5)
docker exec -ti -u root vfoss_web bash
===== - Upload / Publish image =====
Upload to docker hub:
docker push visrccom/vfoss_img_web
Pull image on production server and restart container:
docker pull visrccom/vfoss_img_web
docker-compose restart vfoss_web
===== - Mounting Host Folders =====
Volume settings in docker-compose.yml
vfoss_db:
container_name: vfoss_db
image: vfoss_img_postgres:9.5
build: ../../vfoss_db
volumes:
# - /path/on/host:/path/in/container
# - volume_name:/path/in/container
# - "vfoss_db_data:/var/lib/postgresql/data"
- "../../../docker_data/var/lib/postgresql/data:/var/lib/postgresql/data"
When the container is created for the first time, the folder is created on host machine. It will bound with the container, that means, if the path is changed later, starting the container still mounts previous folder. We must delete and recreate the container to use new host folder.
Mount host folders for django static and media files. Serve static with nginx!
====== Old ======
===== - S2I Image Project =====
Create 1st image: Stock image --docker build Dockerfile.s2i--> image:s2i --s2i build--> image:latest
Development: image:latest --docker build Dockerfile.dev--> image:dev --> run + edit code
docker build -t vfoss_img_web:dev -f Dockerfile.development .
docker run -it -u root -v $(pwd)/vfoss_org:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfoss_img_web:dev bash
Produce release image: image:s2i --s2i build--> image:release
Commit public latest: