My Wiki!

Table of Contents

Docker Development Workflow

Taking project vfoss as example, we describe workflow when working with the project.

Summary of steps:

  1. Create local image
    1. Download and Install additional software to base image: Dockerfile.s2i.base
      1. Create s2i workspace for: Dockerfile, src folder in the image (/opt/app-root).
      2. Modify Dockerfile, install packages, update s2i files, etc.
    2. Build local image: docker build s2ibaseimg
      1. Result: s2ibaseimg
    3. Run s2i to push source code to the base image and produce local image.
      1. s2i build sourcefolder s2ibaseimg myimage - (Optional) Rerun S2I to add source code to image with scripts: s2i build … - (Optional) other methods: commit changes to of running image to the local image. - Development Dockerfile / Compose.yml - Upload image - Tag local image: docker tag … - docker tag imageid dockerhubid/repoimagename:version - Upload: docker push …* - docker push dockerhubid/repoimage_name - Production Dockerfile / Compose.yml Create 1st image: Stock image –docker build Dockerfile.s2i–> image:s2i –s2i build–> image:latest Development: image:latest –docker build Dockerfile.dev–> image:dev –> run + edit code * https://blog.openshift.com/create-s2i-builder-image/ * https://github.com/openshift/source-to-image

        1. Local Images and Source

        1.1 Building Base Image with Dockerfile.s2i.base

        The purpose of this step is to create an image with minimal packages required to run our application (django-cms). We generate an s2i workspace and update the generated Dockerfile (Dockerfile.s2i.base). To generate s2i workspace refer to TBD: Create s2i workspace for: Dockerfile, src folder in the image (/opt/app-root). Modify Dockerfile, install packages, update s2i files, etc. Sample Dockerfile.s2i.base below. We can now build an s2i image using the Dockerfile: docker build -t vfossimgweb:s2i -f Dockerfile.s2i.base . | or make build (Makefile)

        1.1.1 Dockerfile.s2i.base

        <code> # This image provides a Python 2.7 environment you can use to run your Python # applications. ### 2 Steps # 1. Make (pre)build image: docker build -t vfossimgweb:s2i -f Dockerfile.s2i.base . | or make build (Makefile) # 2. build s2i command # s2i build –loglevel=4 vfosssrc vfossimgweb:s2i vfossimgweb:dev # s2i build –loglevel=4 vfossorg vfossimgweb:s2i vfossimgweb:dev # change src file and run more s2i build to create final image. # 3. Commit vfossimgweb:latest/release # 4. Use uploaded image for production with production Dockerfile FROM centos/s2i-base-centos7 MAINTAINER Thuy Dang thuydang.de@gmail.com EXPOSE 8080 ENV PYTHON_VERSION=2.7 \ PATH=$HOME/.local/bin/:$PATH LABEL io.k8s.description=“Platform for building and running Python 2.7 applications” \ io.k8s.display-name=“Python 2.7” \ io.openshift.expose-services=“8080:http” \ io.openshift.tags=“builder,python,python27,rh-python27” USER root RUN yum install -y centos-release-scl && \ INSTALLPKGS=“libjpeg-turbo libjpeg-turbo-devel python27 python27-python-devel python27-python-setuptools python27-python-pip nsswrapper httpd httpd-devel atlas-devel gcc-gfortran gettext postgresql-libs nmap-ncat ” && \ yum install -y –setopt=tsflags=nodocs –enablerepo=centosplus $INSTALLPKGS && \ rpm -V $INSTALLPKGS && \ # Some additional packages #pip install virtualenvwapper && \ #echo “source /usr/local/bin/virtualenvwrapper.sh” » ~/.bashrc && \ # Remove centos-logos (httpd dependency, ~20MB of graphics) to keep image # size smaller. #rpm -e –nodeps centos-logos && \ yum clean all -y # Each language image can have 'contrib' a directory with extra files needed to # run and build the applications. COPY ./contrib/ /opt/app-root # Copy the S2I scripts from the specific language image to $STISCRIPTSPATH. #COPY ./s2i/bin/ $STISCRIPTSPATH #USER root #COPY ./s2i/bin/* /usr/libexec/s2i/ LABEL io.openshift.s2i.scripts-url=image:/usr/libexec/s2i COPY ./vfoss_org/.s2i/bin/ /usr/libexec/s2i #RUN rm -rf /opt/app-root/* # Copy the S2I scripts from the specific language image to other location and update. #USER 1001 #RUN mkdir -p /opt/app-root/s2i/bin #COPY ./s2i/bin /opt/app-root/s2i/ #LABEL io.openshift.s2i.scripts-url=image:/opt/app-root/s2i/bin # App specifics #RUN mkdir -p /opt/app-root/logs # In order to drop the root user, we have to make some directories world # writable as OpenShift default security model is to run the container under # random UID. RUN chown -R 1001:0 /opt/app-root && chmod -R ug+rwx /opt/app-root USER 1001 # Set the default CMD to print the usage of the language image. CMD /usr/libexec/s21/usage </code>

        1.2 Produce Image With Source Code

        Run s2i to push source code to the base image and produce local image. s2i build sourcefolder s2ibaseimg myimage s2i build –loglevel=4 vfosssrc vfossimgweb:s2i vfossimg_web:latest
        Before tagging the local image and publish it, we may changes source code and rerun s2i build multiple times.
        - (Optional) Rerun S2I to add source code to image with scripts: s2i build … - (Optional) other methods: commit changes to of running image to the local image.

        1.3 Developing / Changing Source

        The generated image is kept minimal for production. Changing source code with the image is inconvenient e.g., when doing web development. Moreover, while developing, dev profile / settings may be more desirable. Consequently we create a docker image for developing using Dockerfile.development below. We use generated image (vfossimageweb:lates) and add convenient packages, ENV variables for the development. Creating development image: docker build -t vfossimgweb:dev -f Dockerfile.development . (the dot)
        We can run bash shell to directly changes source code on the development image. Here we mount source and logs folders so we can edit and debug using host machine's development tools. sudo docker run -it -u root -v $(pwd)/vfossorg:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfossimgweb:dev bash docker run -it -u root -v $(pwd)/vfossorg:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfossimgweb:dev bash We can also use a docker-compose file to quickly start container for development. docker-compose -f docker-compose-dev.yml run –service-ports vfossweb bash

        1.3.1 docker-compose-dev.yml

        <code> ### Compose file to work with source # 0. create network for containers # 1. start db container # 2. start app container from production image with mounted appsrc, logs, version: “2” services: vfossweb:
        container        name: vfossweb #image: vsrccom/vfossimgweb image: vfossimgweb:dev build: context: ./ dockerfile: Dockerfile.development networks: - vfossbridge ports: #- “host–>container” - “9002:8080” volumes: # Path on the host, relative to the Compose file, abs path - ./vfossorg:/opt/app-root/src - ./logs:/opt/app-root/logs # - ./api:/var/www/app #envfile: ../../vfossweb/vfoss/.s2i/environment environment: - DJANGOSETTINGSMODULE=vfossorg.settings.dev #- DBENGINE=django.db.backends.postgresqlpsycopg2 #- DBNAME=vfossdb #- DBUSER=vfossdbuser ##DBPASSWORD= #- DBHOST=db #- DBPort=5432 #tty: true #dependson: #- vfoss_db #command: /var/www/app/start.sh #user: $UID user: root command: /bin/bash # vfossdb:
        # container        name: vfossdb # image: vfossimgpostgres:9.5 # #build: ../../vfossdb # volumes: # - “vfossdbdata:/var/lib/postgresql/data” # networks: # - vfossbridge # ports: # - “5002:5432” # privileged: true # #tty: true # # DBVARS in setup.sh entry point # #envfile: .env # #environment: # # - POSTGRESUSER=vfossdbuser # # - POSTGRESPASSWORD= # # - POSTGRESDB=vfossdb # #volumes: # vfossdbdata: # #must declare volumes networks: vfoss_bridge: #driver: overlay ## usage: docker-compose up, # ## before: docker-compose build (svc_name) # Compose build only runs Dockerfile. Image with s2i can not be built with compose. # Compose can be use to run (up) container, however. Note set image-name to s2i built image. </code>

        1.3.2 Dockerfile.development

        <code> # This image is built with s2i and ready to use. ### 2 Steps # 1. Make (pre)build image: docker build -t vfossimgwebbuild | or make build (Makefile) # 2. build s2i command # s2i build –loglevel=4 vfosssrc vfossimgwebbuild vfossimgwebfinal # s2i build –loglevel=4 vfoss vfossimgwebbuild vfossimgwebfinal # change src file and run more s2i build to create final image. # 3. Commit vfossimgweb_final # 4. Use uploaded image for production with production Dockerfile FROM vfossimgweb:latest #FROM centos/python-27-centos7 MAINTAINER vsrc.com write.thuy@gmail.com EXPOSE 8080 ## SYSTEM ENV # ENV PYTHON_VERSION=2.7 \ PATH=$HOME/.local/bin/:$PATH LABEL io.k8s.description=“Platform for building and running Python 2.7 applications” \ io.k8s.display-name=“Python 2.7” \ io.openshift.expose-services=“8080:http” \ io.openshift.tags=“builder,python,python27,rh-python27” USER root RUN yum install -y centos-release-scl && \ INSTALLPKGS=“libjpeg-turbo libjpeg-turbo-devel python27-python-pip nsswrapper atlas-devel gcc-gfortran gettext postgresql-libs nmap-ncat vim-enhanced net-tools” && \ yum install -y –setopt=tsflags=nodocs –enablerepo=centosplus $INSTALLPKGS && \ rpm -V $INSTALLPKGS && \ # Some additional packages #pip install virtualenvwapper && \ #echo “source /usr/local/bin/virtualenvwrapper.sh” » ~/.bashrc && \ # Remove centos-logos (httpd dependency, ~20MB of graphics) to keep image # size smaller. #rpm -e –nodeps centos-logos && \ yum clean all -y ## WORKING DIR # Each language image can have 'contrib' a directory with extra files needed to # run and build the applications. COPY ./contrib/ /opt/app-root RUN mkdir -p /opt/app-root/logs RUN chown -R 1001:0 /opt/app-root && chmod -R ug+rwx /opt/app-root ## PYTHON PKG ENV PIPREQUIREVIRTUALENV false # Install at build time ADD ./vfossorg/requirements/development.txt /tmp/requirements.txt ADD ./vfossorg/requirements/base.txt /tmp/base.txt RUN /opt/rh/python27/root/usr/bin/pip install -r /tmp/requirements.txt #RUN pip install -r /tmp/requirements.txt ## START APP USER 1001 ENV DJANGOSETTINGSMODULE=vfoss_org.settings.dev WORKDIR /opt/app-root/src ## run dev env. Must be put in entry-point.sh #RUN python createsu.py #CMD python manage.py runserver 0.0.0.0:8080 # # sudo docker run -it -u root -v $(pwd)/vfossorg:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfossimg_web:dev bash # </code>

        2. Create Production Image

        When source code is updated we need to create new production images. Build image from s2i base: s2i build –loglevel=4 vfossorg vfossimgweb:s2i visrccom/vfossimg_web Build image from last version (Creating larger image**): s2i build –loglevel=4 vfossorg vfossimgweb:latest visrccom/vfossimgweb

        3. Test Production Image

        Cleanup: docker-compose -f eco/backend/docker-compose.yml rm –all
        Django static files might not be updated. Run production container locally: docker-compose -f eco/backend/docker-compose.yml up -d # OR docker-compose -f ./eco/backend/docker-compose.yml start vfossdb Starting vfossdb … done docker-compose -f ./eco/backend/docker-compose.yml start vfossweb # Wait a while…
        docker rm -f vfoss        db sudo umount /var/lib/docker… IT WORKS despite ERROR MSG
        # to get a bash shell in the container docker exec -it <container name> /bin/bash docker exec -ti -u root containername bash docker-compose -f ./eco/backend/docker-compose.yml exec vfossdb bash
        docker ps -a docker exec -it madgoldstine bash
        psql -U dbuser -d anthenaodb
        root@f08c71411a8d:/# gosu postgres psql psql (9.5.5)
        docker exec -ti -u root vfoss        web bash

        4. Upload / Publish image

        Upload to docker hub: docker push visrccom/vfossimgweb
        Pull image on production server and restart container: docker pull visrccom/vfossimgweb docker-compose restart vfoss_web

        5. Mounting Host Folders

        Volume settings in docker-compose.yml <code> vfossdb:
        container        name: vfossdb image: vfossimgpostgres:9.5 build: ../../vfossdb volumes: # - /path/on/host:/path/in/container # - volumename:/path/in/container # - “vfossdbdata:/var/lib/postgresql/data” - “../../../dockerdata/var/lib/postgresql/data:/var/lib/postgresql/data” </code> When the container is created for the first time, the folder is created on host machine. It will bound with the container, that means, if the path is changed later, starting the container still mounts previous folder. We must delete and recreate the container to use new host folder. Mount host folders for django static and media files. Serve static with nginx! ====== Old ======

        6. S2I Image Project

        Create 1st image: Stock image –docker build Dockerfile.s2i–> image:s2i –s2i build–> image:latest Development: image:latest –docker build Dockerfile.dev–> image:dev –> run + edit code docker build -t vfossimgweb:dev -f Dockerfile.development . docker run -it -u root -v $(pwd)/vfossorg:/opt/app-root/src -v $(pwd)/logs:/opt/app-root/logs -p 8000:8080 vfossimg_web:dev bash Produce release image: image:s2i –s2i build–> image:release Commit public latest:


Trace: • docker_tutorial • docker_dev_workflow

Navigation