Skip to content

Commit

Permalink
Release Candidate 1.0.1 (#48)
Browse files Browse the repository at this point in the history 
## Changelog for Release Candidate 1.0.1
This release represents the first production release of MISP-Dockerized in version 1.0.0 and an bug fix for container tagging.

### About
MISP dockerized is a project designed to provide an easy-to-use and easy-to-install 'out of the box' MISP instance that includes everything you need to run MISP with minimal host-side requirements.

MISP dockerized uses MISP (Open Source Threat Intelligence Platform - https://github.com/MISP/MISP), which is maintained and developed by the MISP project team (https://www.misp-project.org/).

MISP-dockerized provides the following features:
* 3-steps (in 5 minute or less) ready MISP instance
* Built in MISP modules
* Built in mail transport via postfix
* Prehardened TLS configuration
* Default http to https redirect
* Easy-to-use script based backup and restore function
* Prebuilt management commands to ease maintenance efforts
* Periodically built Docker containers
* Automated test framework to test new MISP releases

-----------------------------
### Update Informations
This release includes the new MISP server container version 2.4.99

In addition, we have changed the following:
- the proxy container version to the latest version 1.X
- the robot container version to the latest version 2.X
- the misp-module container version to the latest version 1.X

### General Changes
- We have changed our complete directories. Now you have access to all unsupported and old version of MISP-dockerized.
- There is a new NEW_FRESH_INSTAL.sh script
- We have added a new UPGRADE.sh script
- We have added a new UPDATE.sh script
- Added the new MISP-Server Container with MISP Version 2.4.99 (#39)

### Detailed Changes
#### Main Repository
##### Detailed Changes, Fixes & Improvements
- Deactivated the Ansible script for database and server, because since version 2.4.97 the misp-server container does not require Ansible anymore
- Changed misp-robot from 1.0.3-ubuntu to 2.x-debian
- Changed misp-server from 2.4.94 to 2.4.99
- Removed unused container content in docker-compose.yml
- Added docker-compose.override.yml to add the user configuration
- Removed misp-postfix container, because postfix is now directly installed in the misp-server
- Changed volume paths from misp-server (done since version 0.3.4)

#### Container `misp-dockerized-server`
##### Update Informations
Features of the release:
- Added a new MISP-Server Container for MISP release 2.4.99 on base of 2.4.97. (closes #39)
  For more information please have a look at the official changelog at https://github.com/MISP/MISP/releases/tag/v2.4.99
- Changed to a new gitlab-ci job structure for all misp-server container
- Built in mail transport via postfix
- Built in redis
- Built in rsyslog-ng
- Built in MariaDB

##### Detailed Changes, Fixes & Improvements
- The following base changes were made in addition to the new `misp-server version 2.4.97`:
   - The base image is now **debian stretch** instead of ubuntu 18.04
   - Postfix is now directly installed - the postfix container is no longer necessary 
   - Added a cron shell script that fetches and caches all enabled feeds and pull all remote server events
   - Changed from syslog-ng to rsyslog, because in debian the syslog-ng doesn't support wildcard configuration for logfiles.
   - **The external requirement for our Ansible playbook is no longer necessary for this container**
- Gitlab-CI Strucutre
  Included statements in the Gitlab CI main file allow us to improve and simplify our Gitlab CI main file.
- Added an echo tag  to know from which entrypoint script a message comes.
- Added on the misp-server 2.4.97-2.4.99 functions to create self-signed certificate, private key and DH params file



#### Container `misp-dockerized-robot`
##### Update Informations 
Features of the release:
- Built in Docker
- Built in Ansible
- Built in Test framework
- Docker Container image is based now on debian:stretch-slim

In the past, we decided to use a three-digit version number for small and large changes for this container repository. This has proved unnecessary as the main repository is the central repository for new releases and all changes are already tracked here. We have therefore decided to change the 3-digit version number in all minor repositories to a 2-digit version number.
For example: 1.0.1 -> 1.1

The second digit is changed any time we make minor changes. The first digit changes whenever we make major changes, e.g. a new base image.

##### Detailed Changes, Fixes & Improvements
- Changed from the base image debian:stable-slim to debian:stretch-slim 
- Remove unused robot versions:
    - 1.0.2-debian
    - 1.0.3-debian
    - 1.0.4-ubuntu
- All folders have been renamed, e.g. from 1.0.0.0-alpin to 1.0-alpin.
- We have made working with gitlab-ci easier with new extensions and includes.
- We have also reduced the readme as much as possible, as the documentation is now available centrally in our own documentation repository `https://dcso.github.io/MISP-dockerized-docs/`.

#### Container `misp-dockerized-proxy`
##### Update Informations 
Features of the release:
- Prehardened TLS configuration
- Default http to https redirect
- Built in User defined IP restriction
- Added the ability to create self-signed certificates.

In the past, we decided to use a three-digit version number for small and large changes for this container repository. This has proved unnecessary as the main repository is the central repository for new releases and all changes are already tracked here. We have therefore decided to change the 3-digit version number in all minor repositories to a 2-digit version number.
For example: 1.0.1 -> 1.1

The second digit is changed any time we make minor changes. The first digit changes whenever we make major changes, e.g. a new base image.


##### Detailed Changes, Fixes & Improvements
- Added an own entrypoint script to create user defined files
- Added the possibility to create self-signed certificates
- All folders have been renamed, e.g. from 1.0.0.0-alpin to 1.0-alpin.
- We have made working with gitlab-ci easier with new extensions and includes.
- We have also reduced the readme as much as possible, as the documentation is now available centrally in our own documentation repository `https://dcso.github.io/MISP-dockerized-docs/`.


#### Container `misp-dockerized-misp-modules`
##### Update Informations 
Features of the release:
- Pre-Installed MISP modules

In the past, we decided to use a three-digit version number for small and large changes for this container repository. This has proved unnecessary as the main repository is the central repository for new releases and all changes are already tracked here. We have therefore decided to change the 3-digit version number in all minor repositories to a 2-digit version number.
For example: 1.0.1 -> 1.1

The second digit is changed any time we make minor changes. The first digit changes whenever we make major changes, e.g. a new base image.

##### Detailed Changes, Fixes & Improvements
- Pre-Installed debian stretch MISP modules container
- All folders have been renamed, e.g. from 1.0.0.0-debian to 1.0-debian.
- We have made working with gitlab-ci easier with new extensions and includes.
- We have also reduced the readme as much as possible, as the documentation is now available centrally in our own documentation repository `https://dcso.github.io/MISP-dockerized-docs/`.
  • Loading branch information
@8ear
8ear authored Dec 17, 2018
1 parent 9cced07 commit a30bc74
Show file tree
Hide file tree
Showing 6 changed files with 46 additions and 45 deletions.
37 changes: 15 additions & 22 deletions .gitlab-ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,10 @@ variables:
CURRENT_VERSION: "1.0.0"
FOLDER: "./current"
ENV_OPTION: "$FOLDER/"
MYmake_travis: "make -C $FOLDER/.travis/"
makefile_travis: "make -C $FOLDER/.travis/"
CI: "true"
REPOURL: "dockerhub.dcso.de"


# Include other YML Files for Jobs
# Source: https://docs.gitlab.com/ee/ci/yaml/README.html#include
include:
#- '0.1.0/.gitlab-ci.yml'
#- '0.1.1/.gitlab-ci.yml'
Expand All @@ -38,8 +35,6 @@ include:
#- '0.3.3/.gitlab-ci.yml'
#- '0.3.4/.gitlab-ci.yml'
- '1.0.0/.gitlab-ci.yml'



before_script:
- apk add --no-cache make bash sudo git curl python py2-pip coreutils grep python3
Expand All @@ -50,7 +45,7 @@ before_script:
# Choose the Environment Version
- bash ./FOR_NEW_INSTALL.sh $CURRENT_VERSION
# Pull all latest tagged container
- make -C $FOLDER/.travis/ pull-latest REPOURL=${REPOURL}
- $makefile_travis pull-latest REPOURL=${REPOURL}

# On all other pull requests and branches exclude dev and master: Build Environment, but do nothing more. No tests. Tests with our Test Framework are only done in the dev branch.
.env_test_only:
Expand Down Expand Up @@ -80,7 +75,7 @@ before_script:
- make -C $FOLDER deploy
- docker ps -a
# Testing
- make -C $FOLDER/.travis/ test
- $makefile_travis test
- make -C $FOLDER delete
# https://docs.gitlab.com/ee/ci/yaml/README.html#artifacts-reports
artifacts:
Expand All @@ -100,30 +95,28 @@ before_script:
- make -C $FOLDER deploy
- docker ps -a
# Testing
- make -C $FOLDER/.travis/ test
- $makefile_travis test
# prepare retagging
- export server_tag=$(cat $ENV_FOLDER.env |grep MISP_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export proxy_tag=$(cat $ENV_FOLDER.env |grep PROXY_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export robot_tag=$(cat $ENV_FOLDER.env |grep ROBOT_CONTAINER|cut -d = -f 2|sed 's,....$,,')
#- export postfix_tag=$(cat $ENV_FOLDER.env |grep POSTFIX_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export modules_tag=$(cat $ENV_FOLDER.env |grep MISP_MODULES_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export db_tag=$(cat $ENV_FOLDER.env |grep DB_CONTAINER_TAG|cut -d = -f 2|sed 's,....$,,')
- export redis_tag=$(cat $ENV_FOLDER.env |grep REDIS_CONTAINER_TAG|cut -d = -f 2|sed 's,....$,,')
- export server_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep server|cut -d ':' -f 3)
- export proxy_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep proxy|cut -d ':' -f 3)
- export robot_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep robot|cut -d ':' -f 3)
#- export postfix_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep server|cut -d ':' -f 3)
- export modules_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep modules|cut -d ':' -f 3)
- export db_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep db|cut -d ':' -f 3)
- export redis_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep redis|cut -d ':' -f 3)
# retag all existing tags dev 2 public repo
- $MYmake_travis tag server_tag=${server_tag} proxy_tag=${proxy_tag} robot_tag=${robot_tag} postfix_tag=${postfix_tag} modules_tag=${modules_tag} REPOURL=${REPOURL};
- $makefile_travis tag REPOURL=$REPOURL server_tag=${server_tag} proxy_tag=${proxy_tag} robot_tag=${robot_tag} postfix_tag=${postfix_tag} modules_tag=${modules_tag} db_tag=${modules_tag} redis_tag=${modules_tag}
# Push Images to registry
- $MYmake_travis push server_tag=${server_tag} proxy_tag=${proxy_tag} robot_tag=${robot_tag} postfix_tag=${postfix_tag} modules_tag=${modules_tag} REPOURL=${REPOURL};
# If we need separate Images fro DB and Redsi
#- ./travis/push.sh $(REPOURL)/misp-dockerized-db $(db_tag);
#- ./travis/push.sh $(REPOURL)/misp-dockerized-redis $(redis_tag);
- $makefile_travis push REPOURL=$REPOURL server_tag=${server_tag} proxy_tag=${proxy_tag} robot_tag=${robot_tag} postfix_tag=${postfix_tag} modules_tag=${modules_tag};
- make -C $FOLDER delete
# https://docs.gitlab.com/ee/ci/yaml/README.html#artifacts-reports
artifacts:
paths:
- ./current/.travis/reports/*.xml
reports:
junit: ./current/.travis/reports/*.xml



# deactivated:
# https://docs.gitlab.com/ee/ci/examples/container_scanning.html
# https://docs.gitlab.com/ee/ci/yaml/README.html#artifactsreportscontainer_scanning
Expand Down
14 changes: 8 additions & 6 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,15 @@ script:
- if [[ "$TRAVIS_BRANCH" == "dev" ]]; then $makefile_travis test; fi

after_success:
- export server_tag=$(cat $FOLDER.env |grep MISP_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export proxy_tag=$(cat $FOLDER.env |grep PROXY_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export robot_tag=$(cat $FOLDER.env |grep ROBOT_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export postfix_tag=$(cat $FOLDER.env |grep POSTFIX_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export modules_tag=$(cat $FOLDER.env |grep MISP_MODULES_CONTAINER|cut -d = -f 2|sed 's,....$,,')
- export server_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep server|cut -d ':' -f 3)
- export proxy_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep proxy|cut -d ':' -f 3)
- export robot_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep robot|cut -d ':' -f 3)
#- export postfix_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep server|cut -d ':' -f 3)
- export modules_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep modules|cut -d ':' -f 3)
- export db_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep db|cut -d ':' -f 3)
- export redis_tag=$(cat $FOLDER/docker-compose.yml |grep image|grep redis|cut -d ':' -f 3)
# retag all existing tags dev 2 public repo
- $makefile_travis tag server_tag=${server_tag} proxy_tag=${proxy_tag} robot_tag=${robot_tag} postfix_tag=${postfix_tag} modules_tag=${modules_tag}
- $makefile_travis tag server_tag=${server_tag} proxy_tag=${proxy_tag} robot_tag=${robot_tag} postfix_tag=${postfix_tag} modules_tag=${modules_tag} db_tag=${modules_tag} redis_tag=${modules_tag}
# check if branch=master and the current build is no pull request, then push it to docker hub
- if [[ "$TRAVIS_BRANCH" == "master" ]]; then
if [[ "$TRAVIS_PULL_REQUEST" == "false" ]]; then
Expand Down
17 changes: 8 additions & 9 deletions 1.0.0/.travis/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ pull-latest:
$(foreach c, $(CONTAINER), docker pull $(REPOURL)/misp-dockerized-$(c):latest-dev;)

tag:
./tagging.sh $(REPOURL)/misp-dockerized-server $(server_tag);
./tagging.sh $(REPOURL)/misp-dockerized-proxy $(proxy_tag);
./tagging.sh $(REPOURL)/misp-dockerized-robot $(robot_tag);
./tagging.sh $(REPOURL)/misp-dockerized-misp-modules $(modules_tag);
bash ./tagging.sh $(REPOURL)/misp-dockerized-server $(server_tag);
bash ./tagging.sh $(REPOURL)/misp-dockerized-proxy $(proxy_tag);
bash ./tagging.sh $(REPOURL)/misp-dockerized-robot $(robot_tag);
bash ./tagging.sh $(REPOURL)/misp-dockerized-misp-modules $(modules_tag);
@echo "###########################################"
docker images
@echo "###########################################"
Expand All @@ -36,8 +36,7 @@ test:
@docker cp misp-robot:/srv/MISP-dockerized-testbench/reports/. reports/

push:
./push.sh $(REPOURL)/misp-dockerized-server $(server_tag);
./push.sh $(REPOURL)/misp-dockerized-proxy $(proxy_tag);
./push.sh $(REPOURL)/misp-dockerized-robot $(robot_tag);
./push.sh $(REPOURL)/misp-dockerized-misp-modules $(modules_tag);

bash ./push.sh $(REPOURL)/misp-dockerized-server $(server_tag);
bash ./push.sh $(REPOURL)/misp-dockerized-proxy $(proxy_tag);
bash ./push.sh $(REPOURL)/misp-dockerized-robot $(robot_tag);
bash ./push.sh $(REPOURL)/misp-dockerized-misp-modules $(modules_tag);
2 changes: 1 addition & 1 deletion 1.0.0/.travis/main.sh
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash

# Set an option to exit immediately if any error appears
set -o errexit
set -e

# Main function that describes the behavior of the
# script.
Expand Down
2 changes: 1 addition & 1 deletion 1.0.0/.travis/tagging.sh
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
set -exv
set -ex

DOCKER_REPO="$1"
tag="$2"
Expand Down
19 changes: 13 additions & 6 deletions 1.0.0/scripts/build_config.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,11 @@ function default_container_version() {
PROXY_CONTAINER_TAG="$(cat $DOCKER_COMPOSE_FILE |grep image:|grep proxy|cut -d : -f 3)"
ROBOT_CONTAINER_TAG="$(cat $DOCKER_COMPOSE_FILE |grep image:|grep robot|cut -d : -f 3)"
MISP_MODULES_CONTAINER_TAG="$(cat $DOCKER_COMPOSE_FILE |grep image:|grep modules|cut -d : -f 3)"
#[ -z $(echo $POSTFIX_CONTAINER_TAG|grep dev) ] && POSTFIX_CONTAINER_TAG="$POSTFIX_CONTAINER_TAG-dev"
[ -z $(echo $MISP_CONTAINER_TAG|grep dev) ] && MISP_CONTAINER_TAG="$MISP_CONTAINER_TAG-dev"
[ -z $(echo $PROXY_CONTAINER_TAG|grep dev) ] && PROXY_CONTAINER_TAG="$PROXY_CONTAINER_TAG-dev"
[ -z $(echo $ROBOT_CONTAINER_TAG|grep dev) ] && ROBOT_CONTAINER_TAG="$ROBOT_CONTAINER_TAG-dev"
[ -z $(echo $MISP_MODULES_CONTAINER_TAG|grep dev) ] && MISP_MODULES_CONTAINER_TAG="$MISP_MODULES_CONTAINER_TAG-dev"
###
MISP_TAG=$(echo $MISP_CONTAINER_TAG|cut -d - -f 1)
###################### END GLOBAL ###########
Expand Down Expand Up @@ -436,12 +441,6 @@ if [ "$AUTOMATE_BUILD" = "true" ]
# ask no questions only defaults
echo "automatic build..."
####
#[ -z $(echo $POSTFIX_CONTAINER_TAG|grep dev) ] && POSTFIX_CONTAINER_TAG="$POSTFIX_CONTAINER_TAG-dev"
[ -z $(echo $MISP_CONTAINER_TAG|grep dev) ] && MISP_CONTAINER_TAG="$MISP_CONTAINER_TAG-dev"
[ -z $(echo $PROXY_CONTAINER_TAG|grep dev) ] && PROXY_CONTAINER_TAG="$PROXY_CONTAINER_TAG-dev"
[ -z $(echo $ROBOT_CONTAINER_TAG|grep dev) ] && ROBOT_CONTAINER_TAG="$ROBOT_CONTAINER_TAG-dev"
[ -z $(echo $MISP_MODULES_CONTAINER_TAG|grep dev) ] && MISP_MODULES_CONTAINER_TAG="$MISP_MODULES_CONTAINER_TAG-dev"

# set hostname to an fix one
myHOSTNAME="misp.example.com"
IMAGE_MISP_MODULES="image: ${DOCKER_REGISTRY}/misp-dockerized-misp-modules:${MISP_MODULES_CONTAINER_TAG}"
Expand Down Expand Up @@ -473,6 +472,14 @@ if [ "$AUTOMATE_BUILD" = "true" ]
[ "$QUERY_PGP" == "yes" ] && query_pgp_settings
# LOG_SETTINGS
[ "$QUERY_LOG_SETTINGS" == "yes" ] && query_log_settings

if [ "$DEV" == true ]
then
IMAGE_MISP_MODULES="image: ${DOCKER_REGISTRY}/misp-dockerized-misp-modules:${MISP_MODULES_CONTAINER_TAG}"
IMAGE_MISP_SERVER="image: ${DOCKER_REGISTRY}/misp-dockerized-server:${MISP_CONTAINER_TAG}"
IMAGE_MISP_PROXY="image: ${DOCKER_REGISTRY}/misp-dockerized-proxy:${PROXY_CONTAINER_TAG}"
IMAGE_MISP_ROBOT="image: ${DOCKER_REGISTRY}/misp-dockerized-robot:${ROBOT_CONTAINER_TAG}"
fi
fi
###################################
# Write Configuration
Expand Down

0 comments on commit a30bc74

Please sign in to comment.