docker log collector

Collect all logs from other docker containers with label dockerLogCollector.logAppName and post the logs to solr server.

Usage

docker run -it -v /var/run/docker.sock:/var/run/docker.sock --link solrContainerNameOrId:solr -e TZ="Asia/Shanghai" -e SOLR_PORT=80 -e SOLR_HOST=solr dbjtech/docker-log-collector

then start other docker

docker run -l dockerLogCollector.logAppName=aNodeJSApp node:alpine node -e "setInterval(()=>console.log(new Date()), 1000)"

Config

Check the following code

const config = _.defaults({
	verbose: process.env.VERBOSE,
	solrHost: process.env.SOLR_HOST,
	solrPort: process.env.SOLR_PORT,
	solrCollectionName: process.env.SOLR_COLLECTION_NAME,
	logHostName: process.env.LOG_HOST_NAME,
	logMaxCount: process.env.LOG_MAX_COUNT,
	logMaxTime: process.env.LOG_MAX_TIME,
}, {
	verbose: false, // print the logs it collects
	solrHost: 'localhost', // solr host
	solrPort: 80, // solr port
	solrCollectionName: 'collection1', // solr collection name
	logHostName: 'UNKNOW_HOST', // machine name set to the doc
	logMaxCount: 1000, // post a doc when collect more than logMaxCount lines
	logMaxTime: 30000, // post a doc when now > lastPostTime + logMaxTime, unit ms
})

Time parser

Collector use the following code to parse log time, Using TZ env variable to indicate timezone.

If the log contain no time string with the format, it use localtime.

moment.tz(aTimeString, 'YYMMDD HH:mm:ss', process.env.TZ || 'UTC')

Notice

• container run without dockerLogCollector.logAppName label will be ignore
• new created container will be add to the watching list automatically