Updated 5/11/2023
When I am testing for various issues I find that I need to create some test data. The built in sample data is great but sometimes you need to keep ingesting data or ingest some older data to reproduce issues. I found this repo and it was the answers to my issue.
I did not want to install this and wanted to containerize it so that I can leave my system clean yet be able to use this and re-use it again.
makelogs runs fine as it is however if your ES endpoint is secured with TLS and does not have proper certificates it makelogs will fail.
Please follow the steps below to create the docker image
$ git clone https://github.com/elastic/makelogs.git $ cd makelogs
NOTE: below step is no longer needed as source was updated to add --insecure flag
##########
We will be editing _client.js.
Find the line
host: url
and replace it with
host: url,
ssl:{ rejectUnauthorized: false, pfx: [] }
##########
Now run the following to build the docker image
$ docker build -t makelogs . [+] Building 14.8s (10/10) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 37B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [internal] load metadata for docker.io/library/node:16.13.0 1.0s => [auth] library/node:pull token for registry-1.docker.io 0.0s => [internal] load build context 0.0s => => transferring context: 4.72kB 0.0s => CACHED [1/4] FROM docker.io/library/node:16.13.0@sha256:580a0850049c59a48f06090edd48c9f966c5e6572bbbabc369ba3ecbc4855dba 0.0s => [2/4] COPY / /app 0.0s => [3/4] WORKDIR /app 0.0s => [4/4] RUN yarn install 12.3s => exporting to image 1.4s => => exporting layers 1.4s => => writing image sha256:d181f96bb76508bac1e84fe806aeeb8d95c609c0b816e27c0665d6cae62f4dd5 0.0s => => naming to docker.io/library/makelogs
Some examples on how to use this.
Lets say that our endpoint is https://192.168.1.10:9200 and our login is elastic:password for all of our examples
- To get help
docker run --rm makelogs --help - To connect and send events
docker run --url "https://elastic:password@192.168.1.10:9200"
Various options you can add and remove
-c- defaults to 14000 documents but you can specify it-c 1tfor 1k records-d- defaults to today but you can setX/Ywhere X is number of days before and Y is number of days after--indexPrefix- defaultlogstash-however you can specify it-s- default 1 - number of shards-r- default 0 - number of replica shards--dry- dry run--reset- clear allindexPrefix-*indicies and recreate--no-reset- does not reset but just adds to the existingindexPrefix-*--insecure- if your endpoint is self-signed or private
Sample run
$ docker run --rm makelogs --url "https://elastic:password@192.168.1.10:9200" --reset --indexPrefix "test-" Generating 14000 events from 2022-01-25T00:00:00Z to 2022-01-27T23:59:59Z --insecure creating index template for "test-*" created 14000 events in 10 seconds.
If you are using my deploy-elastic.sh script to run your ES cluster in docker containers please use your host's IP address to run the makelogs.
$ docker run --rm makelogs --url "https://elastic:password@192.168.1.10:9200" --indexPrefix "testagain-" -c 1t --days "10/10" --insecure Generating 1000 events from 2022-01-16T00:00:00Z to 2022-02-05T23:59:59Z creating index template for "testagain-*" created 1000 events in 1 seconds.
