0

How to run setup for various beats(filebeat, metricbeat, & more) in kubernetes to load dashboards and more

Whenever you install beats or update beats its best if you run the setup so that the setup will update your index templates, ILM, visualizations, etc. I found that its always best before installing or upgrading to use a seed host to just run the setup before deploying or updating across your environment and to turn off template updates and dashboard loading on the beats locally so that your elasticsearch cluster is not flooded.

In a baremetal environment this is easy to do but in docker and in kubernetes it gets a bit difficult.

Sure you can setup a initContainer for your beats to run the setup before starting your pods however the initContainer will run each time there is a change or update so if you have a lot of different beats deployed in a lot of places or if you are running this as a Beat or DaemonSet it can potentially flood your elasticsearch cluster if you have a lot of hosts.

So what can you do?

There are many ways to do this but this method worked for me.

In Docker you can run a docker image to run commands and let the container exit once the command is complete. You can also do similar in kubernetes.

This will work on all beats - filebeat, metricbeat, heartbeat, journalbeat, packetbeat, etc.

Lets go ahead and create a quick test:

Create a simple elastic stack

kubectl create -f https://download.elastic.co/downloads/eck/1.9.1/crds.yaml
kubectl apply -f https://download.elastic.co/downloads/eck/1.9.1/operator.yaml

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.17.0
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
EOF

cat <<EOF | kubectl apply -f -
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: quickstart
spec:
  version: 7.17.0
  count: 1
  elasticsearchRef:
    name: quickstart
EOF

kubectl get pod,svc
NAME                                 READY   STATUS    RESTARTS   AGE
pod/quickstart-es-default-0          1/1     Running   0          3m7s
pod/quickstart-kb-6d86f77f4b-fgjnn   1/1     Running   0          3m6s

NAME                              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/kubernetes                ClusterIP   10.96.0.1       <none>        443/TCP    4m33s
service/quickstart-es-default     ClusterIP   None            <none>        9200/TCP   3m7s
service/quickstart-es-http        ClusterIP   10.104.56.200   <none>        9200/TCP   3m10s
service/quickstart-es-transport   ClusterIP   None            <none>        9300/TCP   3m10s
service/quickstart-kb-http        ClusterIP   10.99.77.239    <none>        5601/TCP   3m10s

Grab the elastic users password:

kubectl get secrets

NAME                                       TYPE                                  DATA   AGE
default-quickstart-kibana-user             Opaque                                3      3m45s
default-token-tb4f8                        kubernetes.io/service-account-token   3      4m51s
quickstart-es-default-es-config            Opaque                                1      3m42s
quickstart-es-default-es-transport-certs   Opaque                                3      3m43s
quickstart-es-elastic-user                 Opaque                                1      3m43s
quickstart-es-http-ca-internal             Opaque                                2      3m44s
quickstart-es-http-certs-internal          Opaque                                3      3m43s
quickstart-es-http-certs-public            Opaque                                2      3m43s
quickstart-es-internal-users               Opaque                                3      3m42s
quickstart-es-remote-ca                    Opaque                                1      3m43s
quickstart-es-transport-ca-internal        Opaque                                2      3m43s
quickstart-es-transport-certs-public       Opaque                                1      3m43s
quickstart-es-xpack-file-realm             Opaque                                3      3m42s
quickstart-kb-config                       Opaque                                1      3m41s
quickstart-kb-es-ca                        Opaque                                2      3m43s
quickstart-kb-http-ca-internal             Opaque                                2      3m43s
quickstart-kb-http-certs-internal          Opaque                                3      3m43s
quickstart-kb-http-certs-public            Opaque                                2      3m42s
quickstart-kibana-user                     Opaque                                1      3m45s

PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')

echo $PASSWORD
43Q0G7Y6529VBQkSn0oQam2J

Test to ensure that we are good to go

kubectl port-forward service/quickstart-es-http 9200
Forwarding from 127.0.0.1:9200 -> 9200
Forwarding from [::1]:9200 -> 9200

curl -u "elastic:$PASSWORD" -k "https://localhost:9200"
{
  "name" : "quickstart-es-default-0",
  "cluster_name" : "quickstart",
  "cluster_uuid" : "CJGwkSjeQVm48YfQAhXWmg",
  "version" : {
    "number" : "7.17.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "bee86328705acaa9a6daede7140defd4d9ec56bd",
    "build_date" : "2022-01-28T08:36:04.875279988Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Awesome! now we have a working elasticsearch cluster and we know the elastic users password and we also know CLUSTER-IP's and PORTS to access both elasticsearch and kibana

For this example we will use filebeats but this will work with any beats

Run a seed pod so that we can run the setup

 kubectl run -i --tty filebeat-seed --image=docker.elastic.co/beats/filebeat:7.17.0 -- sh
If you don't see a command prompt, try pressing enter.

$

This will take a while to run since the container image needs to be pulled first.

From https://www.elastic.co/guide/en/beats/filebeat/current/command-line-options.html#setup-command we see that there are various options you can add to the setup but we will just run the setup so that it will load the defaults.

Plug in all the information and run

./filebeat setup -e \
-E output.logstash.enabled=false \
-E output.elasticsearch.hosts=['10.104.56.200:9200'] \
-E output.elasticsearch.protocol=https \
-E output.elasticsearch.username=elastic \
-E output.elasticsearch.password=43Q0G7Y6529VBQkSn0oQam2J \
-E output.elasticsearch.ssl.verification_mode=none \
-E setup.kibana.host='10.99.77.239:5601' \
-E setup.kibana.protocol=https \
-E setup.kibana.username=elastic \
-E setup.kibana.password=43Q0G7Y6529VBQkSn0oQam2J \
-E setup.kibana.ssl.verification_mode=none

2022-02-09T04:22:28.972Z    INFO    instance/beat.go:686    Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs] Hostfs Path: [/]
2022-02-09T04:22:28.972Z    INFO    instance/beat.go:694    Beat ID: cef84f35-59a9-4fbc-baf8-ef3fbfbbf4ab
2022-02-09T04:22:31.974Z    WARN    [add_cloud_metadata]    add_cloud_metadata/provider_aws_ec2.go:79   read token request for getting IMDSv2 token returns empty: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.
2022-02-09T04:22:31.975Z    INFO    [beat]  instance/beat.go:1040   Beat info   {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "cef84f35-59a9-4fbc-baf8-ef3fbfbbf4ab"}}}
2022-02-09T04:22:31.975Z    INFO    [beat]  instance/beat.go:1049   Build info  {"system_info": {"build": {"commit": "93708bd74e909e57ed5d9bea3cf2065f4cc43af3", "libbeat": "7.17.0", "time": "2022-01-28T09:53:30.000Z", "version": "7.17.0"}}}
2022-02-09T04:22:31.975Z    INFO    [beat]  instance/beat.go:1052   Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":8,"version":"go1.17.5"}}}
2022-02-09T04:22:31.976Z    INFO    [beat]  instance/beat.go:1056   Host info   {"system_info": {"host": {"architecture":"x86_64","boot_time":"2022-01-16T03:22:05Z","containerized":true,"name":"filebeat-seed","ip":["127.0.0.1/8","172.17.0.7/16"],"kernel_version":"3.10.0-1160.49.1.el7.x86_64","mac":["02:42:ac:11:00:07"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.3 LTS (Focal Fossa)","major":20,"minor":4,"patch":3,"codename":"focal"},"timezone":"UTC","timezone_offset_sec":0}}}
2022-02-09T04:22:31.976Z    INFO    [beat]  instance/beat.go:1085   Process info    {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 31, "ppid": 7, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2022-02-09T04:22:28.150Z"}}}
2022-02-09T04:22:31.976Z    INFO    instance/beat.go:328    Setup Beat: filebeat; Version: 7.17.0
2022-02-09T04:22:31.976Z    INFO    [index-management]  idxmgmt/std.go:184  Set output.elasticsearch.index to 'filebeat-7.17.0' as ILM is enabled.
2022-02-09T04:22:31.976Z    WARN    [cfgwarn]   tlscommon/config.go:100 DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0
2022-02-09T04:22:31.977Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.104.56.200:9200
2022-02-09T04:22:31.977Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:22:31.977Z    INFO    [publisher] pipeline/module.go:113  Beat name: filebeat-seed
2022-02-09T04:22:31.978Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.104.56.200:9200
2022-02-09T04:22:31.978Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:22:31.978Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:22:31.987Z    INFO    [esclientleg]   eslegclient/connection.go:284   Attempting to connect to Elasticsearch version 7.17.0
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

2022-02-09T04:22:32.024Z    INFO    [index-management]  idxmgmt/std.go:261  Auto ILM enable success.
2022-02-09T04:22:32.030Z    INFO    [index-management.ilm]  ilm/std.go:170  ILM policy filebeat exists already.
2022-02-09T04:22:32.030Z    INFO    [index-management]  idxmgmt/std.go:397  Set setup.template.name to '{filebeat-7.17.0 {now/d}-000001}' as ILM is enabled.
2022-02-09T04:22:32.030Z    INFO    [index-management]  idxmgmt/std.go:402  Set setup.template.pattern to 'filebeat-7.17.0-*' as ILM is enabled.
2022-02-09T04:22:32.030Z    INFO    [index-management]  idxmgmt/std.go:436  Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.17.0 {now/d}-000001} as ILM is enabled.
2022-02-09T04:22:32.030Z    INFO    [index-management]  idxmgmt/std.go:440  Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2022-02-09T04:22:32.047Z    INFO    template/load.go:197    Existing template will be overwritten, as overwrite is enabled.
2022-02-09T04:22:33.667Z    INFO    template/load.go:131    Try loading template filebeat-7.17.0 to Elasticsearch
2022-02-09T04:22:33.744Z    INFO    template/load.go:123    Template with name "filebeat-7.17.0" loaded.
2022-02-09T04:22:33.744Z    INFO    [index-management]  idxmgmt/std.go:297  Loaded index template.
2022-02-09T04:22:33.746Z    INFO    [index-management.ilm]  ilm/std.go:126  Index Alias filebeat-7.17.0 exists already.
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
2022-02-09T04:22:33.747Z    INFO    kibana/client.go:180    Kibana url: https://10.99.77.239:5601
2022-02-09T04:22:33.747Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:22:33.747Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:22:34.975Z    INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:101    add_cloud_metadata: hosting provider type not detected.
2022-02-09T04:22:35.877Z    INFO    kibana/client.go:180    Kibana url: https://10.99.77.239:5601
2022-02-09T04:22:35.877Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:22:35.877Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.

2022-02-09T04:23:54.220Z    INFO    instance/beat.go:869    Kibana dashboards successfully loaded.
Loaded dashboards
2022-02-09T04:23:54.220Z    WARN    [cfgwarn]   instance/beat.go:594    DEPRECATED: Setting up ML using Filebeat is going to be removed. Please use the ML app to setup jobs. Will be removed in version: 8.0.0
Setting up ML using setup --machine-learning is going to be removed in 8.0.0. Please use the ML app instead.
See more: https://www.elastic.co/guide/en/machine-learning/current/index.html
It is not possble to load ML jobs into an Elasticsearch 8.0.0 or newer using the Beat.
2022-02-09T04:23:54.221Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.104.56.200:9200
2022-02-09T04:23:54.221Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:23:54.221Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:23:54.230Z    INFO    [esclientleg]   eslegclient/connection.go:284   Attempting to connect to Elasticsearch version 7.17.0
2022-02-09T04:23:54.230Z    INFO    kibana/client.go:180    Kibana url: https://10.99.77.239:5601
2022-02-09T04:23:54.230Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:23:54.230Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:23:54.330Z    WARN    fileset/modules.go:463  X-Pack Machine Learning is not enabled
Loaded machine learning job configurations
2022-02-09T04:23:54.333Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.104.56.200:9200
2022-02-09T04:23:54.333Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:23:54.333Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-09T04:23:54.342Z    INFO    [esclientleg]   eslegclient/connection.go:284   Attempting to connect to Elasticsearch version 7.17.0
2022-02-09T04:23:54.342Z    INFO    cfgfile/reload.go:262   Loading of config files completed.
Loaded Ingest pipelines
 $ exit

Session ended, resume using 'kubectl attach filebeat-seed -c filebeat-seed -i -t' command when the pod is running

 kubectl get pod

NAME                             READY   STATUS    RESTARTS   AGE
filebeat-seed                    1/1     Running   1          9m50s
quickstart-es-default-0          1/1     Running   0          26m
quickstart-kb-6d86f77f4b-fgjnn   1/1     Running   0          26m

kubectl delete pod filebeat-seed
pod "filebeat-seed" deleted

Went ahead and cleanup and deleted the pod.

Alternatively, you can edit filebeat.yml and just run ./filebeat setup -e if you don't want to pass all the env vars.

Now we can log into kibana and see that we have templates/ILM/dashboards!

file

file

file

file

ALTERNATIVE 1

Thanks to my friend Camilo Sierra! you can also perform the same using kubernets jobs

Create the manifest

apiVersion: batch/v1
kind: Job
metadata:
  name: seed-filebeat
spec:
  template:
    spec:
      containers:
      - name: seed-filebeat
        image: docker.elastic.co/beats/filebeat:7.17.0
        command: ["/bin/sh", "-c"]
        args: ["filebeat setup -e -E output.elasticsearch.hosts=['10.103.121.252:9200'] -E output.elasticsearch.protocol=https -E output.elasticsearch.username=elastic -E output.elasticsearch.password=d71gR579ON53p3rc7C2IHcVa -E output.elasticsearch.ssl.verification_mode=none -E setup.kibana.host='10.97.152.126:5601' -E setup.kibana.protocol=https -E setup.kibana.username=elastic -E setup.kibana.password=d71gR579ON53p3rc7C2IHcVa -E setup.kibana.ssl.verification_mode=none"]
      restartPolicy: Never
  backoffLimit: 1
  • side note: endpoints and the passwords are different in this example since this is a new test

Create the job

kubectl apply -f job.yaml

Once the job is applied it will create a pod

 kubectl get pod

NAME                             READY   STATUS      RESTARTS   AGE
quickstart-es-default-0          1/1     Running     0          48m
quickstart-kb-57955bbc9f-g9htz   1/1     Running     0          48m
seed-filebeat-zxggl              0/1     Completed   0          6m20s

Lets look at the logs from the pod

kubectl logs -f seed-filebeat-zxggl

2022-02-10T18:50:57.646Z    INFO    instance/beat.go:686    Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs] Hostfs Path: [/]
2022-02-10T18:50:57.650Z    INFO    instance/beat.go:694    Beat ID: 74377805-61a5-4d2f-af19-3e4b961cbafb
2022-02-10T18:51:00.652Z    WARN    [add_cloud_metadata]    add_cloud_metadata/provider_aws_ec2.go:79   read token request for getting IMDSv2 token returns empty: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.
2022-02-10T18:51:00.654Z    INFO    [beat]  instance/beat.go:1040   Beat info   {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "74377805-61a5-4d2f-af19-3e4b961cbafb"}}}
2022-02-10T18:51:00.654Z    INFO    [beat]  instance/beat.go:1049   Build info  {"system_info": {"build": {"commit": "93708bd74e909e57ed5d9bea3cf2065f4cc43af3", "libbeat": "7.17.0", "time": "2022-01-28T09:53:30.000Z", "version": "7.17.0"}}}
2022-02-10T18:51:00.654Z    INFO    [beat]  instance/beat.go:1052   Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":8,"version":"go1.17.5"}}}
2022-02-10T18:51:00.656Z    INFO    [beat]  instance/beat.go:1056   Host info   {"system_info": {"host": {"architecture":"x86_64","boot_time":"2022-01-16T03:22:05Z","containerized":true,"name":"seed-filebeat-zxggl","ip":["127.0.0.1/8","172.17.0.7/16"],"kernel_version":"3.10.0-1160.49.1.el7.x86_64","mac":["02:42:ac:11:00:07"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.3 LTS (Focal Fossa)","major":20,"minor":4,"patch":3,"codename":"focal"},"timezone":"UTC","timezone_offset_sec":0}}}
2022-02-10T18:51:00.657Z    INFO    [beat]  instance/beat.go:1085   Process info    {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 7, "ppid": 1, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2022-02-10T18:50:56.810Z"}}}
2022-02-10T18:51:00.657Z    INFO    instance/beat.go:328    Setup Beat: filebeat; Version: 7.17.0
2022-02-10T18:51:00.658Z    INFO    [index-management]  idxmgmt/std.go:184  Set output.elasticsearch.index to 'filebeat-7.17.0' as ILM is enabled.
2022-02-10T18:51:00.658Z    WARN    [cfgwarn]   tlscommon/config.go:100 DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0
2022-02-10T18:51:00.659Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.103.121.252:9200
2022-02-10T18:51:00.660Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:51:00.660Z    INFO    [publisher] pipeline/module.go:113  Beat name: seed-filebeat-zxggl
2022-02-10T18:51:00.664Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.103.121.252:9200
2022-02-10T18:51:00.664Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:51:00.665Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:51:00.798Z    INFO    [esclientleg]   eslegclient/connection.go:284   Attempting to connect to Elasticsearch version 7.17.0
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

2022-02-10T18:51:00.839Z    INFO    [index-management]  idxmgmt/std.go:261  Auto ILM enable success.
2022-02-10T18:51:00.878Z    INFO    [index-management.ilm]  ilm/std.go:180  ILM policy filebeat successfully created.
2022-02-10T18:51:00.879Z    INFO    [index-management]  idxmgmt/std.go:397  Set setup.template.name to '{filebeat-7.17.0 {now/d}-000001}' as ILM is enabled.
2022-02-10T18:51:00.879Z    INFO    [index-management]  idxmgmt/std.go:402  Set setup.template.pattern to 'filebeat-7.17.0-*' as ILM is enabled.
2022-02-10T18:51:00.879Z    INFO    [index-management]  idxmgmt/std.go:436  Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.17.0 {now/d}-000001} as ILM is enabled.
2022-02-10T18:51:00.879Z    INFO    [index-management]  idxmgmt/std.go:440  Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2022-02-10T18:51:00.881Z    INFO    template/load.go:197    Existing template will be overwritten, as overwrite is enabled.
2022-02-10T18:51:02.619Z    INFO    template/load.go:131    Try loading template filebeat-7.17.0 to Elasticsearch
2022-02-10T18:51:02.988Z    INFO    template/load.go:123    Template with name "filebeat-7.17.0" loaded.
2022-02-10T18:51:02.988Z    INFO    [index-management]  idxmgmt/std.go:297  Loaded index template.
2022-02-10T18:51:03.624Z    INFO    [index-management.ilm]  ilm/std.go:140  Index Alias filebeat-7.17.0 successfully created.
2022-02-10T18:51:03.624Z    INFO    kibana/client.go:180    Kibana url: https://10.97.152.126:5601
2022-02-10T18:51:03.624Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:51:03.624Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
2022-02-10T18:51:03.655Z    INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:101    add_cloud_metadata: hosting provider type not detected.
2022-02-10T18:51:05.824Z    INFO    kibana/client.go:180    Kibana url: https://10.97.152.126:5601
2022-02-10T18:51:05.824Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:51:05.824Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
Loaded dashboards
Setting up ML using setup --machine-learning is going to be removed in 8.0.0. Please use the ML app instead.
See more: https://www.elastic.co/guide/en/machine-learning/current/index.html
It is not possble to load ML jobs into an Elasticsearch 8.0.0 or newer using the Beat.
2022-02-10T18:52:24.446Z    INFO    instance/beat.go:869    Kibana dashboards successfully loaded.
2022-02-10T18:52:24.446Z    WARN    [cfgwarn]   instance/beat.go:594    DEPRECATED: Setting up ML using Filebeat is going to be removed. Please use the ML app to setup jobs. Will be removed in version: 8.0.0
2022-02-10T18:52:24.446Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.103.121.252:9200
2022-02-10T18:52:24.447Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:52:24.447Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:52:24.456Z    INFO    [esclientleg]   eslegclient/connection.go:284   Attempting to connect to Elasticsearch version 7.17.0
2022-02-10T18:52:24.456Z    INFO    kibana/client.go:180    Kibana url: https://10.97.152.126:5601
2022-02-10T18:52:24.456Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:52:24.456Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:52:24.550Z    WARN    fileset/modules.go:463  X-Pack Machine Learning is not enabled
Loaded machine learning job configurations
2022-02-10T18:52:24.552Z    INFO    [esclientleg]   eslegclient/connection.go:105   elasticsearch url: https://10.103.121.252:9200
2022-02-10T18:52:24.552Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:52:24.552Z    WARN    [tls]   tlscommon/tls_config.go:101 SSL/TLS verifications disabled.
2022-02-10T18:52:24.561Z    INFO    [esclientleg]   eslegclient/connection.go:284   Attempting to connect to Elasticsearch version 7.17.0
2022-02-10T18:52:24.561Z    INFO    cfgfile/reload.go:262   Loading of config files completed.
Loaded Ingest pipelines

You can add/change the setup to just load individual or module specfic items as well.

To clean up the pod and the job just delete the job

kubectl delete -f job.yaml

ALTERNATIVE 2

This is similar to the filebeat-seed method but you can do this with just 2 commands

$ kubectl run -it beats-setup --image=docker.elastic.co/beats/${1}:${VERSION} -- sh -c "${1} setup -E output.elasticsearch.hosts=\"${ESIP}:9200\" -E output.elasticsearch.protocol=https -E output.elasticsearch.username=elastic -E output.elasticsearch.password=${PASSWORD} -E output.elasticsearch.ssl.verification_mode=none -E setup.kibana.host=\"${KIBANAIP}:5601\" -E setup.kibana.protocol=https -E setup.kibana.username=elastic -E setup.kibana.password=${PASSWORD} -E setup.kibana.ssl.verification_mode=none -E setup.ilm.overwrite=true"

$ kubectl delete pod beats-setup

where:

  • ${1} is the type of beat like filebeat, metricbeat
  • ${VERSION} is the version of your stack
  • ${ESIP} is the ES endpoint
  • ${PASSWORD} is the password for the elastic user
  • ${KIBANAIP} is the KB endpoint

jlim0930

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.