logstash

2

New elastic kubernetes script – deploy-elastick8s.sh

Posted on by jlim0930

The old deploy-eck.sh script have grown and matured a bit and now it encompasses helm charts and native installs. I renamed the script to be more inclusive of its features. All of the older articles will reference this post. The new script is located deploy-elastick8s.sh This script was designed to easily stand up elastic and its stack on kubernetes in various modes to easily test and replicate issues for troubleshooting and to setup examples for various workloads. The deployments that this script creates is not for production use but to be used as lab or test environments. This post will… Continue Reading →

2

Fun things with Elastic’s Fleet server & elastic-agent

Posted on by jlim0930

There are many fun things we can do with the fleet server & elastic-agent To understand how to edit and change settings to do fun things with fleet server, first you have to understand how fleet server works with kibana & elasticsearch and elastic-agent. Fleet server itself is elastic-agent that runs in a server mode. It needs to communicate with both kibana and elasticsearch as well as elastic-agent. It uses elasticsearch as its config manager to store configurations so that it can configure & keep track of elastic-agents that is registered against it. Kibana is used to configure fleet server,… Continue Reading →

2

Fleet server with logstash output – elastic-agent

Posted on by jlim0930

I think one of the biggest issue with fleet and elastic-agent was that it was limited on the outputs so if you had tons and tons of elastic-agents in the wild it would all connect back to your elasticsearch and can overwhelm the cluster. Starting 8.2 of elasticsearch logstash output type was introduced.. it is still in BETA at this time but I just tried it and it worked very nicely! The steps to setup the logstash output is listed on https://www.elastic.co/guide/en/fleet/8.2/secure-logstash-connections.html but I wanted to try it out and document it. Install & configure logstash Install yum install logstash-8.2.2… Continue Reading →

0

Generate test data using elastic’s makelogs

Posted on by jlim0930

When I am testing for various issues I find that I need to create some test data. The built in sample data is great but sometimes you need to keep ingesting data or ingest some older data to reproduce issues. I found this repo and it was the answers to my issue. I did not want to install this and wanted to containerize it so that I can leave my system clean yet be able to use this and re-use it again. makelogs runs fine as it is however if your ES endpoint is secured with TLS and does not… Continue Reading →

1

kafka – stand up a test instance with SSL for testing

Posted on by jlim0930

I had a need to stand up a Kafka instance with SSL to test SSL handshakes. Today was the first time looking at Kafka so needless to say I was a bit lost. 🙁 Started out looking up various projects and found some interesting things that enabled me to setup a instance of kafka using docker containers with SSL. Requirements: git, docker, docker-compose installed on your server. Clone wurstmeister/kafka repo Get confluent’s kafka-generate-ssl.sh script, run it and follow all the instructions. Please make sure to remember the passphrase and the truststore/keystore passwords. This will generate the following Create “certs” directory… Continue Reading →