0

TROUBLESHOOT swiss-army knife container image

Posted on by jlim0930

I created a container image to help me troubleshoot issues for both docker and kubernetes. Many container images come very slim without the default OS tools so when you are troubleshooting it makes it very hard. With this image you can easily deploy this container onto the same network as your containers/pods or even as a sidecar container or even map the host network to troubleshoot host network issues. github link: https://github.com/jlim0930/troubleshoot docker hub link: https://hub.docker.com/r/jlim0930/troubleshoot TROUBLESHOOT Container to help troubleshoot issues for docker & kubernetes environments This container image was created to help troubleshoot various network/storage/OS issues inside of… Continue Reading →

1

ESXi on old hardware Error 10 – Out of resources failed to malloc MMIO

Posted on by jlim0930

At home I use a old Dell Precision T5600. It has 16 cores total with 128GB of memory and while old its still good for running multiple workloads via ESXi. In the 6.x days I had no issues with ESXi, however with 7.0 I now get the CPU is unsupported as well as many other warnings. However I was able to run 7.0 Update 2 build 17867351 without issues. When I first installed ESXi I used BIOS to install and to run it since it never worked with EFI and at the time I did not bother to figure out… Continue Reading →

2

New elastic kubernetes script – deploy-elastick8s.sh

Posted on by jlim0930

Changelogs 2023/02/26 Added legacy APM server integration The old deploy-eck.sh script have grown and matured a bit and now it encompasses helm charts and native installs. I renamed the script to be more inclusive of its features. All of the older articles will reference this post. The new script is located deploy-elastick8s.sh This script was designed to easily stand up elastic and its stack on kubernetes in various modes to easily test and replicate issues for troubleshooting and to setup examples for various workloads. The deployments that this script creates is not for production use but to be used as… Continue Reading →

0

How to increase space for your elasticsearch instances in k8s on ECK

Posted on by jlim0930

One of the most common issues of running elasticsearch on k8s is the need to increase space for your elasticsearch-data volume. It is very simple to do so lets demo it. I stood up an environment using my deploy-eck.sh script $ kubectl get pods,pvc NAME READY STATUS RESTARTS AGE pod/eck-lab-es-data-0 1/1 Running 0 112m pod/eck-lab-es-data-1 1/1 Running 0 112m pod/eck-lab-es-data-2 1/1 Running 0 112m pod/eck-lab-es-master-0 1/1 Running 0 112m pod/eck-lab-es-master-1 1/1 Running 0 112m pod/eck-lab-es-master-2 1/1 Running 0 8m12s pod/eck-lab-kb-794785d7f7-zxqlk 1/1 Running 0 110m NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/elasticsearch-data-eck-lab-es-data-0 Bound pvc-4081bb0d-1664-46fe-97bd-08520075bbdc 1Gi RWO standard 112m persistentvolumeclaim/elasticsearch-data-eck-lab-es-data-1… Continue Reading →

2

Fun things with Elastic’s Fleet server & elastic-agent

Posted on by jlim0930

There are many fun things we can do with the fleet server & elastic-agent To understand how to edit and change settings to do fun things with fleet server, first you have to understand how fleet server works with kibana & elasticsearch and elastic-agent. Fleet server itself is elastic-agent that runs in a server mode. It needs to communicate with both kibana and elasticsearch as well as elastic-agent. It uses elasticsearch as its config manager to store configurations so that it can configure & keep track of elastic-agents that is registered against it. Kibana is used to configure fleet server,… Continue Reading →

1

deploy-eck update 1

Posted on by jlim0930

OUTDATED – Please use this link for the new script and instructions https://www.gooksu.com/2022/09/new-elastic-kubernetes-script-deploy-elastick8s-sh/ I developed the deploy-eck.sh script to easily deploy the elastic stack in k8s to quickly test things. Did a quick update to the script. operator mode. Now you can run the script just to deploy the operator only and apply a trial license. You can develop your own test cases etc in this mode. If you work out of ~/eckstack and name your manifest files *.yaml when you run the deploy-eck.sh cleanup it will cleanup all of your items as well. > ./deploy-eck.sh cleanup ********** Cleaning up… Continue Reading →

0

Install elasticsearch using enrollment tokens

Posted on by jlim0930

I have not installed elasticsearch using rpm/tarball in a super long time since I use my deploy-elastic.sh script to install elasticsearch onto docker containers. Decided to have a look today on the new way of standing up a cluster using enrollment tokens. Following the steps from https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html – a bit adapated for my liking. Install elasticsearch onto first host: import GPG keys # rpm –import https://artifacts.elastic.co/GPG-KEY-elasticsearch create /etc/yum.repos.d/elasticsearch.repo [elasticsearch] name=Elasticsearch repository for 8.x packages baseurl=https://artifacts.elastic.co/packages/8.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md install elasticsearch via yum # yum install elasticsearch -y … Dependencies Resolved ================================================================================================================================================================================================================== Package Arch Version Repository Size ================================================================================================================================================================================================================== Installing:… Continue Reading →

0

Mounting certificates/CAs for elasticsearch pods in k8s for custom configurations – ECK

Posted on by jlim0930

When you deploy elasticsearch in k8s via ECK by default it takes care of the transport certificates/http certificates/CA creation if you don’t use your own custom ones. Lets say that your cluster is up and running and now you are trying to configure something like SAML/LDAP/OIDC or something else that requires content to be made available to the elasticsearch pod but its a bit sensitive so you need it to be stored as a secret such as CA, certificates, etc. How would you go about doing this? Following example will add a custom certificate authority to be used with your… Continue Reading →

0

Updated deploy-elastic script – version 9

Posted on by jlim0930

Last update to version 8 is listed here The original post for the deploy-elastic.sh script is here Changes: Starting 7.12.x added node.roles. es01 & es02 is tagged as data_content & data_hot and es03 is tagged as data_frozen. This will allow users to play with the frozen tier, configure searchable snapshots, partically mounted indices, restored indices, and more. (make sure to run the script with snapshot option so that minio is stood up) LDAP mode was added. To test various role_mappings and for basic ldap authentication configurations and settings LDAP mode was added and it will work with any versions of… Continue Reading →

1

Elastic Fleet Server & elastic-agent common troubleshooting

Posted on by jlim0930

Common troubleshooting items for any fleet & elastic-agent issues can be found on https://www.elastic.co/guide/en/fleet/current/fleet-troubleshooting.html however there are a few items that I would like to add. Whitespaces When you start the elastic-agent on an endpoint and even if you have everything configured for your elastic-agent, the right policy, integrations, etc however no data is going into elasticsearch. The most common issue is that there is something wrong with your fleet settings. This can be edited from kibana -> fleet -> settings. Either your ES endpoint / Fleet server endpoint / Advanced YAML settings are wrong or there might even be… Continue Reading →