Spring Boot Microservices + ELK(Elasticsearch, Logstash, and Kibana) Stack Hello World Example
In this tutorial we will be using ELK stack along with Spring Boot Microservice for analyzing the generated logs. In next tutorial we will see how use FileBeat along with the ELK stack.
You can make use of the Online Grok Pattern Generator Tool for creating, testing and dubugging grok patterns required for logstash.
The implementation architecture will be as follows-
What is ELK? Need for it?The ELK Stack consists of three open-source products - Elasticsearch, Logstash, and Kibana from Elastic.
- Elasticsearch is a NoSQL database that is based on the Lucene search engine.
- Logstash is a log pipeline tool that accepts inputs from various sources, executes different transformations, and exports the data to various targets. It is a dynamic data collection pipeline with an extensible plugin ecosystem and strong Elasticsearch synergy
- Kibana is a visualization UI layer that works on top of Elasticsearch.
- Consider you have a single application running and it produces logs. Now suppose you want analyze the logs generated. One option is to manually analyze them. But suppose these logs are large, then manually analyzing them is not feasible.
- Suppose we have multiple Application running and all these applications produce logs. If we have to analyze the logs manually we will need to go through all the log files. These may run into hundreds.
VideoThis tutorial is explained in the below Youtube Video.
Lets BeginWe will first download the required stack.
Download the latest version of elasticsearch from Elasticsearch downloads
Run the elasticsearch.bat using the command prompt. Elasticsearch can then be accessed at localhost:9200
- Download the latest version of elasticsearch from Elasticsearch downloads
Download the latest version of kibana from Kibana downloads
Modify the kibana.yml to point to the elasticsearch instance. In our case this will be 9200. So uncomment the following line in
Run the kibana.bat using the command prompt. kibana UI can then be accessed at localhost:5601
- Download the latest version of kibana from Kibana downloads
Download the latest version of logstash from Logstash downloads
- Create a configuration file named logstash.conf. In further section we will be making the changes for this file and starting logstash.
- Download the latest version of logstash from Logstash downloads