{"id":7961,"date":"2023-11-10T01:04:39","date_gmt":"2023-11-10T01:04:39","guid":{"rendered":"https:\/\/cheapwindowsvps.com\/blog\/step-by-step-guide-installing-loki-and-visualizing-logs-using-grafana\/"},"modified":"2025-01-20T11:29:48","modified_gmt":"2025-01-20T11:29:48","slug":"step-by-step-guide-installing-loki-and-visualizing-logs-using-grafana","status":"publish","type":"post","link":"https:\/\/cheapwindowsvps.com\/blog\/step-by-step-guide-installing-loki-and-visualizing-logs-using-grafana\/","title":{"rendered":"Step-by-Step Guide: Installing Loki and Visualizing Logs Using Grafana"},"content":{"rendered":"<div>Loki is an open-source log aggregation system developed by Grafana Labs and tailored for cloud-native and containerized apps. Loki is often used in conjunction with <a href=\"https:\/\/4sysops.com\/archives\/create-your-first-grafana-dashboard\/\" target=\"_blank\" rel=\"nofollow noopener\">Grafana<\/a> for log visualization. In this guide, I will explain how to install Loki and outline how you can integrate the log aggregation solution with Grafana.<\/div>\n<h2>Grafana Loki benefits<\/h2>\n<p>Here are a few key features of Grafana Loki:<\/p>\n<ul>\n<li><strong>Scalability:<\/strong> Loki was designed with horizontal scaling in mind, making it capable of managing large volumes of log data.<\/li>\n<li><strong>Label-based indexing:<\/strong> Loki uses <a href=\"https:\/\/grafana.com\/blog\/2020\/04\/21\/how-labels-in-loki-can-make-log-queries-faster-and-easier\/\" target=\"_blank\" rel=\"nofollow noopener\">label-based indexing<\/a> (metadata to describe a log stream) to organize and query data.<\/li>\n<li><strong>Distributed and highly available: <\/strong>Loki offers distributed and highly available deployment models to improve log availability and resilience.<\/li>\n<li><strong>Integrations:<\/strong> Loki seamlessly integrates with Grafana dashboards so that you can visualize logs.<\/li>\n<\/ul>\n<h2>Install Loki and Promtail<\/h2>\n<p>Loki can be installed on various systems, including Docker and Kubernetes, or as a standalone system on Linux. Promtail is the log collection agent used to collect and send logs to Loki. I will use apt-get for this guide to install Loki and Promtail on Ubuntu.<\/p>\n<pre>sudo apt-get install loki promtail<\/pre>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Install-Loki-and-Promtail-to-collect-logs.png\" target=\"_blank\" rel=\"nofollow noopener\">Install Loki and Promtail to collect logs<\/a><\/p>\n<p>Install Loki and Promtail to collect logs<\/p>\n<p>You can now start and enable the Loki service using the commands below.<\/p>\n<pre>sudo systemctl start loki<\/pre>\n<p>sudo systemctl enable loki<\/p>\n<p>Use the status command to verify that Loki is running.<\/p>\n<pre>sudo systemctl status loki<\/pre>\n<div><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Start-and-enable-the-Loki-service-on-Ubuntu.png\" target=\"_blank\" rel=\"nofollow noopener\">Start and enable the Loki service on Ubuntu<\/a><\/div>\n<p>Now you can enable the promtail service using these commands:<\/p>\n<pre>sudo systemctl start promtail<\/pre>\n<p>sudo systemctl enable promtail<\/p>\n<p>Use the status command to check whether Promtail is running.<\/p>\n<pre>sudo systemctl status promtail<\/pre>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Start-and-enable-the-Promtail-service-on-Ubuntu.png\" target=\"_blank\" rel=\"nofollow noopener\">Start and enable the Promtail service on Ubuntu<\/a><\/p>\n<h2>Grafana Loki integration<\/h2>\n<p>Once you have installed Loki and Promtail to collect logs, you can use Grafana for querying and visualization. Follow the steps below.<\/p>\n<h3>Add Loki as a data source<\/h3>\n<p>Go to Grafana&#8217;s home and navigate to the <strong>Data sources<\/strong> section.<\/p>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Navigate-to-Data-sources-section-of-Grafana.png\" target=\"_blank\" rel=\"nofollow noopener\">Navigate to Data sources section of Grafana<\/a><\/p>\n<p>Click <strong>Add new data source<\/strong>.<\/p>\n<div><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Add-a-new-data-source-in-Grafana.png\" target=\"_blank\" rel=\"nofollow noopener\">Add a new data source in Grafana<\/a><\/div>\n<p>In the <strong>Choose a data source type<\/strong> field, search for Loki, and click it.<\/p>\n<div><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Search-for-the-Loki-data-source-in-Grafana.png\" target=\"_blank\" rel=\"nofollow noopener\">Search for the Loki data source in Grafana<\/a><\/div>\n<p>By default, Loki runs at https:\/\/localhost:3100. Enter this as the URL.<\/p>\n<div><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Add-the-URL-of-the-Loki-service.png\" target=\"_blank\" rel=\"nofollow noopener\">Add the URL of the Loki service<\/a><\/div>\n<p>Click <strong>Save &amp; test<\/strong>.<\/p>\n<div><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Save-and-test-the-settings-of-the-Loki-data-source.png\" target=\"_blank\" rel=\"nofollow noopener\">Save and test the settings of the Loki data source<\/a><\/div>\n<p>You should now see the message <em>Data source successfully connected<\/em>. Click the <strong>building a dashboard<\/strong> link.<\/p>\n<div><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/You-will-get-a-success-message-when-the-Loki-data-source-is-added.png\" target=\"_blank\" rel=\"nofollow noopener\">You will get a success message when the Loki data source is added<\/a><\/div>\n<p>Set the data source as Loki for the dashboard.<\/p>\n<div><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Select-the-Loki-data-source-to-visualize-data-on-the-dashboard.png\" target=\"_blank\" rel=\"nofollow noopener\">Select the Loki data source to visualize data on the dashboard<\/a><\/div>\n<p>The dashboard will open, and you will see the time series visualization for the logs collected by Loki and visualized by Grafana.<\/p>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Default-Grafana-dashboard-for-the-Loki-data-source.png\" target=\"_blank\" rel=\"nofollow noopener\">Default Grafana dashboard for the Loki data source<\/a><\/p>\n<h3>Query Loki logs<\/h3>\n<p>Once the Loki data source collects the logs, you can search for specific logs using PromQL, Loki&#8217;s query language. Go to the side navigation bar, and click <strong>Explore<\/strong>.<\/p>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Go-to-Grafanas-Explore-option.png\" target=\"_blank\" rel=\"nofollow noopener\">Go to Grafanas Explore option<\/a><\/p>\n<p>From the dropdown list in the top left corner, select <strong>Loki<\/strong> as the data source. Now, you will get a query editor to run queries on the logs. Click <strong>Label filters<\/strong>.<\/p>\n<p>Query editor for the Loki data source in Grafana<\/p>\n<p>You will get two options to select a label to search. Choose a filename, and then <strong>select a log file<\/strong>.<\/p>\n<p>Label browser in Grafana query editor<\/p>\n<p>Scroll down, and click <strong>Show logs<\/strong>.<\/p>\n<div>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Show-logs-based-on-the-selected-labels.png\" target=\"_blank\" rel=\"nofollow noopener\">Show logs based on the selected labels<\/a><\/p>\n<p>Show logs based on the selected labels<\/p>\n<\/div>\n<p>In the query editor, you can see that the query was automatically created, and the visualization for alternatives.log has started coming in.<\/p>\n<div>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Visualization-of-the-logs-collected-by-Loki.png\" target=\"_blank\" rel=\"nofollow noopener\">Visualization of the logs collected by Loki<\/a><\/p>\n<p>Visualization of the logs collected by Loki<\/p>\n<\/div>\n<p>When you scroll down, you will see the logs collected with dates and times.<\/p>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Details-of-the-logs-collected-by-Loki.png\" target=\"_blank\" rel=\"nofollow noopener\">Details of the logs collected by Loki<\/a><\/p>\n<p>You can write queries directly to filter and search logs based on labels, keywords, and time ranges. Here are some examples:<\/p>\n<ul>\n<li>To search for logs with a specific keyword: <em>{|=~&#8221;keyword&#8221;}<\/em><\/li>\n<li>To filter logs by labels: <em>{job=&#8221;varlogs&#8221;}<\/em><\/li>\n<li>To search for logs within a time range: <em>{job=&#8221;varlogs&#8221;} |= &#8220;keyword&#8221; |~ &#8220;5m&#8221;<\/em><\/li>\n<\/ul>\n<p>Below is an example where I am searching for only error logs in the bootstrap.log file. I am filtering the logs based on the keyword &#8220;error.&#8221;<\/p>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Querying-logs-using-label-filters.png\" target=\"_blank\" rel=\"nofollow noopener\">Querying logs using label filters<\/a><\/p>\n<p>When you click <strong>Run query<\/strong>, you will get the list of all error logs in the bootstrap.log file.<\/p>\n<p><a href=\"https:\/\/4sysops.com\/wp-content\/uploads\/2023\/11\/Filtering-out-error-logs-collected-by-Loki.png\" target=\"_blank\" rel=\"nofollow noopener\">Filtering out error logs collected by Loki<\/a><\/p>\n<p>Similarly, if you edit the query using the keyword &#8220;update,&#8221; you will get the details of all the updates in the log file.<\/p>\n<div>\n<h2>Subscribe to 4sysops newsletter!<\/h2>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Loki is an open-source log aggregation system developed by Grafana Labs and tailored for cloud-native and containerized apps. Loki is often used in conjunction with Grafana for log visualization. In this guide, I will explain how to install Loki and outline how you can integrate the log aggregation solution with Grafana. Grafana Loki benefits Here [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":7962,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[92,95,109],"tags":[],"class_list":["post-7961","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-articles","category-devops","category-monitoring"],"_links":{"self":[{"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/posts\/7961","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/comments?post=7961"}],"version-history":[{"count":1,"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/posts\/7961\/revisions"}],"predecessor-version":[{"id":10436,"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/posts\/7961\/revisions\/10436"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/media\/7962"}],"wp:attachment":[{"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/media?parent=7961"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/categories?post=7961"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cheapwindowsvps.com\/blog\/wp-json\/wp\/v2\/tags?post=7961"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}