To have a fully operational server, you need to be across a range of different technologies. Whether that’s security and intrusion detection, Email, DNS, User Management, or a host of other areas, there’s a lot to keep track of. But one of the essentials is logging.
Logging, whether it’s a part of a DevOps or software development, is an essential ingredient. Why? Because when things start going wrong, as they invariably do from time to time, you need to have information at hand which allows you to get to the root cause of the issue.
Getting in depth about what you should and shouldn’t log is beyond the scope of this post. Instead today, I’m going to show you how to integrate your Plesk 12 server setup with one of the best Logging services around, Loggly.
What Is Loggly?
To keep it concise, Loggly is:
An agent-less, pay-as-you-go, cloud-based logging service; one which allows you to store an enormous amount of log data along with a simple and intuitive interface to find the information you need
In the screenshot below, you can see a sample dashboard, one which, if you follow this tutorial, will become quite familiar to you. It sports an easy search interface at the top, allowing for setting the start and end dates, shows a graph of the number of events within the date range, along with a snapshot of the event information.
There’s much more to it than I can cover here; suffice to say, it’s a nice setup.
Getting Started
Before we go any further, you need to have an account with Loggly. If you don’t, you need to sign up for one. After that, you need to run the commands below, from the command line, once logged in to your Plesk 12 server. These will download and run the Loggly configuration script.
curl -O https://www.loggly.com/install/configure-linux.sh sudo bash configure-linux.sh -a SUBDOMAIN -t TOKEN -u USERNAME -p PASSWORD
You can see a number of parameters specified above, these are as follows:
- SUBDOMAIN: your account subdomain that you created when you signed up for Loggly
- TOKEN (optional): your customer token from the source setup page
- USERNAME: your Loggly username
- PASSWORD: (optional): your Loggly password. This argument is optional and it will prompt you if you don’t supply it.
What Does The Script Do?
I’m sure you’re likely not really comfortable running an unknown script on your server. So here’s what the script documentation says:
The script will perform over 10 system checks below to make sure your system is compatible with Loggly and that it can successfully send logs to Loggly. To configure the system, it executes the same instructions in rsyslog manual configuration by writing a configuration file named /etc/rsyslog.d/22-loggly.conf which will send the system logs to Loggly. It then restarts the rsyslog service so the changes take effect. The verification step sends a test log message with a UUID, and then retrieves it back through Loggly’s search API. If it is able to retrieve the log, then the configuration is successful. It also logs the status of your setup experience to Loggly which you can use to trouble-shoot if you experience difficulty during setup.
As you run the script, you’ll likely see the following output in your terminal, which I’ve truncated for sakes of readability.
INFO: Initiating Configure Loggly for Linux. INFO: Operating system is CentOS. INFO: Checking if logs-01.loggly.com is reachable. INFO: Restarting the rsyslog service. Shutting down system logger: [ OK ] Starting system logger: [ OK ] INFO: Creating directory mkdir: created directory `/var/spool/rsyslog' INFO: Sending test message to Loggly. SUCCESS: Verification logs successfully transferred to Loggly! You are now sending Linux system logs to Loggly.
What Changes Does It Make?
If you now look in /etc/rsyslog.d/22-loggly.conf, you’ll see that the following configuration has been added to it. This will send any log record, which is sent to the system Syslog daemon, to your account on Loggly, using their standard log format.
# Define the template used for sending logs to Loggly. Do not change this format. $template LogglyFormat,"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [8fbf831a-c1c9-4763-a3fe-efcae5c8273c@41058] %msg%n" # Send messages to Loggly over TCP using the template. *.* @@logs-01.loggly.com:514;LogglyFormat
Searching For Data
Now that you’ve got data being sent to your account, it’s time to learn the basics of the Loggly interface, so you can search it. From the Loggly home page, login to your account, where you’ll be greeted with the default dashboard.
This shows all entries for the last 24 hours, any alerts, along with a set of 4 saved searches. If you’ve got even a moderately busy server, you’ll already have a good number of records which you can search from.
In the top navigation bar, click on Search which will open a new dashboard tab. You can see mine is showing all records for the last seven days.
By default, no search query will be entered. Under the main search box, click the double-down arrow, to see the basic examples of how to search.
For more information, click ‘Advanced’. You can see there are a 7 core search types: Full Text, Ordered Tokens, Wildcards, Boolean, Field Value, Numeric Range, and Regular Expression.
What we’re going to search for is a combination of Ordered Tokens and Boolean. I’ve used the search term www-data AND "processing" OR "read". You can see that once the search has completed, the results are filtered, and the matching records have also been highlighted.
This helps you know why the result was returned, and how it matched your search criteria. Picking the second record, you can then step in to it and inspect it further, seeing when it arrived, what the message, tag and other information contained in it is.
There’s a lot to the interface, the rest of which is beyond the scope of this post. So I encourage you to take the time to explore it, and see what kinds of searches you can run, and data you can uncover.
Wrapping Up
And that’s all you have to do to send your Syslog messages to Loggly. Over the last several months I’ve researched and reviewed a number of logging services, and Loggly, amongst one or two others, is definitely a favourite of mine.
If you have any questions about what’s been covered here, make sure you check out the online documentation. As well as that, make sure you have a play around with the Loggly interface. There’s a lot to it, so becoming familiar will take some time.
