Unable to send logs from rsyslog to logstash and elasticsearch
0
votes
0
answers
1083
views
I am using ubuntu and I installed the ELK stack version 8.5 on the same machine. I did the necessary configurations for each of the services(logstash, elasticsearch, kibana) and I equally configured rsyslog to send logs to logstash(defining an index to be created each day) and from logstash to elasticsearch. The issue is that I can't see any log in elasticsearch when rsyslog is at input in logstash meanwhile when I use the file input with the file's path, it works(I can see the index in elasticsearch and kibana too) but I also realised it doesn't show for some files. So it works for some files and it doesn't work for others. What can be the issue then?
rsyslog.conf file
#################
#### MODULES ####
#################
module(load="imuxsock") # provides support for local system logging
#module(load="immark") # provides --MARK-- message capability
# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")
# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")
# provides kernel logging support and enable non-kernel klog messages
module(load="imklog" permitnonkernelfacility="on")
Configuration files in the /etc/rsyslog.d directory
01-json-template.conf file
template(name="json-template"
type="list") {
constant(value="{")
constant(value="\"@timestamp\":\"") property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"@version\":\"1")
constant(value="\",\"message\":\"") property(name="msg" format="json")
constant(value="\",\"sysloghost\":\"") property(name="hostname")
constant(value="\",\"severity\":\"") property(name="syslogseverity-text")
constant(value="\",\"facility\":\"") property(name="syslogfacility-text")
constant(value="\",\"programname\":\"") property(name="programname")
constant(value="\",\"procid\":\"") property(name="procid")
constant(value="\"}\n")
}
50-default.conf file
# Default rules for rsyslog.
#
# For more information see rsyslog.conf(5) and /etc/rsyslog.conf
*.* @localhost:514
#
# First some standard log files. Log by facility.
#
auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none -/var/log/syslog
#cron.* /var/log/cron.log
#daemon.* -/var/log/daemon.log
kern.* -/var/log/kern.log
#lpr.* -/var/log/lpr.log
mail.* -/var/log/mail.log
#user.* -/var/log/user.log
#
# Logging for the mail system. Split it up so that
# it is easy to write scripts to parse these files.
#
#mail.info -/var/log/mail.info
#mail.warn -/var/log/mail.warn
mail.err /var/log/mail.err
60-output.conf file
# This line sends all lines to defined IP address at port 10514,
# using the "json-template" format template
*.* @localhost:10514;json-template
logstash configuration file for rsyslog
input {
udp {
host => "localhost"
port => 10514
codec => "json"
type => "rsyslog"
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "rsyslog-%{+YYYY.MM.dd}"
}
}
Asked by Ngouaba Rosalie
(31 rep)
Nov 29, 2022, 09:44 AM
Last activity: Nov 29, 2022, 02:12 PM
Last activity: Nov 29, 2022, 02:12 PM