1
00:00:06,690 --> 00:00:09,210
- All right, let's talk about logging.

2
00:00:09,210 --> 00:00:12,240
So most Linux distributions
currently host two systems

3
00:00:12,240 --> 00:00:15,600
that take care of logging,
and that's a bit confusing.

4
00:00:15,600 --> 00:00:17,130
There is systemd-journald,

5
00:00:17,130 --> 00:00:20,250
which is the systemd
integrated logging solution.

6
00:00:20,250 --> 00:00:21,630
And there is rsyslogd,

7
00:00:21,630 --> 00:00:23,793
which is the legacy Linux logging system.

8
00:00:24,750 --> 00:00:25,650
And apart from that

9
00:00:25,650 --> 00:00:28,173
some services may take
care of their own logging.

10
00:00:29,640 --> 00:00:32,790
Now in case you are wondering,
why do we have two systems?

11
00:00:32,790 --> 00:00:35,220
Simple, rsyslogd is offering features

12
00:00:35,220 --> 00:00:37,650
that systemd-journald is not offering,

13
00:00:37,650 --> 00:00:40,650
but systemd-journald is a part of systemd.

14
00:00:40,650 --> 00:00:42,420
And as such, it captures all messages

15
00:00:42,420 --> 00:00:46,260
that are generated by
systemd units anyway.

16
00:00:46,260 --> 00:00:47,820
And that is convenient,

17
00:00:47,820 --> 00:00:48,720
because that allows you

18
00:00:48,720 --> 00:00:52,260
to use commands like
systemctl show on the unit

19
00:00:52,260 --> 00:00:56,040
to check messages that were
generated by a specific unit.

20
00:00:56,040 --> 00:00:57,150
And alternatively,

21
00:00:57,150 --> 00:01:01,380
there is a journalctl
utility, which allows you

22
00:01:01,380 --> 00:01:04,533
to check the journal which
contains all messages.

23
00:01:05,850 --> 00:01:08,973
Now the systemd journal by
default is not persistent.

24
00:01:10,110 --> 00:01:11,610
That is because it's very rich.

25
00:01:11,610 --> 00:01:13,590
It contains a lot of information.

26
00:01:13,590 --> 00:01:15,630
And when you reboot it's all gone.

27
00:01:15,630 --> 00:01:18,000
You don't like it, you
can do something about it.

28
00:01:18,000 --> 00:01:19,800
I'll tell you later what you can do.

29
00:01:21,240 --> 00:01:24,390
But first we need to talk about rsyslogd.

30
00:01:24,390 --> 00:01:28,350
So rsyslogd is compatible with
the legacy syslogd service

31
00:01:28,350 --> 00:01:31,590
that has been around
for a long, long time.

32
00:01:31,590 --> 00:01:32,580
In rsyslogd,

33
00:01:32,580 --> 00:01:35,880
we use facility: severity
destination to ensure

34
00:01:35,880 --> 00:01:39,030
that messages are written
to the appropriate location.

35
00:01:39,030 --> 00:01:43,440
And typically, messages are
written to files in /var/log,

36
00:01:43,440 --> 00:01:47,190
but output modules can
also be used in rsyslogd.

37
00:01:47,190 --> 00:01:48,023
I'll tell you

38
00:01:48,023 --> 00:01:50,700
in more detail later how
to configure rsyslogd.

39
00:01:50,700 --> 00:01:52,710
For now, I'd like to have a look

40
00:01:52,710 --> 00:01:56,700
at the logs that are generated
by either of these systems.

41
00:01:56,700 --> 00:01:59,640
Oh, and just one thing, you
might be on a distribution

42
00:01:59,640 --> 00:02:02,707
that is not offering both systemd-journald

43
00:02:02,707 --> 00:02:04,500
as well as rsyslog.

44
00:02:04,500 --> 00:02:06,330
That's a choice of the distribution.

45
00:02:06,330 --> 00:02:09,030
So you might be missing rsyslogd.

46
00:02:09,030 --> 00:02:11,880
You are probably not going
to miss systemd-journald,

47
00:02:11,880 --> 00:02:13,800
because it's a part of systemd

48
00:02:13,800 --> 00:02:16,230
and it works very well in systemd.

49
00:02:16,230 --> 00:02:18,720
All systemd Linux distributions

50
00:02:18,720 --> 00:02:20,823
are using systemd-journald.

51
00:02:22,800 --> 00:02:27,582
So for a change we are going
to run this demo on Ubuntu,

52
00:02:27,582 --> 00:02:32,582
sudo systemctl status
chron.service for instance,

53
00:02:34,500 --> 00:02:36,000
Cron is just a random service

54
00:02:36,000 --> 00:02:37,800
that I want to investigate here,

55
00:02:37,800 --> 00:02:41,460
and I want to show you all
the information right here

56
00:02:41,460 --> 00:02:43,230
in the lower part of the output.

57
00:02:43,230 --> 00:02:45,603
This comes straight from
the systemd journal.

58
00:02:46,710 --> 00:02:49,547
Now if you want more
about the systemd journal,

59
00:02:49,547 --> 00:02:53,190
journalctl is the utility
that you wanna use.

60
00:02:53,190 --> 00:02:55,050
If you just type journalctl,

61
00:02:55,050 --> 00:02:58,593
you get the journal and
everything that's inside.

62
00:02:59,550 --> 00:03:03,900
The pager is really less so
you can use the key sequences

63
00:03:03,900 --> 00:03:05,973
that you are using in less as well.

64
00:03:06,900 --> 00:03:10,140
Now as you can see, almost 10,000 lines,

65
00:03:10,140 --> 00:03:13,080
since this journal was
started on February 20th.

66
00:03:13,080 --> 00:03:13,983
And that's a lot.

67
00:03:14,940 --> 00:03:18,270
Fortunately, there are
filtering options as well

68
00:03:18,270 --> 00:03:23,010
like journalctl -u chron.service,

69
00:03:23,010 --> 00:03:27,783
where -u stands for the unit
that you wants to investigate.

70
00:03:29,160 --> 00:03:31,770
Now by default, you only
see the last couple of lines

71
00:03:31,770 --> 00:03:33,390
that were logged in a unit,

72
00:03:33,390 --> 00:03:37,080
but here we can see
that everything is here.

73
00:03:37,080 --> 00:03:39,300
Oh, and look at that, there's also a boot.

74
00:03:39,300 --> 00:03:42,030
So apparently we rebooted this thing,

75
00:03:42,030 --> 00:03:45,390
and that means that this Ubuntu system

76
00:03:45,390 --> 00:03:48,810
is using a persistent journal by default.

77
00:03:48,810 --> 00:03:51,660
That's good to know, but that
won't always be the case.

78
00:03:51,660 --> 00:03:53,280
Now how about rsyslog?

79
00:03:53,280 --> 00:03:55,260
Do we even have rsyslog?

80
00:03:55,260 --> 00:03:58,833
ps aux | grep syslog.

81
00:04:00,041 --> 00:04:02,070
And what do we see?

82
00:04:02,070 --> 00:04:06,600
We see that we have
rsyslogd, that is available.

83
00:04:06,600 --> 00:04:08,433
So rsyslogd running Ubuntu,

84
00:04:09,343 --> 00:04:12,690
as well as on Red Hat family by default,

85
00:04:12,690 --> 00:04:17,690
normally writes the messages
to the /var/log directory.

86
00:04:18,540 --> 00:04:22,200
I would suggest use ls -l on /var/log,

87
00:04:22,200 --> 00:04:25,170
so that you can see
which files are written.

88
00:04:25,170 --> 00:04:27,630
And the important files, the dpkg,

89
00:04:27,630 --> 00:04:29,760
which is your packets log.

90
00:04:29,760 --> 00:04:31,770
We have the journal directory

91
00:04:31,770 --> 00:04:33,090
that's a persistent journal

92
00:04:33,090 --> 00:04:36,150
that Ubuntu workstation
is creating by default.

93
00:04:36,150 --> 00:04:37,301
And we have syslog.

94
00:04:37,301 --> 00:04:39,720
syslog is the main log file

95
00:04:39,720 --> 00:04:42,793
that is created by our
rsyslog, on Ubuntu that is.

96
00:04:44,370 --> 00:04:45,900
On Red Hat, you will notice

97
00:04:45,900 --> 00:04:49,200
that the main log file
is /var/log/messages.

98
00:04:49,200 --> 00:04:50,970
Now if you want to investigate,

99
00:04:50,970 --> 00:04:55,920
well, sudo less on
/var/log/syslog, for instance.

100
00:04:55,920 --> 00:04:58,950
Oops, /var/log/syslog, for instance,

101
00:04:58,950 --> 00:05:02,070
is showing everything that has happened

102
00:05:02,070 --> 00:05:05,460
and that has been committed to the syslog.

103
00:05:05,460 --> 00:05:08,400
Much of this information
is redundant by the way,

104
00:05:08,400 --> 00:05:12,270
but in rsyslog, some additional
features are available.

105
00:05:12,270 --> 00:05:13,720
We'll talk about those later.