1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
|
Mount ordering and automount options in systemd
###############################################
:date: 2015-12-13 15:13
:slug: systemd_automount_entry
:tags: systemd, fstab, automount
:author: copyninja
:summary: Brief notes on systemd.mount and systemd.autmount features.
Just to give a brief recap of why I'm writing this post, I've to
describe a brief encounter I had with systemd which rendered my system
unbootable. I first felt its systemd's problem but later figured out
it was my mistake. So below is brief story divided into 2 sections
*Problem* and *Solution*. Here *Problem* describes issue I faced with
systemd and *Solution* discusses the *.mount* and *.automount* suffix
files used by systemd.
Problem
=======
I have several local bin folders and I don't want to modify *PATH*
environment variable adding every other folder path. I first started
using *aufs* as alternative to symlinks or modifying *PATH*
variable. Recently I learnt about much more light weight union fs which is
in kernel, *overlayfs*. And Google led me to Arch wiki article which
showed me a fstab entry like below
.. code-block:: fstab
overlay /usr/local/bin overlay noauto,x-systemd.automount,lowerdir=/home/vasudev/Documents/utilities/bin,upperdir=/home/vasudev/Documents/utilities/jonas-bin,workdir=/home/vasudev/Documents/utilities/bin-overlay 0 0
And after adding this entry, on next reboot systemd is not able to
mount my LVM home and swap partition. It does however mount root
partition. It gives me emergency target but login never returns. So
without any alternative I had to reinstall the system. But funnily
enough I started encountering the same issue (yeah after I had added
above entry into fstab). Yeah that time I never thought that its
culprit. My friend Ritesh finally got my system booting after the
weird x-systemd.automount option. I never investigated further that
time on why this problem occurred.
Solution
========
After re-encounter with similar problem and some other project I read
the manual on *systemd.mount*, *systemd-fstab-generator* and
*systemd.automount* and I had some understanding of what really went
wrong in my case above.
So now let us see what really is happening. All the above happened
because, *systemd translates the /etc/fstab into .mount units at run
time using systemd-fstab-generator*. Every entry in fstab translates
into a file named after the mount point. The */* in the path of the
mount point is replaced by a *-*. So */* mount point is named as
*-.mount* and */home* is named as *home.mount* and */boot* becomes
*boot.mount*. All these files can be seen in directory
*/run/systemd/generator*. And all these mount points are needed by
*local-fs.target* , if any of these mount points fail,
*local-fs.target* fails. And if *local-fs.target* fails it will invoke
*emergency.target*.
*systemd-fstab-generator* manual suggests the ordering information in
*/etc/fstab* is discarded. That means if you have union mounts, bind
mounts or fuse mounts in the fstab which is normally at end of fstab
and uses path from /home or some other device mount points it will
fail to get mounted. This happens because *systemd-fstab-generator*
didn't consider the ordering in fstab. This is what happened in my
case my *overlay* mount point which is *usr-local-bin.mount* was some
how happened to be tried before *home.mount* because there is no
explicit dependency like *Requires=* or *After=* was declared. When
systemd tried to mount it the path required under /home were not
really present hence failed which in return invoked *emergency.target*
as *usr-local-bin.mount* is *Requires=* dependency for
*local-fs.target*.
Now what I didn't understand is why *emregency.target* never gave me
the root shell after entering login information. I feel that this part
is unrelated to the problem I described above and is some other bug.
To over come this what we can do is provided *systemd-fstab-generator*
some information on dependency of each mount point. *systemd.mount*
manual page suggests several such options. One which I used in my case
is *x-systemd.requires* which should be placed in options column of
fstab and specify the mount point which is needed before it has to be
mounted. So my overlay fs entry translates to something like below
.. code-block:: fstab
overlay /usr/local/bin overlay noauto,x-systemd.requires=/home,x-systemd.automount,lowerdir=/home/vasudev/Documents/utilities/bin,upperdir=/home/vasudev/Documents/utilities/jonas-bin,workdir=/home/vasudev/Documents/utilities/bin-overlay 0 0
There is another special option called *x-systemd.automount* this will
make systemd-fstab-generator to create a *.automount* file for this
mount point. What does *systemd.automount* do? It achieves on-demand
file mounting and parallelized file system mounting. Something similar
to *systemd.socket* the file system will be mounted when you access
the mount point for first time.
So now if you try to see dependency of usr-local-bin.mount you will
see following.
.. code-block:: shell
systemctl show -p After usr-local-bin.mount
After=-.mount systemd-journald.socket local-fs-pre.target system.slice usr-local-bin.automount
This means now *usr-local-bin.mount* depends on
*usr-local-bin.automount*. And let us see what usr-local-bin.automount
needs.
.. code-block:: shell
systemctl show -p Requires usr-local-bin.automount
Requires=-.mount home.mount
systemctl show -p After usr-local-bin.automount
After=-.mount home.mount
So clearly *usr-local-bin.automount* is activated after *-.mount* and
*home.mount* is activated. Similar can be done for any bind mounts or
fuse mounts which require other mount points before it can be
mounted. Also note that *x-systemd.autmount* is not mandatory option
for declaring dependencies, I used it just to make sure
*/usr/local/bin* is mounted only when its really needed.
Conclusion
===========
A lot of traditional way has been changed by systemd. I never
understood why my system failed to boot in first place this happened
because I was not really aware of how systemd works and was trying to
debug the problem with traditional approach. So there is really a
learning curve involved for every sysadmin out there who is going to
use systemd. Most of them will read documentation before hand but
others like me will learn after having a situation like above. :-).
So systemd interfering into /etc/fstab is good?. I don't know but
since systemd parallelizes the boot procedure something like this is
really needed. Is there a way to make systemd not touch
/etc/fstab?. Yes there is you need to pass *fstab=0* option in kernel
command line and systemd-fstab-generator doesn't create any .mount or
.swap files from your /etc/fstab.
**!NB Also it looks like x-systemd.requires option was introduced
recently and is not available in systemd <= 215 which is default in
Jessie. So how do you declare dependencies in Jessie system?. I don't
have any answer!. I did read that x-systemd.automount which is
available in those versions of systemd can be used, but I'm yet to
experiment this. If it succeeds I will write a post on it.**
|
