dash maximum variable length under systemd
5
votes
2
answers
746
views
I have a shell script that uses a single variable as an associative array (one
KEY=VALUE
per line).
Throughout the execution of the script, the variable is manipulated to add, remove or modify entries:
Append:
lang-bash
VARIABLE="$(printf "%s\n%s" "$VARIABLE" "KEY=VALUE")"
Modify:
lang-bash
VARIABLE="$(printf "%s\n" "$VARIABLE" | sed -E "s,^(KEY=).*$,\1VALUE,")"
Remove:
lang-bash
VARIABLE="$(printf "%s\n" "$VARIABLE" | grep -E -v "^KEY=.*$")"
When executed in a terminal (or as a service on an old machine under *sysv* through an init script), the script runs fine, but when run as a service under *systemd*, after a while, the script starts to spit out error messages in the log :
sh: printf: I/O error
After a lot of trial and error, I couldn't determine exactly what command(s) in the script produced those errors, but I noticed that they start to appear when the length of the variable reaches 8000 bytes (I guess 8192, but I couldn't pinpoint it exactly since whole lines are appended).
I'm pretty sure the variable length is the problem because I implemented a routine that trims the oldest entries of the array whenever the variable length approaches 8192 bytes, and now the script does run under systemd
for a long time without errors; but that's of course not ideal, as there is some information lost.
I searched the web for information about maximum variable length in shell scripts, but didn't find anything useful:
- dash
manual page doesn't say anything about maximum variable length.
- [GNU sed documentation](https://www.gnu.org/software/sed/manual/html_node/Limitations.html) says:
> For those who want to write portable *sed* scripts, be aware that some implementations have been known to limit line lengths (for the pattern and hold spaces) to be no more than 4000 bytes. The *POSIX* standard specifies that conforming *sed* implementations shall support at least 8192 byte line lengths. *GNU sed* has no built-in limit on line length; as long as it can *malloc()* more (virtual) memory, you can feed or construct lines as long as you like.
...but this applies to *line* length, not to the whole text length (the individual lines don't exceed 80 characters)
Anyway, since the errors appear only when the script is run through systemd
, I tried to increase LimitMSGQUEUE
and/or LimitSTACK
in the unit file, to no avail (this was a blind guess, as I don't understand exactly the concepts of message queues or process stack, but the numbers shown by systemctl show
looked like 8 KB or so). All other limits regarding memory (LimitRSS
, LimitAS
, LimitMEMLOCK
) seem high enough (way past 8192 bytes), so I don't know what to do next.
What can I do to get this script running under systemd
without errors when the variable length exceeds 8 KB ?
Asked by MoonSweep
(428 rep)
Apr 10, 2019, 10:07 AM
Last activity: Sep 4, 2023, 11:57 PM
Last activity: Sep 4, 2023, 11:57 PM