Sample Header Ad - 728x90

dash maximum variable length under systemd

5 votes
2 answers
746 views
I have a shell script that uses a single variable as an associative array (one KEY=VALUE per line). Throughout the execution of the script, the variable is manipulated to add, remove or modify entries: Append:
lang-bash
VARIABLE="$(printf "%s\n%s" "$VARIABLE" "KEY=VALUE")"
Modify:
lang-bash
VARIABLE="$(printf "%s\n" "$VARIABLE" | sed -E "s,^(KEY=).*$,\1VALUE,")"
Remove:
lang-bash
VARIABLE="$(printf "%s\n" "$VARIABLE" | grep -E -v "^KEY=.*$")"
When executed in a terminal (or as a service on an old machine under *sysv* through an init script), the script runs fine, but when run as a service under *systemd*, after a while, the script starts to spit out error messages in the log : sh: printf: I/O error After a lot of trial and error, I couldn't determine exactly what command(s) in the script produced those errors, but I noticed that they start to appear when the length of the variable reaches 8000 bytes (I guess 8192, but I couldn't pinpoint it exactly since whole lines are appended). I'm pretty sure the variable length is the problem because I implemented a routine that trims the oldest entries of the array whenever the variable length approaches 8192 bytes, and now the script does run under systemd for a long time without errors; but that's of course not ideal, as there is some information lost. I searched the web for information about maximum variable length in shell scripts, but didn't find anything useful: - dash manual page doesn't say anything about maximum variable length. - [GNU sed documentation](https://www.gnu.org/software/sed/manual/html_node/Limitations.html) says: > For those who want to write portable *sed* scripts, be aware that some implementations have been known to limit line lengths (for the pattern and hold spaces) to be no more than 4000 bytes. The *POSIX* standard specifies that conforming *sed* implementations shall support at least 8192 byte line lengths. *GNU sed* has no built-in limit on line length; as long as it can *malloc()* more (virtual) memory, you can feed or construct lines as long as you like. ...but this applies to *line* length, not to the whole text length (the individual lines don't exceed 80 characters) Anyway, since the errors appear only when the script is run through systemd, I tried to increase LimitMSGQUEUE and/or LimitSTACK in the unit file, to no avail (this was a blind guess, as I don't understand exactly the concepts of message queues or process stack, but the numbers shown by systemctl show looked like 8 KB or so). All other limits regarding memory (LimitRSS, LimitAS, LimitMEMLOCK) seem high enough (way past 8192 bytes), so I don't know what to do next. What can I do to get this script running under systemd without errors when the variable length exceeds 8 KB ?
Asked by MoonSweep (428 rep)
Apr 10, 2019, 10:07 AM
Last activity: Sep 4, 2023, 11:57 PM