I have a Ubuntu 24.04.2 system where /dev/shm gets remounted (I assume) every now and then (roughly each 10 seconds), but I have no idea why. There's no mention of that mounting point in /etc/fstab and even if I would add an entry in there it would still be remounted with other options than the one I expect. Here's the output from /proc/self/mountinfo (see final column):

$ while true; do column -t -N mountID,parentID,"major:minor",rootMount,mountPoint,mountOpts,optionalFields,optFieldSeparator,fsType,mountSource,superOpts /proc/self/mountinfo | grep -E 'mountID|shm'; sleep 1; done
mountID  parentID  major:minor  rootMount  mountPoint  mountOpts        optionalFields  optFieldSeparator  fsType  mountSource  superOpts
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64
32       26        0:27         /          /dev/shm    rw,nosuid,nodev  shared:4        -                  tmpfs   tmpfs        rw,size=4011076k,nr_inodes=1002769,inode64

I've tried checking dmesg -wH, using strace (though I do not remember the parameters anymore) on pid 1 (not sure if this is sane or not - ChatGPT suggestion), checking journalctl -k -f and even finding all files on the system and executing grep -IHn '/dev/shm' without finding anything useful. There was a couple of mentions of apparmor, although I disabled that service and it still got remounted. I compared this to another system running the same version of Ubuntu (although with a slightly different version of the kernel - 6.8.0-63 vs 6.8.0-71) and the issue do not happen there. How do I troubleshoot this?

I'm using Ubuntu 20.04.2 LTS with kernel 5.4.0-70-generic. Shared memory keeps piling up until system hangs as it doesn't have any memory left. I'm using Prometheus + Grafana to monitor my system resources and from their graph it's visible that it takes approximately 4-7 days since restart until shared memory reached 20 GB. But it doesn't stop there, it keeps growing until I make another restart. Same amount of shared memory usage can be seen from free -h command. The usage of tmpfs seems to be low: Shared memory keeps growing even if I close all the apps and not using the computer. What could be the cause of such behavior? How could I start investigating the problem? How could I find out which processes use that much shared memory?

$ ipcs -m --human

------ Shared Memory Segments --------
key        shmid      owner      perms      size       nattch     status      
0x00000000 3145729    martsraits 600          256M     2          dest         
0x00000000 2588677    martsraits 600          512K     2          dest         
0x00000000 1245191    martsraits 600          512K     2          dest         
0x00000000 5832713    martsraits 600          512K     2          dest         
0x00000000 10         martsraits 600          512K     2          dest         
0x00000000 1212427    martsraits 600          512K     2          dest         
0x00000000 12         martsraits 600          512K     2          dest         
0x00000000 1212429    martsraits 600           64M     2          dest         
0x00000000 32785      martsraits 600          512K     2          dest         
0x00000000 2064402    martsraits 600          512K     2          dest         
0x00000000 5537814    martsraits 600          512K     2          dest         
0x00000000 1114138    martsraits 600          512K     2          dest         
0x00000000 8945695    martsraits 600         23,3K     2          dest         
0x00000000 1507360    martsraits 600          512K     2          dest         
0x00000000 2916388    martsraits 600          512K     2          dest         
0x00000000 4816933    martsraits 606          8,2M     2          dest         
0x00000000 4816934    martsraits 606          8,2M     2          dest         
0x00000000 4816935    martsraits 600          128M     2          dest         
0x00000000 3670057    martsraits 600          512K     2          dest         
0x00000000 3309611    martsraits 600          512K     2          dest         
0x00000000 1441844    martsraits 600          512K     2          dest         
0x00000000 2555960    martsraits 600            8M     2          dest         
0x00000000 917564     martsraits 600          512K     2          dest         
0x00000000 3899453    martsraits 600          512K     2          dest

The sum of size column in ipcs -m is only 500M.

I recently learned about the convenient "shared memory" file system at /dev/shm. I wanted to see if I could use this to speed up a sometimes-disk-bound program that writes to and reads from a directory, so I ran a little experiment on an EC2 instance (c4.8xlarge running Ubuntu 16.04): $ time yes asdfjkl | head -1000000000 > /mnt/fake.txt real 0m21.381s $ time yes asdfjkl | head -1000000000 > /dev/shm/fake.txt real 0m20.266s $ time yes asdfjkl | head -1000000000 > /dev/null real 0m14.334s The EC2 instance seems to have a write throughput to /dev/shm/, that is comparable to an EBS drive, which was surprising. htop indicates that the machine isn't using swap space in order to write to /dev/shm. The noticeably faster write to /dev/null in the third case indicates that I'm probably not bound by some other factor (e.g. CPU via the implementation of yes) in the first two cases. I ran the same experiment on my personal computer -- enough memory that /dev/shm can hold 7.5 GB of asdfjkl\n by default, I can dig up more hardware details if anyone thinks they'll matter -- also running Ubuntu 16.04: $ time yes asdfjkl | head -1000000000 > /mnt/fake.txt real 0m36.520s $ time yes asdfjkl | head -1000000000 > /dev/shm/fake.txt real 0m12.516s $ time yes asdfjkl | head -1000000000 > /dev/null real 0m11.252s This is much closer to what I expected. Read throughput (writing to /dev/null) on both machines and from both file systems is roughly proportional to write throughput in the corresponding case. Two other observations, which I don't know quite how to interpret: - On the EC2 instance, htop indicates memory usage comparable to the size of /dev/shm/fake.txt once it's written, while on my desktop, it does not. - On the EC2 instance, concurrent disk write congestion appears to slow down a write to shared memory by an amount comparable to how much the writes to disk were slowed down, while on my desktop, it does not.

I have switched from Windows 10 to linux mint 21.1 I tried playing GTA 5 on lutris (wine emulation) however the game is almost unplayable on linux as big parts of the map will not load because of my low VRAM(2GB only) on windows the game ran fine since there was shared memory. Which I dont know how to enable on Linux mint. Any help appreciated. below is an example of shared gpu memory on windows:

According to the RHEL document , the total amount of shared memory available on the system equals shmall*PAGE_SIZE. After I completed the installation of RHEL 6, the value of the shmall kernel parameter defaults to 4294967296， which means that the total amount of shared memory pages that can be used system-wide is 4294967296, and the page size is 4096 B. So, based on the formula, the size of shared memory is 4294967296*4096/1024/1024/1024/1024 = 16 TB ... which is much more than the size of RAM (8 GB) the operating system has. How can an OS find 16 TB of shared memory to allocate? So, is the size of /dev/shm equal to the size of shared memory? If not, how can I get the actual size of the shared memory?

I'm having a PC freeze issue that I can't seem to figure out. I have three identical PCs. They are each custom builds with i7 and 64GB of RAM. The OS drives are 512GB nvme drives. They each run CentOS v7.8.2003 with kernel v3.10. I'm running some custom software that stores huge chunks of data, processes it, and then re-stores it. It may be relevant to show the custom configurations done to the OS. Custom fstab: UUID=xxxxxx / ext4 noatime,nodiratime,discard 1 1 UUID=xxxxxx swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults,noatime,mode=1777 0 0 tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 LABEL=drive1 /data/drive1 auto noatime,nodiratime,discard,nofail 0 0 LABEL=drive2 /data/drive2 auto noatime,nodiratime,discard,nofail 0 0 LABEL=drive3 /data/drive3 auto noatime,nodiratime,discard,nofail 0 0 LABEL=drive4 /data/drive4 auto noatime,nodiratime,discard,nofail 0 0 LABEL=drive5 /data/drive5 auto noatime,nodiratime,discard,nofail 0 0 LABEL=drive6 /data/drive6 auto noatime,nodiratime,discard,nofail 0 0 Custom sysctl.conf: vm.swappiness=1 vm.vfs_cache_pressure=50 I don't want to go into too much detail on the software but what is relevant is that it receives data at an extremely fast rate. The data is stored to five SATA SSDs connected directly to the motherboard. (writing to 5 in parallel was the only way for the write speeds to keep up with the inflow of data) The data is stored in large chunks (files) that are later processed and then stored to a sixth SATA SSD. I don't know the details of how some of the software works but I do know that some of the processes use large chunks of shared memory. The Problem: Sometimes the PC will freeze when a new chunk of data comes in. This freeze is unrecoverable and requires a hard reboot. When it occurs is random but it is always at the start of a new data collect. Changing the software parameters and enabling/disabling certain processes can affect the frequency of when the freeze occurs. This doesn't allow me to narrow it down though as it doesn't seem to be tied to a particular process. Its just that some configurations make it more likely to occur. I did find a particular configuration that allows me to consistently recreate the issue so at least that helps for testing. I have worked with the devs that wrote the software but we are unable to find any issues that could cause a PC to freeze. I have also tested the software for memory leaks but cannot find any. To my mind all of this points to a memory issue. However, I can't actually find a memory issue. I have used top and free on the command line and System Monitor in Gnome but there are no indications of any memory issues. The CPU load isn't too high and the PC isn't even using half of the 64GB of RAM. If this is a memory issue then I'm either missing something or it is happening so fast that monitoring and logging can't catch it. Now to the heart of the matter: In an act of desperation, one of my partners did this: vm.overcommit_memory=2 vm.overcommit_ratio=80 This "seemed" to fix the issue. I can no longer recreate the freeze. If I monitor the memory while going through the steps to recreate the issue, I don't see anything unusual. None of the processes crashed and the software just did what it was designed to do. This makes me feel like the cause of the freeze isn't the software but the way the OS is configured relative to the atypical way the PC is being used. I have read the documentation but it is unhelpful with regards to how changing overcommit_memory might affect other parts of the system or with providing any examples of when you would want to change it. I've looked for more information on the forums but I mostly find "don't touch it unless you know what you're doing". I also worry about OOM killing important processes if too much memory is requested. Can someone please tell me when you would want to change the overcommit_memory settings. Does my situation count as one where you would need to change them? If so, what should I look out for in regards to how it could affect other aspects of the system?

https://dvdhrm.wordpress.com/2014/06/10/memfd_create2/ > Theoretically, you could achieve [memfd_create()] behavior without introducing new syscalls, like this: > >

fd = open("/tmp", O_RDWR | O_TMPFILE | O_EXCL, S_IRWXU);

(Note, to more portably guarantee a tmpfs here, we can use "/dev/shm" instead of "/tmp"). > Therefore, the most important question is why the hell do we need a third way? > > [...] > > * The backing-memory is accounted to the process that owns the file and is not subject to mount-quotas. ^ Am I right in thinking the first part of this sentence cannot be relied on? The [memfd_create() code](https://elixir.bootlin.com/linux/latest/source/mm/shmem.c#L3679) is literally implemented as an "[unlinked file living in [a] tmpfs which must be kernel internal](https://elixir.bootlin.com/linux/latest/source/mm/shmem.c#L4254) ". Tracing the code, I understand it differs in not implementing LSM checks, also memfds are created to support "seals", as the blog post goes on to explain. However, I'm extremely sceptical that memfds are _accounted_ differently to a tmpfile in principle. Specifically, when the [OOM-killer](https://www.kernel.org/doc/gorman/html/understand/understand016.html) comes knocking, I don't think it will account for memory held by memfds. This could total up to 50% of RAM - the value of the [size= option for tmpfs](https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt) . The kernel doesn't set a different value for the internal tmpfs, so it would use the default size of 50%. So I think we can generally expect processes which hold a large memfd, but no other significant memory allocations, will not be OOM-killed. Is that correct?

On Linux, POSIX shared memory objects use a tmpfs via /dev/shm. A tmpfs in turn is said to "live completely in the page cache" (I'm assuming swap has not been enabled). I am wondering what the consistency / no-tearing guarantees are when using a POSIX SHM object, mmaped into a programs address space. Example: Assume a POSIX SHM object shared between two processes A and B, both mmaped into their respective address space. The size of that object is 8kB or two pages, assuming 4kB pages and the object being page-aligned. 1. A issues two sequential writes, the first writes into the first page (first 4k block), the second into the second page. 2. B polls the shared object / both pages. Is it possible that the reads of B are torn, meaning that B reads a fresh and updated second page but a stale first page? \ https://www.man7.org/linux/man-pages/man7/shm_overview.7.html \ https://www.kernel.org/doc/html/latest/filesystems/tmpfs.html \ This would be the associated pseudo-code in C:

int fd = shm_open(...);
void *share = mmap(0, 8192, $flags, fd, 0);
memcpy(share       , data1, 4096);
memcpy(share + 4096, data2, 4096);

I've been curious lately about the various Linux kernel memory based filesystems. *Note:* As far as I'm concerned, the questions below should be considered more or less optional when compared with a better understanding of that posed in the title. I ask them below because I believe answering them can better help me to understand the differences, but as my understanding is admittedly limited, it follows that others may know better. I am prepared to accept any answer that enriches my understanding of the differences between the three filesystems mentioned in the title. Ultimately I think I'd like to mount a usable filesystem with *hugepages,* though some light research (and still lighter tinkering) has led me to believe that a *rewritable hugepage mount* is not an option. Am I mistaken? What are the mechanics at play here? Also regarding *hugepages:* uname -a 3.13.3-1-MANJARO \ #1 SMP PREEMPT \ x86_64 GNU/Linux tail -n8 /proc/meminfo HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 8223772 kB DirectMap2M: 16924672 kB DirectMap1G: 2097152 kB *(Here are full-text versions of /proc/meminfo and /proc/cpuinfo )* What's going on in the above? Am I already allocating *hugepages?* Is there a difference between *DirectMap* memory pages and *hugepages?* **Update** After a bit of a nudge from @Gilles, I've added 4 more lines above and it seems there must be a difference, though I'd never heard of *DirectMap* before pulling that *tail* yesterday... maybe *DMI* or something? Only a little more... Failing any success with the *hugepages* endeavor, and assuming harddisk backups of any image files, what are the risks of mounting loops from *tmpfs?* Is my filesystem being *swapped* the worst-case scenario? I understand *tmpfs* is mounted filesystem cache - can my mounted loopfile be pressured out of memory? Are there mitigating actions I can take to avoid this? Last - exactly what is *shm,* anyway? How does it differ from or include either *hugepages* or *tmpfs?*

I'm interested in writing a program that can create two files, second file would be a "view" of first file and if modified, the first file would also be modified. Is this possible to do with mmap or at all? I know that using mmap i can have shared memory in RAM, but I need shared memory in non-volatile memory aka hard drive. I cannot copy the first file or load it fully into RAM since I assume the file can be of very large size (GBs). After I find how to have the second file showing memory subset of first file I plan to make 3 files, first roleplaing as container and second and third showing different subsets of the first file. Second and third file are to be formatted with filesystem so that first file container holds in it's memory two filesystems accessible via second and third file. This I plan to accomplish by attaching the second and third files as loopback devices and mounting them. Is this doable of am I not seeing something?

I've observed that I can create a shared memory object and give it ostensibly any size with ftruncate, regardless of the resource limits on my system. The code below sets the size to 262 TB, and indeed stat() reports that is the size. However, 262 TB is greater than my /dev/shm mount, and I definitely do not have 262 TB of memory or swap available on the system anyway. I expected an OOM error, but I did not see my system memory usage change significantly. What's going on here? Why does ftruncate succeed with sizes beyond my system's resource limits? I assume there is some special interaction here between ftruncate and shared memory objects (or perhaps more generally, with tmpfs).

import os
import _posixshmem
try:
    fd = _posixshmem.shm_open("test", os.O_CREAT | os.O_EXCL | os.O_RDWR, mode=0o600)
    os.ftruncate(fd, 52428800 * 5000000)
    print(os.stat(fd))
finally:
    _posixshmem.shm_unlink("test")

os.stat_result(st_mode=33152, st_ino=244, st_dev=24, st_nlink=1, st_uid=1000, st_gid=100, st_size=262144000000000, st_atime=1693167355, st_mtime=1693167355, st_ctime=1693167355)

I'm curious because today the only way I know how to give two different processes the same shared-memory is through a memory-mapped file, in other words, both processes open the same memory-mapped file and write/read to/from it. That has penalties / drawbacks as the operating system needs to swap between disk and memory. Apologies in advance if that is a silly question, but is there such a thing as a pure shared memory between processes, not backed by a file. If yes, how would the processes get a hold of it if not using a memory-mapped file or /dev/shm file?

I am trying to figure out what will happen if I use the IPC_CREAT flag with shmget(). I have used a key of an already created shared memory segment from another process. When I did so, the calling process has shared actually this memory segment with the old process ( For sure I have attached this memory segment using shmat()). So can I conclude that if I use IPC_CREAT with a shared memory segment that is already created the effect is that this memory will be shared with the calling process ?

We can create and attach a shared memory to a process using shmget() and shmat(). What will happen if we don't destroy and detach the shared memory in Ubuntu OS ? According to my understanding it will still exist in the physical memory until the system is restarted, but why is that ? I mean we can load the physical memory with many shared memory blocks through creating multiple of shared memory blocks and this can fill up the physical memory inefficiently ???

Using shmget(), we can allocate a shared memory block of certain size in bytes and using shmat(), we attach this shared memory block to the address space of the calling process. I need to check my understanding:
We have one process creating and attaching a shared memory using shmget() and shmat(), and another process that is attaching this shared memory to its address space using shmat(). Now the return address of this shared memory (using the shmat()) is different in the two processes because this is an virtual address. However, the shared memory block itself has a single base physical address that is mapped to different virtual addresses of the processes sharing this memory. Is this correct?

We are debugging a situation where the cached/shared memory increase and increase until the system reach OOM-killer. We have set shmax and shmall in sysctl.conf but without any visible effect. Do we need to enable something more for shmax/shmall to work? Or can some part of the system go beyond this limit, how hard is it enforced? Can buggy user space application or only bugs in kernel/drivers cause it? The application that we debug use graphics and video decoding. Can drivers go beyond the max limits? kernel.shmmax = 2147483648 kernel.shmall = 524288 Linux kernel is 5.15.71(from Yocto meta-intel). Our system has 4GB ram and no swap (we tried to enable swap but it did not help with the stability of the system). We use Wayland/weston but not systemd. We set the value in sysctl.conf and reboot for it to take effect. We also confirmed the values with ipcs. We tried to set the shared memory to max 2 GB. ipcs -l ------ Shared Memory Limits -------- max number of segments = 4096 max seg size (kbytes) = 2097152 max total shared memory (kbytes) = 2097152 min seg size (bytes) = 1 Here is some example output from free, meminfo, smem, etc a few minutes before it reaches OOM. free -w total used free shared buffers cache available Mem: 3844036 479428 263444 2711864 11324 3089840 585716 Swap: 0 0 0 #### cat /proc/meminfo MemTotal: 3844036 kB MemFree: 262680 kB MemAvailable: 584940 kB Buffers: 11324 kB Cached: 3055620 kB SwapCached: 0 kB Active: 98764 kB Inactive: 645792 kB Active(anon): 732 kB Inactive(anon): 394288 kB Active(file): 98032 kB Inactive(file): 251504 kB Unevictable: 2708620 kB Mlocked: 100 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 12 kB Writeback: 0 kB AnonPages: 386388 kB Mapped: 162732 kB Shmem: 2711864 kB KReclaimable: 34208 kB Slab: 68656 kB SReclaimable: 34208 kB SUnreclaim: 34448 kB KernelStack: 4640 kB PageTables: 5904 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 1922016 kB Committed_AS: 4068728 kB VmallocTotal: 34359738367 kB VmallocUsed: 15104 kB VmallocChunk: 0 kB Percpu: 1040 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 72236 kB DirectMap2M: 3938304 kB DirectMap1G: 2097152 kB #### smem PID User Command Swap USS PSS RSS ….. 1306 weston /usr/libexec/wpe-webkit-1.1 0 27192 51419 98928 1379 weston /usr/libexec/wpe-webkit-1.1 0 190268 214958 266040 Area Used Cache Noncache firmware/hardware 0 0 0 kernel image 0 0 0 kernel dynamic memory 3030848 2938432 92416 userspace memory 555656 162732 392924 free memory 257532 257532 0 Map PIDs AVGPSS PSS …… /usr/lib/libcrypto.so.3 20 527 10544 /usr/lib/dri/iris_dri.so 5 2196 10982 /usr/lib/dri/iHD_drv_video.so 1 20356 20356 /usr/lib/libWPEWebKit-1.1.so.0.2.6 5 14539 72697 [heap] 45 2060 92700 45 5970 268688 Edit: Added df info for tmpfs. The tmpfs mounts showed with df does not show any extra ordinary size increase. /# df -h Filesystem Size Used Avail Use% Mounted on /dev/root 9.8G 1.9G 7.4G 21% / devtmpfs 1.9G 2.1M 1.9G 1% /dev tmpfs 1.9G 636K 1.9G 1% /run tmpfs 751M 5.8M 745M 1% /var/volatile tmpfs 40K 0 40K 0% /mnt/.psplash

I am attempting to use a USB GPS without PPS dongle as the sole time source on Ubuntu 18.04. GPSD appears to be working correctly since cgps reports a 3D fix. However, I can't get chrony to see the GPS information.

# gpsd -ND 8 /dev/ttyUSB0
gpsd:INFO: launching (Version 3.17)
gpsd:IO: opening IPv4 socket
gpsd:SPIN: passivesock_af() -> 3
gpsd:IO: opening IPv6 socket
gpsd:SPIN: passivesock_af() -> 4
gpsd:INFO: listening on port gpsd
gpsd:PROG: NTP: shmat(4718600,0,0) succeeded, segment 0
gpsd:PROG: NTP: shmat(4751369,0,0) succeeded, segment 1
gpsd:PROG: NTP: shmat(4784139,0,0) succeeded, segment 2
gpsd:PROG: NTP: shmat(4816908,0,0) succeeded, segment 3
gpsd:PROG: NTP: shmat(4849677,0,0) succeeded, segment 4
gpsd:PROG: NTP: shmat(4882446,0,0) succeeded, segment 5
gpsd:PROG: NTP: shmat(4915215,0,0) succeeded, segment 6
gpsd:PROG: NTP: shmat(4947984,0,0) succeeded, segment 7
gpsd:PROG: successfully connected to the DBUS system bus
gpsd:PROG: shmget(0x47505344, 8936, 0666) for SHM export succeeded
gpsd:PROG: shmat() for SHM export succeeded, segment 4980753
gpsd:INFO: stashing device /dev/ttyUSB0 at slot 0
gpsd:INFO: running with effective group ID 20
gpsd:INFO: running with effective user ID 128
gpsd:INFO: startup at 2022-12-06T16:50:17.000Z (1670345417)

chrony.conf:

refclock SHM 0 poll 3 refid gps1
keyfile /etc/chrony/chrony.keys
driftfile /var/lib/chrony/chrony.drift
logdir /var/log/chrony
maxupdateskew 10
rtcsync
makestep 1 3

# chronyd -df /etc/chrony/chrony.conf
2022-12-06T16:52:15Z chronyd version 3.2 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SECHASH +SIGND +ASYNCDNS +IPV6 -DEBUG)
2022-12-06T16:52:15Z Frequency -22.003 +/- 4.168 ppm read from /var/lib/chrony/chrony.drift

# chronyc sources
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
#? GPS0                          0   4     0     -     +0ns[   +0ns] +/-    0ns

I think the issue is related to shared memory since I think chrony/gpsd should be utilizing SHM ID 0 but it doesn't seem to exist.

# ipcs -m

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status      
0x00000000 262144     localuser    600        67108864   2          dest         
0x00000000 229377     localuser    600        16777216   2          dest         
0x00000000 360450     localuser    600        524288     2          dest         
0x00000000 393219     localuser    600        524288     2          dest         
0x00000000 491524     localuser    600        524288     2          dest         
0x00000000 5177349    localuser    600        524288     2          dest         
0x00000000 5210118    localuser    600        524288     2          dest         
0x00000000 655367     localuser    600        524288     2          dest         
0x4e545030 4718600    root       600        96         2                       
0x4e545031 4751369    root       600        96         1                       
0x00000000 2293770    localuser    600        16777216   2          dest         
0x4e545032 4784139    root       666        96         1                       
0x4e545033 4816908    root       666        96         1                       
0x4e545034 4849677    root       666        96         1                       
0x4e545035 4882446    root       666        96         1                       
0x4e545036 4915215    root       666        96         1                       
0x4e545037 4947984    root       666        96         1                       
0x47505344 4980753    root       666        8936       1                       
0x00000000 5472274    localuser    600        16384      2          dest         
0x4e9c5038 5570579    root       600        96         0                       
0x00000000 5308436    localuser    600        532480     2          dest         
0x00000000 5341205    localuser    600        532480     2          dest         
0x00000000 5439510    localuser    600        16384      2          dest         
0x4ea05041 5603351    root       600        96         0

I've tried starting chronyd before gpsd and that didn't help. Any ideas? Am I missing something obvious? Thanks in advance. Similar question: Using gpsd and chrony without pulse per second Another source I've read: GPSD Time Service HOWTO

I'm currently writing a module for Linux, and I want to pass a queue of data from kernel to user space (my program in userland being responsible for read this data - and then responsible for writing those to a file), and my approach is to get a memory location in user space and push data from kernel to it. How can I implement it? Do you have a better approach? I'm Beginner, and any guides can be nice. Before that, I try to push this data to user space with IOCTL and PROCFS but that approach is not a good idea and I lost some data.

Using Manjaro / Arch linux, I wanted to install another browser. However, no matter whether I installed Opera or Chromium (via pacman) I always get an error when executing it (from both Application Launcher and shell). Running **Chromium** it from the shell I get: $ chromium [11452:11452:0914/225931.419271:ERROR:platform_shared_memory_region_posix.cc(217)] Creating shared memory in /dev/shm/.org.chromium.Chromium.5FNz4h failed: Permission denied (13) [11452:11452:0914/225931.419316:ERROR:platform_shared_memory_region_posix.cc(220)] Unable to access(W_OK|X_OK) /dev/shm: Permission denied (13) [11452:11452:0914/225931.419320:FATAL:platform_shared_memory_region_posix.cc(222)] This is frequently caused by incorrect permissions on /dev/shm. Try 'sudo chmod 1777 /dev/shm' to fix. [0914/225931.424145:ERROR:elf_dynamic_array_reader.h(64)] tag not found [0914/225931.424372:ERROR:elf_dynamic_array_reader.h(64)] tag not found [0914/225931.424389:ERROR:elf_dynamic_array_reader.h(64)] tag not found zsh: trace trap (core dumped) chromium Similar error for **Opera**: $ opera [11882:11882:0914/230407.927506:ERROR:platform_shared_memory_region_posix.cc(217)] Creating shared memory in /dev/shm/.org.chromium.Chromium.VS4HRD failed: Permission denied (13) [11882:11882:0914/230407.927725:ERROR:platform_shared_memory_region_posix.cc(220)] Unable to access(W_OK|X_OK) /dev/shm: Permission denied (13) [11882:11882:0914/230407.927735:FATAL:platform_shared_memory_region_posix.cc(222)] This is frequently caused by incorrect permissions on /dev/shm. Try 'sudo chmod 1777 /dev/shm' to fix. Discarded=1 zsh: illegal hardware instruction (core dumped) opera Discarded=1 The by default installed Firefox however is working fine (so no network issues or such kind). I could imagine it is a configuration / safety issue (like Apparmor etc.). Any ideas?

I'm on Mac Monterey 12.1 and increased my shared memory limits in /etc/sysctl.conf file: kern.sysv.shmmax: 16777216 kern.sysv.shmmin: 1 kern.sysv.shmmni: 128 kern.sysv.shmseg: 512 kern.sysv.shmall: 4096 security.mac.posixshm_enforce: 1 security.mac.sysvshm_enforce: 1 and restarted (and shut down) the machine. However, after restarting when i run sysctl -A | grep shm it shows values different to the file: kern.sysv.shmall: 1024 kern.sysv.shmmax: 4194304 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 8 security.mac.posixshm_enforce: 1 security.mac.sysvshm_enforce: 1 even though /etc/sysctl.conf still shows the changes as persisted. In other words, I have no idea where sysctl -A | grep shm is getting these values from. Does anyone know where I need to modify them?