I'm aware of Linux softirqs **may** run within specific per-cpu kernel thread context -- ksoftirqd/[cpu_id]. Kernel threads are similar to user-space threads, however they only execute kernel code in kernel mode (they don't have user mode stacks). Suppose that at any given point in time, a ksoftirqd kernel thread is running a softirq (i.e. deferred work from a Top half interrupt handler). As far as I know, kernel threads share the virtual address range to which kernel code (including loaded kernel modules) is mapped to. What would happen if the softirq bottom half kernel code attempted to access a virtual address (VA) within the user space VA range ? I'm not sure an error would actually occur, but I believe the result of such an access would be unpredictable though. Does it make sense ? Thanks. BTW, I run the following trace-cmd command to dig into some details root@eve-ng62-28:~# trace-cmd start -e net:netif_receive_skb_entry root@eve-ng62-28:~# trace-cmd show # tracer: nop # # entries-in-buffer/entries-written: 2/2 #P:48 # # _-----=> irqs-off/BH-disabled # / _----=> need-resched # | / _---=> hardirq/softirq # || / _--=> preempt-depth # ||| / _-=> migrate-disable # |||| / delay # TASK-PID CPU# ||||| TIMESTAMP FUNCTION # | | | ||||| | | qemu-system-x86-2064265 b.... 599782.975041: netif_receive_skb_entry: dev=vunl0_2_0 napi_id=0x0 queue_mapping=1 skbaddr=00000000b36764b3 vlan_tagged=0 vlan_proto=0x0000 vlan_tci=0x0000 protocol=0x0800 ip_summed=0 hash=0x00000000 l4_hash=0 len=84 data_len=0 truesize=768 mac_header_valid=1 mac_header=-14 nr_frags=0 gso_size=0 gso_type=0x0 qemu-system-x86-2064265 b.... 599783.973971: netif_receive_skb_entry: dev=vunl0_2_0 napi_id=0x0 queue_mapping=1 skbaddr=00000000dd8af289 vlan_tagged=0 vlan_proto=0x0000 vlan_tci=0x0000 protocol=0x0800 ip_summed=0 hash=0x00000000 l4_hash=0 len=84 data_len=0 truesize=768 mac_header_valid=1 mac_header=-14 nr_frags=0 gso_size=0 gso_type=0x0 As you can check, netif_receive_skb_entry tracepoint within netif_receive_skb() kernel function/routine is triggered. It has the b value for the irqs-off/BH-disabled field. Does it mean netif_receive_skb() is actually running within a BH softirq with Bottom Half(BH) disabled ?

I have a problem with high load average (~2) on my (personal laptop) computer for a long time now. I am running Arch Linux. If I remember correctly, the problem started with a certain kernel update, initially I thought it was related to this bug . The problem was not solved though, when the bug was fixed. I did not really care as I thought it is still a bug, because the performance did not seem to suffer. What made me curious is that, recently, I had a moment of super low load average (~0) while idling. After a reboot, everything went back to "normal", with high load average. So I started investigating: % uptime 14:31:04 up 2:22, 1 user, load average: 1.96, 1.98, 1.99 So far nothing new. Then I tried top: % top -b -n 1 top - 14:33:52 up 2:25, 1 user, load average: 2.02, 2.07, 2.02 Tasks: 146 total, 2 running, 144 sleeping, 0 stopped, 0 zombie %Cpu0 : 2.6/0.9 3[|||| ] %Cpu1 : 2.7/0.9 4[|||| ] %Cpu2 : 2.7/1.0 4[|||| ] %Cpu3 : 2.7/0.8 3[|||| ] GiB Mem :228125107552256.0/7.712 [ GiB Swap: 0.0/7.904 [ ] PID USER PR NI VIRT RES %CPU %MEM TIME+ S COMMAND 2 root 20 0 0.0m 0.0m 0.0 0.0 0:00.00 S kthreadd 404 root 20 0 0.0m 0.0m 0.0 0.0 0:01.09 D `- rtsx_usb_ms_2 1854 root 20 0 0.0m 0.0m 0.0 0.0 0:06.03 D `- kworker/0:2 I cut out all the processes and kernel threads except those two. Here we can see already some suspicious kernel threads (state D). And some suspicious Mem value (see edit)... Looking at CPU: % mpstat Linux 4.13.12-1-ARCH (arch) 30.11.2017 _x86_64_ (4 CPU) 14:36:09 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 14:36:09 all 2.66 0.00 0.88 1.56 0.00 0.01 0.00 0.00 0.00 94.90 % sar -u 1 30 Linux 4.13.12-1-ARCH (arch) 30.11.2017 _x86_64_ (4 CPU) 14:37:04 CPU %user %nice %system %iowait %steal %idle 14:37:05 all 1.00 0.00 0.75 0.00 0.00 98.25 14:37:06 all 1.76 0.00 0.50 0.00 0.00 97.74 14:37:07 all 1.00 0.00 0.25 0.00 0.00 98.75 14:37:08 all 0.50 0.00 0.50 0.00 0.00 99.00 14:37:09 all 0.50 0.00 0.50 0.25 0.00 98.75 14:37:10 all 0.50 0.00 0.50 6.03 0.00 92.96 14:37:11 all 0.75 0.00 0.50 11.75 0.00 87.00 14:37:12 all 0.50 0.00 0.25 0.00 0.00 99.25 [ . . . ] 14:37:21 all 1.26 0.00 0.76 0.00 0.00 97.98 14:37:22 all 0.75 0.00 0.25 2.26 0.00 96.73 14:37:23 all 0.50 0.00 0.50 16.83 0.00 82.16 14:37:24 all 0.75 0.00 0.50 0.00 0.00 98.74 14:37:25 all 0.50 0.00 0.50 0.00 0.00 98.99 14:37:26 all 0.76 0.00 0.50 7.56 0.00 91.18 14:37:27 all 0.25 0.00 0.51 0.00 0.00 99.24 14:37:28 all 1.00 0.00 0.75 0.25 0.00 98.00 14:37:29 all 0.25 0.00 0.76 0.00 0.00 98.99 14:37:30 all 0.75 0.00 0.50 0.00 0.00 98.74 14:37:31 all 0.75 0.00 0.50 3.27 0.00 95.48 14:37:32 all 0.51 0.00 0.51 13.16 0.00 85.82 14:37:33 all 0.75 0.00 0.50 0.25 0.00 98.49 14:37:34 all 1.26 0.00 0.75 0.00 0.00 97.99 Average: all 0.71 0.00 0.56 2.06 0.00 96.67 reveals some peaks in I/O wait. The best guess so far. Looking closer: % iostat -x 1 30 Linux 4.13.12-1-ARCH (arch) 30.11.2017 _x86_64_ (4 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 2.60 0.00 0.87 1.55 0.00 94.98 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.93 3.00 3.71 1.94 95.04 102.27 69.91 0.60 106.78 16.56 279.32 14.47 8.17 avg-cpu: %user %nice %system %iowait %steal %idle 0.75 0.00 0.75 0.25 0.00 98.25 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.01 13.00 0.00 13.00 10.00 1.00 [ . . . ] avg-cpu: %user %nice %system %iowait %steal %idle 0.50 0.00 0.50 17.04 0.00 81.95 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 8.00 0.00 2.00 0.00 40.00 40.00 0.69 346.50 0.00 346.50 346.50 69.30 [ . . . ] avg-cpu: %user %nice %system %iowait %steal %idle 0.25 0.00 0.50 7.29 0.00 91.96 [ . . . ] avg-cpu: %user %nice %system %iowait %steal %idle 1.00 0.00 0.75 16.96 0.00 81.30 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 5.00 0.00 2.00 0.00 28.00 28.00 0.71 357.00 0.00 357.00 356.50 71.30 [ . . . ] avg-cpu: %user %nice %system %iowait %steal %idle 0.50 0.00 0.50 0.00 0.00 99.00 Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Looking at processes with uninterruptable sleep: % for x in seq 1 1 10; do ps -eo state,pid,cmd | grep "^D"; echo "----"; sleep 5; done D 404 [rtsx_usb_ms_2] D 1854 [kworker/0:2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] D 1854 [kworker/0:2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] D 1854 [kworker/0:2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] ---- D 404 [rtsx_usb_ms_2] D 1854 [kworker/0:2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] D 1854 [kworker/0:2] D 2877 [kworker/0:0] ---- D 404 [rtsx_usb_ms_2] D 3177 [kworker/u32:4] ---- and last thing I did: % vmstat 1 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 1 0 5010040 123612 1220080 0 0 23 25 111 433 3 1 95 2 0 0 0 0 5006256 123612 1224164 0 0 0 96 186 839 1 1 97 1 0 1 0 0 5006132 123612 1224164 0 0 0 0 175 714 1 0 99 0 0 0 0 0 5003156 123612 1224156 0 0 0 0 234 1009 2 1 98 0 0 0 0 0 5003156 123612 1224156 0 0 0 0 161 680 0 0 99 0 0 0 1 0 5003156 123616 1224156 0 0 0 60 214 786 1 1 94 5 0 0 0 0 5003280 123620 1224156 0 0 0 4 226 776 1 0 88 11 0 1 0 0 5003156 123620 1224156 0 0 0 0 210 733 1 0 99 0 0 0 0 0 5005388 123620 1224156 0 0 0 0 159 747 1 0 99 0 0 0 0 0 5005388 123620 1224156 0 0 0 0 233 803 1 0 99 0 0 0 0 0 5005512 123620 1224156 0 0 0 0 152 670 1 0 99 0 0 0 0 0 5009664 123620 1220060 0 0 0 0 240 914 1 1 99 0 0 0 0 0 5009540 123620 1220060 0 0 0 0 237 833 1 1 99 0 0 0 0 0 5009664 123620 1220060 0 0 0 0 166 999 1 1 99 0 0 0 1 0 5009664 123620 1220060 0 0 0 4 168 700 1 0 88 11 0 0 0 0 5009540 123628 1220060 0 0 0 12 207 778 1 1 91 8 0 0 0 0 5009788 123628 1220064 0 0 0 0 189 717 0 1 99 0 0 0 0 0 5009664 123628 1220064 0 0 0 0 243 1453 1 1 98 0 0 0 0 0 5009044 123628 1220576 0 0 0 0 166 708 1 0 99 0 0 0 0 0 5009168 123628 1220576 0 0 0 0 146 663 1 0 99 0 0 0 0 0 5009540 123628 1220064 0 0 0 0 175 705 1 1 99 0 0 0 1 0 5009292 123632 1220128 0 0 0 8 223 908 1 0 99 0 0 ^C Now I still don't know what the problem is, but it looks like it comes from some peak I/O operations. There are some suspicious kernel threads. Any further ideas? What else could I do to investigate? **edit:** The Mem value seems strange, but it just occured very recently, a week ago or so, everything seemed to be normal. And % free total used free shared buff/cache available Mem: 8086240 1913860 4824764 133880 1347616 6231856 Swap: 8288252 0 8288252 seems to be fine though. **edit2:** First results of testing sar monitoring my system (very frequently, intervals of 1 second, but for a short duration, to get the peaks): Linux 4.13.12-1-ARCH (arch) 01.12.2017 _x86_64_ (4 CPU) 12:36:25 CPU %user %nice %system %iowait %steal %idle 12:36:26 all 0.50 0.00 0.50 0.00 0.00 99.00 12:36:27 all 0.50 0.00 0.50 0.25 0.00 98.74 12:36:28 all 0.50 0.00 0.75 0.00 0.00 98.75 12:36:29 all 0.50 0.00 0.25 7.52 0.00 91.73 12:36:30 all 0.25 0.00 0.75 9.77 0.00 89.22 12:36:31 all 0.25 0.00 0.75 0.00 0.00 98.99 12:36:32 all 1.00 0.00 0.50 0.25 0.00 98.25 12:36:33 all 1.00 0.00 1.00 0.00 0.00 98.00 12:36:34 all 0.25 0.00 0.25 0.25 0.00 99.24 12:36:35 all 0.50 0.25 0.75 33.25 0.00 65.25 12:36:36 all 0.50 0.00 0.75 0.25 0.00 98.50 12:36:37 all 0.75 0.00 0.25 0.00 0.00 99.00 12:36:38 all 0.25 0.00 0.50 0.00 0.00 99.24 12:36:39 all 0.50 0.00 0.50 0.00 0.00 99.00 12:36:40 all 0.50 0.25 0.50 10.75 0.00 88.00 Average: all 0.52 0.03 0.57 4.16 0.00 94.72 Network (-n) seems to be alright. Looking at devices (-d) reveals: Linux 4.13.12-1-ARCH (arch) 01.12.2017 _x86_64_ (4 CPU) 12:36:25 DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util 12:36:26 dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:26 dev8-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 [ . . . ] 12:36:29 dev8-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:30 dev8-0 2.00 0.00 88.00 44.00 0.41 355.00 207.00 41.40 12:36:30 dev8-1 2.00 0.00 88.00 44.00 0.41 355.00 207.00 41.40 12:36:30 dev8-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:30 dev8-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:30 dev8-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:30 dev8-5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:30 dev8-6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:30 dev8-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:31 dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:31 dev8-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 [ . . . ] 12:36:34 dev8-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:35 dev8-0 2.00 0.00 24.00 12.00 0.70 348.50 348.00 69.60 12:36:35 dev8-1 2.00 0.00 24.00 12.00 0.70 348.50 348.00 69.60 12:36:35 dev8-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:35 dev8-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:35 dev8-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:35 dev8-5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:35 dev8-6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:35 dev8-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:36 dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:36 dev8-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 [ . . . ] 12:36:40 dev8-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-0 0.27 0.00 7.47 28.00 0.12 351.75 455.75 12.15 Average: dev8-1 0.27 0.00 7.47 28.00 0.12 351.75 455.75 12.15 Average: dev8-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: dev8-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 and -b gives: Linux 4.13.12-1-ARCH (arch) 01.12.2017 _x86_64_ (4 CPU) 12:36:25 tps rtps wtps bread/s bwrtn/s 12:36:26 0.00 0.00 0.00 0.00 0.00 12:36:27 0.00 0.00 0.00 0.00 0.00 12:36:28 0.00 0.00 0.00 0.00 0.00 12:36:29 0.00 0.00 0.00 0.00 0.00 12:36:30 2.00 0.00 2.00 0.00 88.00 12:36:31 0.00 0.00 0.00 0.00 0.00 12:36:32 0.00 0.00 0.00 0.00 0.00 12:36:33 0.00 0.00 0.00 0.00 0.00 12:36:34 0.00 0.00 0.00 0.00 0.00 12:36:35 2.00 0.00 2.00 0.00 24.00 12:36:36 0.00 0.00 0.00 0.00 0.00 12:36:37 0.00 0.00 0.00 0.00 0.00 12:36:38 0.00 0.00 0.00 0.00 0.00 12:36:39 0.00 0.00 0.00 0.00 0.00 12:36:40 0.00 0.00 0.00 0.00 0.00 Average: 0.27 0.00 0.27 0.00 7.47 So I assume the issue seems to be related to my hard drive (?). Because the I/O is on partition 1 (my root partition), it should be somewhere outside of /var which has an extra partition. The other partitions are data partitions and not system related. **edit3:** Even more data to that specific peak: paging looks fine (from my perspective with limited knowledge) Linux 4.13.12-1-ARCH (arch) 01.12.2017 _x86_64_ (4 CPU) 12:36:25 pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff 12:36:26 0.00 0.00 0.00 0.00 2233.00 0.00 0.00 0.00 0.00 12:36:27 0.00 0.00 0.00 0.00 88.00 0.00 0.00 0.00 0.00 12:36:28 0.00 0.00 766.00 0.00 185.00 0.00 0.00 0.00 0.00 12:36:29 0.00 40.00 0.00 0.00 47.00 0.00 0.00 0.00 0.00 12:36:30 0.00 4.00 0.00 0.00 45.00 0.00 0.00 0.00 0.00 12:36:31 0.00 0.00 1.00 0.00 46.00 0.00 0.00 0.00 0.00 12:36:32 0.00 0.00 5.00 0.00 560.00 0.00 0.00 0.00 0.00 12:36:33 0.00 0.00 2.00 0.00 85.00 0.00 0.00 0.00 0.00 12:36:34 0.00 0.00 2.00 0.00 47.00 0.00 0.00 0.00 0.00 12:36:35 0.00 12.00 0.00 0.00 44.00 0.00 0.00 0.00 0.00 12:36:36 0.00 0.00 0.00 0.00 47.00 0.00 0.00 0.00 0.00 12:36:37 0.00 0.00 2.00 0.00 45.00 0.00 0.00 0.00 0.00 12:36:38 0.00 0.00 0.00 0.00 47.00 0.00 0.00 0.00 0.00 12:36:39 0.00 0.00 0.00 0.00 77.00 0.00 0.00 0.00 0.00 12:36:40 0.00 8.00 0.00 0.00 47.00 0.00 0.00 0.00 0.00 Average: 0.00 4.27 51.87 0.00 242.87 0.00 0.00 0.00 0.00 It looks like files were created during that peak (-v): Linux 4.13.12-1-ARCH (arch) 01.12.2017 _x86_64_ (4 CPU) 12:36:25 dentunusd file-nr inode-nr pty-nr 12:36:26 186520 4480 195468 2 [ . . . ] 12:36:34 186520 4480 195468 2 12:36:35 186520 4512 195468 2 [ . . . ] 12:36:40 186520 4512 195468 2 Average: 186520 4493 195468 2 **edit4:** It looks like some irq's are responsible. Running iotop -o -a (show only processes with i/o and accumulate them, so keep all processes that had i/o since the start of the program) resulted in: Total DISK READ : 0.00 B/s | Total DISK WRITE : 0.00 B/s Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 0.00 B/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 7 be/4 root 0.00 B 0.00 B 0.00 % 99.99 % [ksoftirqd/0] 17 be/4 root 0.00 B 0.00 B 0.00 % 99.99 % [ksoftirqd/1] 23 be/4 root 0.00 B 0.00 B 0.00 % 99.99 % [ksoftirqd/2] 29 be/4 root 0.00 B 0.00 B 0.00 % 99.99 % [ksoftirqd/3] 292 rt/4 root 0.00 B 0.00 B 0.00 % 99.99 % [i915/signal:0] [ . . . ] So, is this a thing? How could I continue...?

Ubuntu 12.10 alpha 2 works perfectly on my new 11" 2012 Macbook Air, but only when I boot with the "nointremap" option. Supposedly this is not good to use as a permanent solution. What exactly does this option do, and why is it "bad"?

I'm trying to create a petalinux image using the petalinux SDK. After I start a build I get an error when the process tries to build the system.dtb The error is "(phandle references): Reference to non-existent node or label "ps7_scugic_0" - which is relayed twice. I believe this is some sort of interrupt device used in the ZYNQ SoC and is visible and mapped in the xml file used to generate parts of this build. How can I create this node? Do I have to edit the dts and dtsi files?

*Extremely "noob" question:* I have a running sudo memtester ... on an Ubuntu 22.04 machine, but I gave it too much memory to test and it's taking [too much time](https://unix.stackexchange.com/q/789492/515980) (I see it running and updating, I made sure the RAM given to test was less than the free one). I'd like to stop it. I was thinking of trying with a simple Ctrl-c, but I wonder if that would cause some sort of lock-up, forcing me to reboot. Unfortunately the [man pages](https://man.archlinux.org/man/memtester.8.en) don't say anything about how to stop it.

I've noticed that I have high kernel time when doing a lot of network traffic in Linux. I also noticed that 45% of my CPU is spent doing interrupts. How can I further troubleshoot this?

(This is in the context of x86-64 Linux.) I am trying to write a high-reliability userland executable, and I have total control over the generated assembly. I don't want to rely on automatic stack allocation, so I would like to put the stack in a known location. Suppose I have calculated that my program uses at most 414 bytes of stack space (exactly). Is it safe to allocate 414 bytes in the .bss section and point RSP to the top? I want to ensure that no bytes outside this region are touched by stack management at any point. While I can be sure that *my* program won't write outside the region, I need to make some syscalls (using the syscall instruction), and I think at least some part of the kernel code operates in the calling executable context. Will it smash my stack? Also, interrupts can happen at any point in the program, and the story behind the "Red Zone" seems to suggest that an arbitrarily large region beyond RSP-128 can be written to at will by interrupt handlers, possibly mangling my data. What kinds of guarantees do I have about this behavior?

I am using a PC to control a robot at 1000Hz. My machine has Ubuntu 22.04 LTS with Linux 6.7 patched with PREEMPT_RT. The machine communicates with the robot via UDP trough a point-to-point Ethernet connection (full duplex, 1000Mbps). The Ethernet cable is CAT6 (as per the robot manual) and my machine's NIC is an Intel I219-V running the e1000e driver . The control loop is a synchronous read-execute-write-sleep loop, which means that each iteration starts with a blocking read from the UDP socket and ends with a write operation on the socket. The control thread is run with an high priority of 98. As the operating frequency is 1000Hz, each iteration should start 1ms after the start of the previous one. However, I am exeperiencing latency issues while running this application, as sometimes the read operation lasts 3ms to 4ms. I captured a packet trace by using Wireshark during the control loop execution. In the following trace, the IP of the local machine is 192.168.1.2 whereas the IP of the robot is 192.168.1.1. This is the result I got: As you can see after packet #11322 is received at 10.935, the next packet #11325 is received at 10.939 (after 4ms) together with packets #11328, #11330, #11332. I also used LTTng to trace both the Linux kernel and the user-space application, and it seems that when the problem occurs, no hardware interrupt is fired by the NIC during that time delay. Therefore I suspect a problem with the network driver or the NIC firmware. In this documentation page for the e1000e driver I found out that the InterruptThrottleRate parameter can be used to configure IRQ coalescing , however setting it to 0 did not solve the issue. I wonder wether I should try with a different NIC, but I don't have a criteria to select a proper one as I still haven't figured out the exact cause of the issue.

I am running the Linux 3.10 kernel on an ARM processor and writing a gpio driver. The system assigned interrupt number 182. I used the echo command to bind the core and wanted to bind the interrupt to CPU2, but it failed. cat /proc/irq/182/smp_affinity 4 cat /proc/irq/182/smp_affinity_list 2 However, I checked the following information and there was no problem. What is going on? My cpu2 and cpu3 are isolated. cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 29: 2000407 1993990 308737 1328118 GIC arch_timer 30: 0 0 0 0 GIC arch_timer 32: 0 0 0 0 GIC axp22_irq_chip 33: 410 0 0 0 GIC uart0 39: 1090 0 0 0 GIC twi0 40: 0 0 0 0 GIC twi1 41: 48 0 0 0 GIC twi2 43: 6211 0 115847 0 GIC spi1 54: 0 0 0 0 GIC sunxi_timer 59: 0 0 0 0 GIC 1c02000.dma-controller 60: 95854 0 31972 0 GIC PIN_GRP 66: 13154 0 0 0 GIC sunxi-mmc 71: 0 0 0 0 GIC ehci_hcd:usb1 72: 0 0 0 0 GIC ohci_hcd:usb4 76: 0 0 0 0 GIC dispaly 78: 0 0 0 0 GIC 1e80000.g2d 96: 0 0 0 0 GIC ohci_hcd:usb5 97: 0 0 0 0 GIC ohci_hcd:usb6 98: 0 0 0 0 GIC mdfs 108: 0 0 0 0 GIC ehci_hcd:usb2 110: 0 0 0 0 GIC ehci_hcd:usb3 117: 3069 0 0 0 GIC 1c50000.eth 120: 0 0 0 0 GIC twi3 121: 0 0 0 0 GIC twi4 125: 0 0 0 0 GIC DE-Interlace 128: 0 0 0 0 GIC 1000000.tr 182: 52594 75232 0 0 - spiio_irq IPI0: 0 0 0 0 CPU wakeup interrupts IPI1: 0 0 0 0 Timer broadcast interrupts IPI2: 460761 501962 125084 114712 Rescheduling interrupts IPI3: 2 2 4 4 Function call interrupts IPI4: 2 3 0 0 Single function call interrupts IPI5: 0 0 0 0 CPU stop interrupts IPI6: 0 0 0 0 completion interrupts IPI7: 0 0 0 0 CPU backtrace Err: 0

On the IRQ affinity tuning subject. I've realized that some IRQs cannot be reassigned to other cores. I've read that those are called "managed IRQs". On the Linux Foundation wiki, I've also found a reference about SMI (System managed Interrupts), I had a suspect that those are the same, wish a confirmation on that if possible please. Another thing is: Supposing that they are the same, how can I identify an SMI? I have going through the Kernel docs and man pages, but I didn't find anything on how to interpret the procfs directory: /proc/irq/ which usually contains the files: - /proc/irq//affinity_hint - /proc/irq//effective_affinity - /proc/irq//effective_affinity_list - /proc/irq//node: According to : "The node file on an SMP system shows the node to which the device using the IRQ reports itself as being attached." Does it mean that this represents the physical socket from which CPU the IRQs are being served from? - /proc/irq//spurious I know that these two give me the IRQ affinity in a binary and CPU-list format respectively: - /proc/irq//smp_affinity - /proc/irq//smp_affinity_list And also there is the sysfs directory /sys/kernel/irq/ that seems to be used as source of information for the contents of /proc/interrupts, that usually has these files: - /sys/kernel/irq//actions - /sys/kernel/irq//chip_name - /sys/kernel/irq//hwirq - /sys/kernel/irq//name - /sys/kernel/irq//per_cpu_count - /sys/kernel/irq//type - /sys/kernel/irq//wakeup Despite, there is a man page for /proc/interrrups it doesn't tell much other than what is already obvious by looking at the file, and the sysfs neither the procfs directory for IRQs has a man page. So, Am I missing something? How can I effectively identify an SMI without having to manually try to reassign it and have an Input/Output error (like shown below)?

# echo '7-8' > /proc/irq/53/smp_affinity_list
bash: echo: write error: Input/output error

https://access.redhat.com/solutions/4819541 https://wiki.linuxfoundation.org/realtime/documentation/howto/debugging/smi-latency/smi https://docs.kernel.org/filesystems/proc.html https://docs.kernel.org/admin-guide/kernel-parameters.html#cpu-lists https://man7.org/linux/man-pages/man5/proc_interrupts.5.html

I have read you are not supposed to use a blocking system call in an interrupt handler. For single processor systems this makes sense but, for multiprocessor systems will such a blocking operation truly block the whole system? Or only the processor it runs on?

If a CPU exception in kernel, such as bad memory access or invalid opcode, happens in the context of servicing a user process (like syscall or page-in), or in a kthread process, then, until panic_on_oops was set, useful information will be dumped and the task will just die. No panic. Sometimes the system remains absolutely usable. Enough for the user to attempt syncing their disks, gracefully terminating programs, and in other ways preparing for the emergency restart. But unfortunately, if the exception happens in atomic context (like interrupt or softirq), the action taken is **always Kernel Panic** (with description "Fatal exception in interrupt") — regardless of any settings or build-time configurations. It is sad. Why it's not possible to simulate a return from interrupt, and keep the system running in hope that some parts will still function? Thanks. I know that I can put an infinite mdelay() in the code path of "exception in interrupt" instead of panic(), to just stall the local CPU instead. But usually not much things can be done after that happens... Even if there are hundred CPUs in machine, they all soon lock up. So not very useful.

What are the main stacks in Linux? What I mean is, for example when an interrupt occurs what stack will be used for it, and what is the difference between user process and kernel process stacks?

I am using Linux on a microcontroller with 2 A72 cores, running my app on that, I am trying to measure runtime of a function in app using system call: clock_gettime(CLOCK_PROCESS_CPUTIME_ID , &tv); before and after the function, and calculate the CPU time function used, I do this for thousands of times to profile this function over long run of input data, it happens that I get very large runtime readings randomly in each run(sometimes not, but the overall curve of data is same and consistent), my explanation that this is system interrupts corrupts my runtime measuring, what do you think ?how to disable it? I tried to give my app highest priority but still have the issue

I have the following in a script: yes >/dev/null & pid=$! echo $pid sleep 2 kill -INT $pid sleep 2 ps aux | grep yes When I run it, the output shows that yes is still running by the end of the script. However, if I run the commands interactively then the process terminates successfully, as in the following: > yes >/dev/null & 9967 > kill -INT 9967 > ps aux | grep yes sean ... 0:00 grep yes Why does SIGINT terminate the process in the interactive instance but not in the scripted instance? EDIT Here's some supplementary information that may help to diagnose the issue. I wrote the following Go program to simulate the above script. package main import ( "fmt" "os" "os/exec" "time" ) func main() { yes := exec.Command("yes") if err := yes.Start(); err != nil { die("%v", err) } time.Sleep(time.Second*2) kill := exec.Command("kill", "-INT", fmt.Sprintf("%d", yes.Process.Pid)) if err := kill.Run(); err != nil { die("%v", err) } time.Sleep(time.Second*2) out, err := exec.Command("bash", "-c", "ps aux | grep yes").CombinedOutput() if err != nil { die("%v", err) } fmt.Println(string(out)) } func die(msg string, args ...interface{}) { fmt.Fprintf(os.Stderr, msg+"\n", args...) os.Exit(1) } I built it as main and running ./main in a script, and running ./main and ./main & interactively give the same, following, output: sean ... 0:01 [yes] sean ... 0:00 bash -c ps aux | grep yes sean ... 0:00 grep yes However, running ./main & in a script gives the following: sean ... 0:03 yes sean ... 0:00 bash -c ps aux | grep yes sean ... 0:00 grep yes This makes me believe that the difference has less to do on Bash's own job control, though I'm running all of this in a Bash shell.

According to what I've read so far, "when the kernel receives an interrupt, all the registered handlers are invoked." I understand that the registered handlers for each IRQ can be viewed via /proc/interrupts, and I also understand that the registered handlers come from the drivers that have invoked request_irq passing in a callback roughly of the form: irqreturn_t (*handler)(int, void *) Based on what I know, each of these interrupt handler callbacks associated with the particular IRQ should be invoked, and it is up to the handler to determine whether the interrupt should indeed be handled by it. If the handler should not handle the particular interrupt it must return the kernel macro IRQ_NONE. What I am having trouble understanding is, how each driver is expected to determine whether it should handle the interrupt or not. I suppose they can keep track internally if they're supposed to be expecting an interrupt. If so, I don't know how they'd be able to deal with the situation in which multiple drivers behind the same IRQ are expecting an interrupt. The reason I'm trying to understand these details is because I'm messing with the kexec mechanism to re-execute the kernel in the middle of system operation while playing with the reset pins and various registers on a PCIe bridge as well as a downstream PCI device. And in doing so, after a reboot I'm either getting kernel panics, or other drivers complaining that they're receiving interrupts even though no operation was taking place. How the handler decided that the interrupt should be handled by it is the mystery. Edit: In case it's relevant, the CPU architecture in question is x86.

The program ed, a minimal text editor, cannot be exited by sending it an interrupt through using Ctrl-C, instead printing the error message "?" to the console. Why doesn't ed just exit when it receives the interrupt? Surely there's no reason why a cryptic error message is more useful here than just exiting. This behavior leads many new users into the following sort of interaction: > $ ed > hello > ? > help > ? > exit > ? > quit > ? > ^C > ? > ^C > ? > ? > ? > ^D > $ su > # rm -f /bin/ed Such a tragic waste—easily avoidable if ed simply agreed to be interrupted. Another stubborn program exhibiting similar behavior is less which also doesn't appear to have much reason to ignore C-c. Why don't these programs just take a hint?

I was trying to install Parrot OS installation. But it appears the interrupt message if I install and it will stuck on the terminal I already Flashed parrot os using balena etcher and after that I tried to install using Try/Install button on Parrot OS and it will appear this interrupt and will stuck forever

.006525] __common_interuppt: 1.55 No irq handler for vector

It keep saying that interrupt 3x but 1.55, 2.55 and 3.55 Please help me to install Parrot Os Security. Thank you

I'm running a test program which generates a large number of threads and asynchronous I/O. I'm seeing very high counts of these interrupts in /proc/interrupts, the program cannot scale beyond a certain point because CPUs are 100% saturated with softirq processing. According to: http://www.kernel.org/doc/man-pages/online/pages/man5/proc.5.html CAL stands for 'remote function call interrupt' but that's all the info I can find on google. So...what does that mean? I have smp_affinity assigned for my I/O adapter cards and those are not the CPUs taking a huge number of CAL interrupts. Is it a sign my program is running with incorrect SMP affinity?

I'm not a expert by any means and I worry about doing wrong stuff... is there a standard / approved way to just run a script and have my interrupt tuned correctly on ubuntu 18.04 ? I also have thousands of machines to do this so I really don't want to reinvent the wheel