Let's say we have a CPU-intensive application called multi-threaded-application.out that is running on top of Ubuntu with a PID of 10000. It has 4 threads with tid 10001, 10002, 10003, and 10004. I want to know, at any given time, on which core each of these threads is being scheduled? I tried /proc//tasks//status, but I couldn't find any information regarding the core ID that is responsible for running the given thread. This question is somehow related to this one . Any help would be much appreciated.

A developer friend of mine recently asked the question: On a Linux system when a Java application runs which has threads, how do these threads appear to the underlying Linux OS? So what are Java threads?

I have a computation intensive process that I need to run multiple times on a multi-core processor but "top" isn't showing utilization or load in a useful way. For example, imagine my task runs in 1 minute in a single thread on a single core of my six core, 12 thread, SMT CPU. If I start the same task six times using six threads, it still finishes in 1 minute and top shows the load average as 6.0 and the cpu(s) at 50% us and 50% id. In the top process list, each of the six processes is showing 100% CPU. If I do the same thing but start 12 threads, it finishes the 12 jobs in 2 minutes and top shows the load average as 12.0, cpu(s) at 100% us 0% id, with 12 processes each at 100% CPU. Now, the 6 thread and 12 thread examples are both processing at the same fully loaded rate of completing 1/6 job per minute but why does top show the 6-thread case being 50% idle when clearly it isn't? Is there a better way of determining the actual load of the CPUs? This was run on a Ryzen 5600X processor on Ubuntu 24.12. Edit: top output for 12 tasks:

top - 08:35:37 up 54 days, 20:49,  3 users,  load average: 12.20, 6.70, 2.80
Tasks: 346 total,  13 running, 332 sleeping,   0 stopped,   1 zombie
%Cpu(s): 98.2 us,  1.7 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 st 
MiB Mem :  64221.7 total,   1572.7 free,   4983.4 used,  58684.1 buff/cache     
MiB Swap:   8192.0 total,   7863.7 free,    328.3 used.  59238.3 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                
2249765 user    20   0  126952  64132  51200 R 100.0   0.1   3:48.87 sonicLiquidFoam                                        
2249759 user    20   0  127060  64220  51200 R 100.0   0.1   3:48.93 sonicLiquidFoam                                        
2249757 user    20   0  126624  64064  51328 R 100.0   0.1   3:49.32 sonicLiquidFoam                                        
2249761 user    20   0  128276  64868  50688 R 100.0   0.1   3:47.65 sonicLiquidFoam                                        
2249762 user    20   0  127652  63688  50432 R 100.0   0.1   3:49.13 sonicLiquidFoam                                        
2249755 user    20   0  128844  66128  51200 R 100.0   0.1   3:46.06 sonicLiquidFoam                                        
2249766 user    20   0  126576  63952  51328 R 100.0   0.1   3:47.87 sonicLiquidFoam                                        
2249764 user    20   0  126612  63824  51072 R  99.0   0.1   3:48.59 sonicLiquidFoam                                        
2249760 user    20   0  126888  63972  51072 R  98.7   0.1   3:45.06 sonicLiquidFoam                                        
2249758 user    20   0  127500  64860  51200 R  97.7   0.1   3:48.64 sonicLiquidFoam                                        
2249763 user    20   0  127916  64944  51072 R  97.0   0.1   3:39.58 sonicLiquidFoam                                        
2249756 user    20   0  126828  63948  51072 R  96.0   0.1   3:48.77 sonicLiquidFoam

For 6 tasks:

top - 08:40:22 up 54 days, 20:53,  3 users,  load average: 6.11, 6.67, 3.90
Tasks: 335 total,   7 running, 327 sleeping,   0 stopped,   1 zombie
%Cpu(s): 50.0 us,  1.0 sy,  0.0 ni, 49.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st 
MiB Mem :  64221.7 total,   1616.2 free,   4914.6 used,  58710.3 buff/cache     
MiB Swap:   8192.0 total,   7863.7 free,    328.3 used.  59307.1 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                
2250032 user    20   0  127392  64676  51200 R 100.0   0.1   2:39.15 sonicLiquidFoam                                        
2250027 user    20   0  126828  63096  50176 R 100.0   0.1   2:39.23 sonicLiquidFoam                                        
2250028 user    20   0  127060  63260  50176 R 100.0   0.1   2:39.23 sonicLiquidFoam                                        
2250029 user    20   0  128844  66124  51200 R 100.0   0.1   2:39.12 sonicLiquidFoam                                        
2250030 user    20   0  128276  65508  51200 R 100.0   0.1   2:39.21 sonicLiquidFoam                                        
2250031 user    20   0  126596  63808  51072 R 100.0   0.1   2:39.21 sonicLiquidFoam

I am computing Monte-Carlo simulations using GNU Octave 4.0.0 on my 4-core PC. The simulation takes almost 4 hours to compute the script for 50,000 times (specific to my problem), which is a lot of time spent for computation. I was wondering if there is a way to run Octave on multiple cores simultaneously to reduce the time of computations. Thanks in advance.

I have a 100 M line file that fits in RAM on a GNU/Linux system. This is rather slow: sort bigfile > bigfile.sorted and does not use all 48 cores on my machine. How do I sort that file fast?

On chips 12th gen and newer in the Intel lineup, (and I'm told on ARM chips) there are distinct classes of cores with different clock speeds * P-cores for performance, higher clock speed * E-cores for energy efficiency, lower clock speed This is different from the chips that I understand where the frequency can be throttled but it's the same across cores. Is there a method to bind a thread to the class of P-cores or E-cores?

As I understand, the default stack size for a pthread on Linux is 16K. I am getting strange results on my 64-bit Ubuntu install. $ ulimit -s 8192 Also: pthread_attr_init(&attr); pthread_attr_getstacksize(&attr, &stacksize); printf("Thread stack size = %d bytes \n", stacksize); Prints Thread stack size = 8388608 bytes I'm quite sure the stack size is not "8388608". What could be wrong?

I use

tar -cJvf resultfile.tar.xz files_to_compress

to create tar.xz and

tar -xzvf resultfile.tar.xz

to extract the archive in current directory. How to use multi threading in both cases? I don't want to install any utilities.

I implemented thread's switching using functions like thread_schedule(), and thread_yield(), which aim at saving current thread's registers and state and loading next thread's one. I now want to implement the same thing using timer-interrupt in xv6, but I don't know how it works.

A Linux thread or forked process may change its name and/or its commandline as visible by ps or in the /proc filesystem. When using the python-setproctitle package, the same change occurs on /proc/pid/cmdline, /proc/pid/comm, the Name: line of /proc/pid/status and in the second field of /proc/pid/stat, where only cmdline is showing the full length and the other three locations are showing the first 15 chars of the changed name. When watching multithreaded ruby processes, it looks like the /proc/pid/cmdline remains unchanged but the other three locations are showing a thread name, truncated to 15 chars. man prctl tells that /proc/pid/comm is modified by the PR_SET_NAME operation of the prctl syscall but it does not say anything about /proc/pid/status and /proc/pid/stat. man proc says /proc/pid/comm provides a superset of prctl PR_SET_NAME which is not explained anymore. And it tells that the second field of /proc/pid/stat would still be available even if the process gets swapped out. When watching JVM processes, all the mentioned locations give identical contents for all threads (the three places other than cmdline all showing java), but jcmd pid Thread.print still shows different thread names for the existing threads, so it looks like Java threads are using some non-standard mechanism to change their name. Is /proc/pid/comm always identical to the Name: line of /proc/pid/status and the second field of /proc/pid/stat or are there circumstances where one of these three places is offering different contents ? Please provide an (easy to reproduce) example if differences are possible.

I have a command which I want to run on all free cores to speed up the execution time. Specifically I am running the Pitman-Yor Adaptor-Grammar Sampler software I downloaded from here ./py-cfg/py-cfg-mp -r 0 -d 10 -x 10 -D -E -e 1 -f 1 -g 10 -h 0.1 -w 1 -T 1 -m 0 -n 500 -G x.tgt y.tgt < z.tgt I tried adding parallel -j "$(nproc)" before the command as specified in this answer but it is generating the following error: Error in ./py-cfg/py-cfg-mp, argc = 29, optind = 27

When I "flood" my CPU with 8 high priority (nice=-20) OS threads (the number of cores I have), operation becomes "halty" for obvious reasons, but is still usable. Note that when I say "high-priority thread" I mean a thread that was spawned by the same high-priority process. However doing something like 64 threads will completely make my PC unusable. What is the relationship between max priority threads and their distribution between cores? Can we figure out roughly how many threads I need to spawn to completely flood the CPU for a given nice value?

The question refers to the output of a multi-threaded application, where each thread merely prints its ID (user assigned number) on the standard output. Here all threads have equal priority and compete for CPU quota in order to print on the standard output. However, running the same application a sufficiently large number of times will result in different order of IDs being printed on the screen. This order is due to the OS scheduler which is a piece of software and therefore deterministic. Nonetheless, its behavior seems non-deterministic, which brings me back to the initial question.

I made a search but couldn't find anything similar. I have huge codebases ( AOSP or Yocto ) stored in ext4 formatted NVME drives ( such as SAMSUNG 980Pro ). These codebases usually consist of hundreds of thousands of files in thousands of subdirectories. When I search for a file with find , I usually do: find . -iname "filename.txt" I don't want to operate on the files, I just want to find the file. I feel this is not multithreaded ( please correct me if I am wrong ). Please note that, I know that searching for files are generally IO bound, but my drives are fast, and I believe a tool that can iterate through subdirectories in multiple threads can dramatically speed up finding results. Please correct me if I am wrong. If I am right, and if there is such tool, please let me know.

Modern HDDs all are "Advanced Format " ones, e.g. by default they report a logical/physical sector size of 512/4096. By default, most Linux formatting tools use a block size of 4096 bytes (at least that's the default on Debian/EXT4). Until today, I thought that this was kind of optimized : Linux/EXT4 sends chunks of 4K data to the HDD, which can handle them optimally, even though its logical sector size is 512K. But today I read this quite recent (2021) post . The guy did some HDD benchmarks, in order to check if switching his HDD's logical sector size from 512e to 4Kn would provide better performances. His conclusion : > Remember: My theory going in was that the filesystem uses 4k blocks, and everything is properly aligned, so there shouldn’t be a meaningful difference.\ \ Does that hold up? Well, no. Not at all. (...) Using 4kb blocks… there’s an awfully big difference here. This is single threaded benchmarking, but there is consistently a huge lead going to the 4k sector drive here on 4kb block transfers. (...)\ \ **Conclusions: Use 4k Sectors!**\ As far as I’m concerned, the conclusions here are pretty clear. If you’ve got a modern operating system that can handle 4k sectors, and your drives support operating either as 512 byte or 4k sectors, convert your drives to 4k native sectors before doing anything else. Then go on your way and let the OS deal with it. Basically, his conclusion was that there was quite a performance improvement in switching the HDD's logical sector size to 4Kn, vs the out-of-box 512e : Now, an important thing to note : that particular benchmark was single threaded. He also did a 4-threaded benchmark, which didn't show any significant differences between 512e and 4Kn. Thus my questions : - His conclusion holds up only if you have single threaded processes that read/write on the drive. Does Linux have such single threaded processes ? - And thus, would you recommend to set a HDD's logical sector size to 4Kn ?

I wrote the following kernel module:

#include 
#include 
#include 
#include 
#include 
#include 
#include  
     
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Madhuparna Bhowmik ");
MODULE_DESCRIPTION("Example for using threads");
    
static struct task_struct *t1;
static struct task_struct *t2;

static int t1_f(void *unused)
{
	printk(KERN_INFO "Current value in t1 is %d",7);
	do_exit(0);
	return 0;
}

static int t2_f(void *unused)
{
	printk(KERN_INFO "Current value in t2 is %d",8);
	do_exit(0);
	return 0;
}

static int __init start_init(void)
{
   	printk(KERN_INFO "Thread Creating...\n");
   	t1 = kthread_run(t1_f,NULL,"mythread1");
   	t2 = kthread_run(t2_f,NULL,"mythread2");
   	
   	return 0;
}

static void __exit end_exit(void)
{
   	printk(KERN_INFO "Cleaning Up...\n");
}

	
module_init(start_init)
module_exit(end_exit)

When I load this module using :

sudo insmod test1.ko

Checking kern.log file gives the following result:

Oct  8 00:24:13 madhuparna-VirtualBox kernel: [ 5752.075918] Thread creating ...

Oct  8 00:24:13 madhuparna-VirtualBox kernel: [ 5752.076009] Current value in t1 is 7

There is no log for thread2 yet. After I execute :

sudo rmmod test1.ko

The log is:

Oct  8 00:24:13 madhuparna-VirtualBox kernel: [ 5752.075918] Thread creating ...
Oct  8 00:24:13 madhuparna-VirtualBox kernel: [ 5752.076009] Current value in t1 is 7
Oct  8 00:24:54 madhuparna-VirtualBox kernel: [ 5752.077780] Current value in t2 is 8
Oct  8 00:24:54 madhuparna-VirtualBox kernel: [ 5793.099359] Cleaning up ...

So, can someone please explain that why does thread 2 does not start running until I use rmmod and unload the kernel? Why is only thread1 executing?

I am looking for a linux backup tool to upgrade my backup strategy. Today I do full backups everyday on Linux server (Ubuntu 20.04). I prefer to backup data directly inside our VMs and send to another site (not at VMware vSphere level) to not be dependent of the virtualisation system (in order to be able to restart from any kind of servers with a Linux and LAMP&Co tools). We have more than a million of files (csv, json, xlsx) and almost 1To of data. Compressed, it is reduced to 100Go in 4 hours thanks to lbzip2 and its multithreaded structure. It is starting to be too much space and time per night. I want to make full backups during weekends and only incremental ones on weeknights. I tried Dar and Duplicity but there are not multithreaded and Dar only uses Gz compression (Duplicity can do bz2 from GPG but only on a single thread). My dreamed solution : - multithreaded to save time - encryption - good compression (lbzip2 or equivalent) to reduce disk usage - opensource software Does anyone know a tool or a solution I could use to optimise our backups?

I write some threads of my app to a Linux cpu cgroup, created by me, explicitly (let us talk e.g. about v1 for now, but if you know a way for v2, that would be also great). However, I see that descendant threads (created with pthread_create() called by their parent thread, which is written to the cgroup explicitly or again after its parent thread) also appear in the cgroup without me writing them directly. This affects the app not in a good way. Some threads get to the cgroup unexpectedly. You need be always aware of this caveat and be careful e.g. with thread pools etc. Could you, please, tell if there is a way to solve this problem? E.g., maybe is there a way to forbid descendant threads from getting into their parent's cgroup automatically? Any help appreciated.

You can see the output from lscpu command - jack@042:~$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 56 On-line CPU(s) list: 0-55 Thread(s) per core: 2 Core(s) per socket: 14 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 79 Model name: Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz Stepping: 1 CPU MHz: 2600.000 CPU max MHz: 2600.0000 CPU min MHz: 1200.0000 BogoMIPS: 5201.37 Virtualization: VT-x Hypervisor vendor: vertical Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 35840K NUMA node0 CPU(s): 0-13,28-41 NUMA node1 CPU(s): 14-27,42-55 I can see that there are 2 sockets (which is like a processor ??) and inside each of the socket we have 14 cores. So, in total 2x14=28 physical cores. Normally, a CPU can contain multiple cores, so number of CPUs can never be smaller than number of Cores. But, as shown in the output CPUs(s): 56 and this is what is confusing me. I can see that Thread(s) per core: 2, so these 28 cores can behave like 2x28=56 logical cores. **Question 1:** What does this CPUs(s): 56 denote? Does CPU(s) denote number of Virtual/Logical core, as it cannot be a Physical core core atleast? **Question 2:** What does this NUMA node mean? Does it represent the socket?

I wrote a simple program with a thread which runs on a CPU core. It spins kind of aggressively, and it takes 100% of the CPU core. I can see that with top + 1. After N minutes, I would like to be able to know: How many times has the kernel preempted (interrupted) my running thread?