I bootstrapped an arch x86_64 rootfs on arch linux arm vm with qemu-x86_64 binfmt registered and copied static qemu-x86_64 to the same path in the chroot:

[root@archlinux ~]# cat /proc/sys/fs/binfmt_misc/qemu-x86_64
enabled
interpreter /usr/bin/qemu-x86_64
flags: PF
offset 0
magic 7f454c4602010100000000000000000002003e00

[root@archlinux ~]# file rfs/usr/bin/qemu-x86_64 
rfs/usr/bin/qemu-x86_64: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), static-pie linked, BuildID[sha1]=1640ccd48aea1547a7b9d5a265cd35cfd9367cbd, for GNU/Linux 3.7.0, with debug_info, not stripped

It threw errors like:

:: Running post-transaction hooks...
( 1/11) Creating system user accounts...
call to execv failed (No such file or directory)
error: command failed to execute correctly

I tried to use

rfs qemu-x86_64 /usr/bin/bash

, the shell seems launch correctly, but any command I type did not work:

[root@archlinux ~]# chroot rfs qemu-x86_64 /usr/bin/bash
ls
/usr/bin/bash: line 1: /usr/bin/ls: cannot execute: required file not found
env
/usr/bin/bash: line 2: /usr/bin/env: cannot execute: required file not found
exit

But running command directly works:

[root@archlinux ~]# chroot rfs qemu-x86_64 /usr/bin/ls
bin
boot
dev
etc
home
lib
lib64
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var

Why and how to fix it?

I've made an ArchLinux RootFS for zynqmp like here . But I haven't installed kernel inside RootFS and deleted hte one which is default, as I want to change kernels frequently, so I've placed it in BOOT partition of my SD. Now I want to chroot in this environment from my x86-64 linux PC, but as there is no kernel inside, I cannot do some things as environment errors me about different security policies that are not present in kernel. Is there any way I can point qemu on my custom kernel with chroot?

How can I make registered extensions for binfmt_misc persist across reboots? Consider the following command, which performs a binfmt_misc registration: echo ':golang:E::go::/tmp/test:OC' | sudo tee /proc/sys/fs/binfmt_misc/register It needs to be executed as root, since only root is allowed to write to /proc/sys/fs/binfmt_misc/register. Thus, I cannot put such registrations into e.g. ~/.bashrc as an unprivileged user.

I plan on creating a script which would help me with detecting long-time unused manually installed packages. For this, it would be neat though if I had a way of finding out the executed time for each executable under, e.g. /usr/bin/, /usr/local/bin/, ~/bin/… atime (ls -ltu /usr/bin/) would not be an option for various reasons (access time does not mean it got executed, also noatime).

In my Linux Mint installation, I can directly execute a PE executable at the command line, and if it's a .NET/CLI executable, then it will automatically execute it with Mono, and otherwise, it will automatically execute it with Wine. This works because binfmt is configured to run /usr/lib/binfmt-support/run-detectors for both CLI and Wine executables. This runs a series of detectors configured in /var/lib/binfmts to determine what type of PE executable it is and execute the appropriate command to run it. In Manjaro, I have binfmt, but I don't have run-detectors. So I guess the first match in /proc/sys/fs/binfmt_misc wins, and I guess since I installed Wine before I installed Mono, that's the one that wins. That my supposition, anyway. The symptom is that whenever I execute an .exe file from the command line, it runs Wine, even if it's a .NET executable. I can't find a package that seems to supply this run-detectors utility. Is there something like it in the standard or community repositories, or is there some other common way to set this up in the Arch world?

I have been testing binfmt_misc feature of Linux on Debian 10, and have found that setting the flags to "OC", to use the credentials of the binary instead of interpreter, causes execution to fail silently. In the POC below, /tmp/test.sh is the interpreter, while qux.go is the binary. Why is /tmp/test.sh executed successfully without flags, when it fails silently with flags "OC"? **POC**: $ touch qux.go $ chmod +x qux.go $ cat /tmp/test.sh > #!/bin/sh > echo Golang > EOF $ chmod +x /tmp/test.sh $ echo ':golang:E::go::/tmp/test.sh:' | sudo tee /proc/sys/fs/binfmt_misc/register :golang:E::go::/tmp/test.sh: $ ./qux.go Golang $ echo -1 | sudo tee /proc/sys/fs/binfmt_misc/golang -1 $ echo ':golang:E::go::/tmp/test.sh:OC' | sudo tee /proc/sys/fs/binfmt_misc/register :golang:E::go::/tmp/test.sh:OC $ ./qux.go # no output Also: mount | grep binfmt_misc systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=658) binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime) **Bonus**: Some resources claim that binfmt_misc could be used for container-to-host escapes. However, as I see it, the interpreter path is evaluated within the chroot'd filesystem of the container, and execution of the interpreter is happening within the container, i.e. ls -la / shows the container root (not the host root). **Resource**: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html

I have Ubuntu server that is running Linux 4.15.0-1082-azure. I updated the usr/include/linux/binfmts.h file in order to increase the size of argument list. I changed the MAX_ARG_PAGES. I need to recompile, install and reboot into the new kernel so the changes takes effect , but I do not know how to do that. I tried these following command, make make modules_install make install reboot but I got error messages when run make commands: scripts/kconfig/conf --silentoldconfig Kconfig *** *** Configuration file ".config" not found! *** *** Please run some configurator (e.g. "make oldconfig" or *** "make menuconfig" or "make xconfig"). *** scripts/kconfig/Makefile:37: recipe for target 'silentoldconfig' failed make: *** [silentoldconfig] Error 1 Makefile:573: recipe for target 'silentoldconfig' failed make: *** [silentoldconfig] Error 2 make: *** No rule to make target 'arch/x86/entry/syscalls/syscall_32.tbl', needed by 'arch/x86/include/generated/asm/syscalls_32.h'. Stop. arch/x86/Makefile:253: recipe for target 'archheaders' failed make: *** [archheaders] Error 2 I tried to use make menuconfig , but I cannot understand it I was worried to breakdown something on the server. How could I make the changes in usr/include/linux/binfmts.h file make effect? How could I know if the new setting is taking the effect or not?

I'm trying to build a portable cross-arch Docker container using QEMU. However, depending on the host system and the programs installed on the host the container behaves differently. Running the container on Windows, the emulation works fine. It also works fine on Ubuntu 16.04 **when** qemu-user-static is installed. But it will stop emulating if it isn't. Emulation works as long as binfmt-support is enabled in the Host kernel (for Docker on Linux) or in the VM's kernel (for Docker on Windows) AND the necessary binaries are avaiblable in the containers filesystem. My goal is to make the container portable and runnable without the need of registering qemu as an interpreter in the kernel since any modifications to the Host-OS itself or its kernel are forbidden by our customer. But currently, that is the result of running the container:

$ sudo docker run -ti qemu:xenial_arm64
qemu-aarch64-static: /usr/bin/groups: cannot execute binary file: Exec format error
/usr/bin/lesspipe: line 28: /usr/bin/basename: cannot execute binary file: Exec format error
/usr/bin/lesspipe: line 282: /usr/bin/dirname: cannot execute binary file: Exec format error
/usr/bin/lesspipe: line 295: : =: unary operator expected
qemu-aarch64-static: /usr/bin/dircolors: cannot execute binary file: Exec format error
root@41f2795e2569:/# uname -m
qemu-aarch64-static: /bin/uname: cannot execute binary file: Exec format error

However if I install qemu-user-static (qemu will also be enabled in binfmt_misc by installing it):

$ sudo apt-get -yqq install qemu-user-static
Selecting previously unselected package qemu-user-static.
(Reading database ... 245713 files and directories currently installed.)
Preparing to unpack .../qemu-user-static_1%3a2.5+dfsg-5ubuntu10.42_amd64.deb ...
Unpacking qemu-user-static (1:2.5+dfsg-5ubuntu10.42) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up qemu-user-static (1:2.5+dfsg-5ubuntu10.42) ...

$ sudo docker run -ti qemu:xenial_arm64
root@7903b45a3c1e:/# uname -m
aarch64

My explanation for that is: - qemu-aarch64-static starts emulating /bin/bash - /bin/bash uses execve(..) to call some other executables - Linux Host: - if binfmt_misc is enabled either by the host or by the container (on Linux, the host and the Docker guest share a kernel), qemu will be dynamically assigned as an interpreter for the arm64 executables - else, executation will fail and "Exec format error" will be thrown - Windows Host: - since Windows and Linux have different kernels, Linux guests cannot be executed by passing syscalls onto the host kernel - so on Windows, containers run inside of a VM with a separate kernel; that kernel is then shared with the containers - if qemu-user-static with binfmt-support was once installed and enabled in any container, it will still serve its purpose and dynamically assign qemu as interpreter for executables of another architecture **Execution on Linux and on Windows yields equivalent results.** A possible fix would be to launch the container using the --privileged flag. Then I would be able to mount binfmt_misc and thereby register qemu-user-static as an interpreter inside the container. But our customer forbids to use that flag. I've already tested the approach described [here as well as qemu-user's -0 flag. Neither one does work. **Dockerfile:**

FROM scratch
ADD xenial-arm64-rootfs /
ADD qemu-aarch64-static /usr/bin/qemu-aarch64-static
RUN chmod +x /usr/bin/qemu-aarch64-static
ENTRYPOINT ["/usr/bin/qemu-aarch64-static", "-0", "/usr/bin/qemu-aarch64-static"]
CMD ["/bin/bash"]

qemu-aarch64-static is taken from /usr/bin/qemu-aarch64-static (installed with qemu-user-static) xenial-arm64-rootfs is created using:

qemu-debootstrap --arch=arm64 --components=main,universe,multiverse,restricted --variant=buildd --foreign xenial xenial-arm64-rootfs http://ports.ubuntu.com/ubuntu-ports/ 

What can I do in order to keep the emulation going?

I have a Fujitsu LifeBook with Lubuntu 18.04.2 which is booting very slow. Here's the output from systemd-analyze blame | head -n20 28.869s systemd-journal-flush.service 24.957s dev-sda1.device 14.620s apparmor.service 14.558s systemd-udevd.service 14.161s binfmt-support.service 8.786s NetworkManager.service 8.366s udisks2.service 8.142s NetworkManager-wait-online.service 8.117s networkd-dispatcher.service 7.187s snapd.service 5.402s grub-common.service 4.975s systemd-tmpfiles-setup-dev.service 4.383s ModemManager.service 4.028s accounts-daemon.service 3.988s gpu-manager.service 3.693s console-setup.service 3.343s keyboard-setup.service 3.203s apt-daily.service 2.780s apport.service 2.410s wpa_supplicant.service I don't know what most of these are, but when I compare other users, I noticed most users don't seem to have binfmt-support.service at the top of their lists. 1. What might be causing binfmt-support to be slow at startup? 2. What can I do to make binfmt-support not take so long?

Ubuntu 18.04.2 LTS on AMD64 with Steam client now at version Feb 18 2019, package 1550534751. Yesterday, I could play Steam Play games (Windows games with the Proton fork of Wine) just fine on my machine. I rebooted, and Steam updated itself, and now none of those games run. When I try to start a game, I get errors like: run-detectors: unable to find an interpreter for /home/jwatte/.local/share/Steam/legacycompat/iscriptevaluator.exe run-detectors: unable to find an interpreter for /opt/steam/steamapps/common/ForgedBattalion/ClientLauncherG.exe (I also get a few assert failures in AssertMsgImplementation(), which the Internet seems to think are red herrings. Asserts in shipping code are of course bad, but as I don't have the source, what can I do but trust this?) Additional searching says that this is because the binfmt database doesn't contain Windows PE32 files, and true enough, update-binfmts --display does not show any binfmt that uses wine/proton. Before I go ahead and try to patch this myself with update-binfmts --install and trial-and-error, is there a ready-made file to import for proton? Or is it supposed to work some other way, and something's broken on my system? I'd rather stay as close to vendor-supplied installations as possible, and not add too much customization myself...

On Lubuntu 18.04, Is /usr/lib/binfmt-support/run-detectors supposed to choose mono or wine to run a windows or .net executable? $ cat /proc/sys/fs/binfmt_misc/cli enabled interpreter /usr/lib/binfmt-support/run-detectors flags: offset 0 magic 4d5a $ file /usr/lib/binfmt-support/run-detectors /usr/lib/binfmt-support/run-detectors: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=06509e829163c8e50781b5335883bf9a8ed47321, stripped Why can't I run a Windows executable? How can I make run-detectors work? $ ./PDFXCview.exe run-detectors: unable to find an interpreter for ./PDFXCview.exe $ file ./PDFXCview.exe ./PDFXCview.exe: PE32 executable (GUI) Intel 80386, for MS Windows $ type wine wine is /usr/bin/wine Thanks.

I following Michael's reply to see what executable formats my Ubuntu can recognize to execute $ ls -l /proc/sys/fs/binfmt_misc/ total 0 -rw-r--r-- 1 root root 0 Apr 19 16:11 cli -rw-r--r-- 1 root root 0 Apr 19 16:11 jar -rw-r--r-- 1 root root 0 Apr 19 16:11 python2.7 -rw-r--r-- 1 root root 0 Apr 19 16:11 python3.5 --w------- 1 root root 0 Apr 19 16:11 register -rw-r--r-- 1 root root 0 Apr 19 16:11 status I have never intentionally changed anything there, and the files were created by default or when I installed some other programs. $ cat /proc/sys/fs/binfmt_misc/cli enabled interpreter /usr/lib/binfmt-support/run-detectors flags: offset 0 magic 4d5a What kind of executable format is this? I googled "magic 4d5a" and found https://en.wikipedia.org/wiki/DOS_MZ_executable , but I am not sure how the file was created there since it is not a native executable format to Linux. Did installation of wine add it? $ cat /proc/sys/fs/binfmt_misc/jar enabled interpreter /usr/lib/jvm/java-9-oracle/lib/jexec flags: offset 0 magic 504b0304 Is the above for JVM bytecode format? $ cat /proc/sys/fs/binfmt_misc/python3.5 enabled interpreter /usr/bin/python3.5 flags: offset 0 magic 160d0d0a Is the above for Python bytecode or Python? $ cat /proc/sys/fs/binfmt_misc/status enabled $ cat /proc/sys/fs/binfmt_misc/register cat: /proc/sys/fs/binfmt_misc/register: Permission denied What is /proc/sys/fs/binfmt_misc/register used for? Does it also allow some executable format? Does ELF format need a file under /proc/sys/fs/binfmt_misc/? Thanks.

I have downloaded some firmware and I want to run some commands in it. I have extracted the firmware, installed qemu and configured binfmts. The problem is that binfmt does not work in the chroot. Mipsel files are automatically executed by qemu-mipsel-static: firmware$ bin/ls /lib/ld-uClibc.so.0: No such file or directory The command qemu-mipsel-static is available within the chroot: firmware$ sudo chroot . /usr/bin/qemu-mipsel-static /bin/ls webroot tmp sbin media home dev When combining the two, it no longer works: firmware$ sudo chroot . /bin/ls chroot: failed to run command ‘/bin/ls’: No such file or directory It says "No such file or directory". Is there any way to get more information about this, for example which file can not be found? Is this indeed a problem with binfmt? My binfmt config looks like this: $ cat /proc/sys/fs/binfmt_misc/qemu-mipsel enabled interpreter /usr/lib/binfmt-support/run-detectors flags: offset 0 magic 7f454c4601010100000000000000000002000800 mask ffffffffffffff00fffffffffffffffffeffffff Is it a problem that /usr/lib/binfmt-support/run-detectors is not available in the chroot?

I'd like to run armhf executables inside an amd64 LXC container. How to do that? update-binfmts --display shows no qemu interpreters.

Systemd has several failed services, and I don't know how to work it out: # systemctl --failed UNIT LOAD ACTIVE SUB JOB DESCRIPTION proc-sys-fs-binfmt_misc.automount loaded failed failed Arbitrary Executable File Formats File System Automount Point systemd-binfmt.service loaded failed failed Set Up Additional Binary Formats More details: # systemctl status systemd-binfmt.service systemd-binfmt.service - Set Up Additional Binary Formats Loaded: loaded (/usr/lib/systemd/system/systemd-binfmt.service; static) Active: failed (Result: exit-code) since Tue, 2012-11-27 07:42:11 CST; 36min ago Docs: man:systemd-binfmt.service(8) man:binfmt.d(5) https://www.kernel.org/doc/Documentation/binfmt_misc.txt Process: 213 ExecStart=/usr/lib/systemd/systemd-binfmt (code=exited, status=1/FAILURE) CGroup: name=systemd:/system/systemd-binfmt.service # systemctl status proc-sys-fs-binfmt_misc.automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point Loaded: loaded (/usr/lib/systemd/system/proc-sys-fs-binfmt_misc.automount; static) Active: failed (Result: resources) Where: /proc/sys/fs/binfmt_misc Docs: https://www.kernel.org/doc/Documentation/binfmt_misc.txt Any suggestions?

I want to be able to run clr (>NET/mono) exes on my redhat machines, so I add the following to /etc/rc.local: echo ':CLR:M::MZ::/opt/mono-head/bin/mono:' > /proc/sys/fs/binfmt_misc/register This works fine, but is there a more appropriate config file specifically intended for configuring binfmt as opposed to the kitchen sink rc.local?

I run docky which is programmed with mono so I think the binfmt is running correctly, but proc-sys-fs-binfmt_misc.automount complains about it, since I don't have /proc/sys/fs/binfmt_misc/CLR I already have CONFIG_BINFMT_MISC=y in my kernel config.