Mysql container image behaving diferently between docker and podman in github actions

I have a github workflow that spins up a MySQL container. I'm using podman right now, because that matches our development workflow, but I'm seeing some weird differences in behavior between "podman in github actions" and "podman running on the local machine". I'm also seeing differences between "podman in github actions" and "docker in github actions". I'm installing podman in my github workflow from https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_22.04 , which gets me:

Client:       Podman Engine
Version:      4.6.2
API Version:  4.6.2
Go Version:   go1.18.1
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64

Locally, I have a newer version of podman:

Client:       Podman Engine
Version:      5.1.1
API Version:  5.1.1
Go Version:   go1.22.3
Built:        Mon Jun  3 20:00:00 2024
OS/Arch:      linux/amd64

I'm starting mysql like this:

podman run --rm -d --name mysql -e MYSQL_ROOT_PASSWORD=secret docker.io/mysql:8

This successfully starts mysql, but the unix socket ends up in /var/lib/mysql/mysql.sock rather than /run/mysqld/mysql.sock. Unfortunately, the latter location is where everything expects to find it:

bash-5.1# mysql -u root -psecret
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
bash-5.1# ls -l /run/mysqld/
total 4
srwxrwxrwx 1 mysql mysql 0 Jun 28 01:04 mysqlx.sock
-rw------- 1 mysql mysql 2 Jun 28 01:04 mysqlx.sock.lock

In contrast, when I run the image locally, I see:

bash-5.1# ls -l /run/mysqld/
total 12
-rw-r-----. 1 mysql mysql 2 Jun 28 00:45 mysqld.pid
srwxrwxrwx. 1 mysql mysql 0 Jun 28 00:45 mysqld.sock
-rw-------. 1 mysql mysql 2 Jun 28 00:45 mysqld.sock.lock
srwxrwxrwx. 1 mysql mysql 0 Jun 28 00:45 mysqlx.sock
-rw-------. 1 mysql mysql 2 Jun 28 00:45 mysqlx.sock.lock

It's the same image in both locations. On github:

runner@fv-az730-813:~/work/workflow-test/workflow-test$ podman image ls
REPOSITORY               TAG         IMAGE ID      CREATED     SIZE
docker.io/library/mysql  8           05247af91864  9 days ago  594 MB

And locally:

$ podman image ls
REPOSITORY                                   TAG              IMAGE ID      CREATED        SIZE
docker.io/library/mysql                      8                05247af91864  9 days ago     594 MB

If I run the same image on github in docker, it works as expected:

runner@fv-az730-813:~/work/workflow-test/workflow-test$ docker exec -it mysql bash
bash-5.1# mysql -u root -psecret
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.4.0 MySQL Community Server - GPL

Copyright (c) 2000, 2024, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

The only difference I see is that when starting the image successfully, the following line is logged to the container output:

'/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'

That line is missing when starting the container with podman in a github workflow. That comes from this section of the entrypoint script:

mysql_socket_fix() {
        local defaultSocket
        defaultSocket="$(mysql_get_config 'socket' mysqld --no-defaults)"
        if [ "$defaultSocket" != "$SOCKET" ]; then
                ln -sfTv "$SOCKET" "$defaultSocket" || :
        fi
}

And the difference boils down to the fact that running mysql --help --verbose | grep '^socket' results in this when running the container with podman locally or with docker:

bash-5.1# mysqld --verbose --help |& grep '^'socket
socket                                                       /var/run/mysqld/mysqld.sock

And this when running podman in the github workflow:

bash-5.1# mysqld --verbose --help |& grep '^'socket
socket                                                       /var/lib/mysql/mysql.sock

--- I am trying to figure out why I am seeing this difference in behavior. It *looks* like a permission problem, but I'm not sure what's causing it. In the github workflow, the runner user is able to create files and sockets in /run/mysqld without a problem when I try it manually (e.g., using nc -U /run/mysqld/foo.sock -l). --- **Update**: Here's the permission problem; running strace on the mysqld process shows:

openat(AT_FDCWD, "/etc/my.cnf", O_RDONLY) = -1 EACCES (Permission denied)

...but that doesn't make any sense:

bash-5.1# ls -l /etc/my.cnf
-rw-r--r-- 1 root root 999 Jun 21 20:50 /etc/my.cnf

And even:

bash-5.1# sudo -u mysql head -2 /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/8.4/en/server-configuration-defaults.html