Sample Header Ad - 728x90

Changing the path opened by openat of a program using LD_PRELOAD

0 votes
1 answer
448 views
I want to change a path that a program actually opens on filesystem for some paths. The reason is that I want to run a program in parallel, but that program uses /tmp/somedir/ as its temporary directory and parallel instances run into conflicts. I found this great answer to do just that: Is it possible to fake a specific path for a process? . Sadly, while this works for cat as advertised, it does not work for my program. I thought the cause is that the program uses C++ API. To reproduce, I first made a very simple program that writes something in a file:
#include 
#include 
#include 

int main() {
    std::ofstream myfile;
    myfile.open("test.log");
    std::string_view text{"hello world\n"};
    myfile.write(text.data(), text.size());
    return 0;
}
I then used strace and saw this at the end:
brk(NULL)                               = 0x558b5d5e3000
brk(0x558b5d604000)                     = 0x558b5d604000
futex(0x7f94e2e7e77c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
openat(AT_FDCWD, "test.log", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
write(3, "hello world\n", 12)           = 12
close(3)                                = 0
exit_group(0)                           = ?
So it looks like C api is called, but the function used is openat. I also saw this for the C so library, this will be relevant later:
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
So I implemented openat in addition to open, and this is the full program. For test purposes, instead of changing a path I just log it into a file.
/*
 * capture calls to a routine and replace with your code
 * g++ -Wall -O2 -fpic -shared -ldl -lstdc++ -o fake_open_file.so fake_open_file.cpp
 * LD_PRELOAD=/home/myname/fake_open_file.so cat
 */
#define _FCNTL_H 1 /* hack for open() prototype */
#undef _GNU_SOURCE
#define _GNU_SOURCE /* needed to get RTLD_NEXT defined in dlfcn.h */
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 


// for the test, I just log anything that was open into a new log file
struct open_reporter
{
    open_reporter() = default;
    void report_filename(std::string_view filename)
    {
        std::lock_guard l{file_report_mutex};
        if(!is_open) {
            myfile.open("/home/myname/fileopen.log");
        }
        std::string tmp = std::string{filename} + "\n";
        myfile.write(tmp.data(), tmp.size());
    }
    std::ofstream myfile;
    std::mutex file_report_mutex;
    bool is_open = false;
};

static open_reporter reporter_;

extern "C" {
    int open(const char *pathname, int flags, mode_t mode)
    {
        static int (*real_open)(const char *pathname, int flags, mode_t mode) = nullptr;

        if (!real_open) {
            real_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
            char *error = dlerror();
            if (error != nullptr) {
                reporter_.report_filename("ERROR OCCURED!");
                reporter_.report_filename(error);
                exit(1);
            }
        }

        reporter_.report_filename(pathname);
        return real_open(pathname, flags, mode);
    }

    int openat(int dirfd, const char *pathname, int flags, mode_t mode)
    {
        static int (*real_openat)(int dirfd, const char *pathname, int flags, mode_t mode) = nullptr;

        if (!real_openat) {
            real_openat = reinterpret_cast(dlsym(RTLD_NEXT, "openat"));
            char *error = dlerror();
            if (error != nullptr) {
                reporter_.report_filename("ERROR OCCURED!");
                reporter_.report_filename(error);
                exit(1);
            }
        }

        reporter_.report_filename(pathname);
        return real_openat(dirfd, pathname, flags, mode);
    }
}
This works with cat still, but not with my test program. Even if I change open and openat to return 0, while this breaks cat it does not have any effect on my test program. I also checked if the symbols are in my binary:
$ nm -gD fake_open_file.so | grep open
0000000000001470 W _ZN13open_reporterD1Ev
0000000000001470 W _ZN13open_reporterD2Ev
0000000000001450 T open
0000000000001460 T openat
I can see both the functions present. Looking into C library, I see a difference, but I don't know what it means. I redacted things that are not open or openat:
$ nm -gD /lib/x86_64-linux-gnu/libc.so.6 |grep open
0000000000114820 W openat@@GLIBC_2.4
0000000000114820 W openat64@@GLIBC_2.4

0000000000114690 W open@@GLIBC_2.2.5
0000000000114690 W open64@@GLIBC_2.2.5

0000000000114690 W __open@@GLIBC_2.2.5
0000000000114690 W __open64@@GLIBC_2.2.5
00000000001147c0 T __open64_2@@GLIBC_2.7
0000000000119b80 T __open64_nocancel@@GLIBC_PRIVATE
0000000000114660 T __open_2@@GLIBC_2.7
0000000000040800 T __open_catalog@@GLIBC_PRIVATE
0000000000119b80 T __open_nocancel@@GLIBC_PRIVATE
0000000000114950 T __openat64_2@@GLIBC_2.7
00000000001147f0 T __openat_2@@GLIBC_2.7
Apart of the @@GLIBC stuff, these are the same. I never have done this before, so this is as far as my debugging ability goes. I am asking here and not SO because here is where I got the original answer and also this looks more like linux knowledge than a programming problem, the program itself is very simple.
Asked by Tomáลก Zato (1806 rep)
Jan 19, 2024, 02:33 PM
Last activity: Jan 19, 2024, 06:22 PM