Sample Header Ad - 728x90

What can cause "listing a directory" on Android to be 50-250 times slower than expected?

2 votes
1 answer
82 views
I ended up with my Android device (Google Pixel 7a, on AOSP 15) having accumulated about **~14k** videos/photos files in /sdcard/DCIM/Camera. As a consequence I have noticed some very *unexpected* delay when listing this directory in this particular folder, *but also* in general. I do run the following tests
[user@archlinux-box ~] $ ## having the Pixle 7a connected via USB-3 port USB-debugging
[user@archlinux-box ~] $ adb shell
lynx:/ $ ## time the listing of /sdcard/DCIM/Camera
lynx:/ $ (cd /sdcard/DCIM/Camera; time ls | wc)
  13980   13980  398700
    0m04.55s real     0m00.07s user     0m00.31s system
lynx:/ $ ## exclude that it is a fault ot ls binary I check via find too
lynx:/ $ (cd /sdcard/DCIM/Camera; time find | wc)
  13981   13981  426662
    0m05.25s real     0m00.07s user     0m00.41s system
Being suspicious that listing ~14k files may very well take some time, yet not ~5seconds (and less so upon 2nd time directory listing reads, given I would assume some caching to occur) I tested if this is related to the specific folder /sdcard/DCIM/Camera doing this
lynx:/ $ ## create and enter a test directory to hold 14k files
lynx:/ $ mkdir /sdcard/14k-files-dir
lynx:/ $ cd  /sdcard/14k-files-dir
lynx:/sdcard/14k-files-dir $ ## create 14k files somewhat efficiently
lynx:/sdcard/14k-files-dir $ ## have filenams be comparable length as photo files
lynx:/sdcard/14k-files-dir $ PREFIX='filename-similar-to-photo-files-file-no-'
lynx:/sdcard/14k-files-dir $ seq 1 14000 | sed 's/^/'"$PREFIX"'/' | xargs sh -c 'touch "$@"' _
lynx:/sdcard/14k-files-dir $ ## rerun tests
130|lynx:/sdcard/14k-files-dir $ time ls | wc
  14000   14000  618894
    0m01.92s real     0m00.10s user     0m00.28s system
lynx:/sdcard/14k-files-dir $ time find | wc
  14001   14001  646896
    0m00.72s real     0m00.03s user     0m00.13s system
lynx:/sdcard/14k-files-dir $ time ls | wc
  14000   14000  618894
    0m00.80s real     0m00.10s user     0m00.13s system
as can be seen this time, the directory listing is far faster (particularly once it appears to be in cache). Lastly I wanted to test - given Android being based on linux kernel - how well my 10+ year laptop would compare.
[user@archlinux-box ~] $ ## running test on a 10 year old laptop
[user@archlinux-box ~] $ mkdir 14k-files-dir
[user@archlinux-box ~] $ cd 14k-files-dir
[user@archlinux-box 14k-files-dir ] $ ## create 14k files somewhat efficiently
[user@archlinux-box 14k-files-dir ] $ ## have filenams be comparable length as photo files
[user@archlinux-box 14k-files-dir ] $ PREFIX='filename-similar-to-photo-files-file-no-'
[user@archlinux-box 14k-files-dir ] $ seq 1 14000 | sed 's/^/'"$PREFIX"'/' | xargs sh -c 'touch "$@"' _
[user@archlinux-box 14k-files-dir ] $ ## run tests on laptop
[user@archlinux-box 14k-files-dir]$ time ls | wc
  14000   14000  632894

real    0m0.025s
user    0m0.015s
sys     0m0.014s
[user@archlinux-box 14k-files-dir]$ time find | wc
  14001   14001  660896

real    0m0.030s
user    0m0.019s
sys     0m0.020s
[user@archlinux-box 14k-files-dir]$ time ls | wc
  14000   14000  632894

real    0m0.021s
user    0m0.009s
sys     0m0.015s
As can be seen the speed listing directories differ vastly. I am puzzled to determine the reasons. 1. It is mostly unclear why the direct "on Android device" comparison of directories listing times is so huge (1-0.7seconds vs 4-5seconds for /sdcard/DCIM/Camera) 2. It is puzzling as I would expect both folders residing on the same block-device and filesystem being both below /sdcard ? 3. In a bigger picture however I cannot wrap my haed around (and seek with this question the causes that a 10 year old linux laptop can list 14k files at 0.02seconds while a rather recent Google Pixel 7a needs 0.7 to 5+ seconds, so plainly 250 times as long? #Update# I have found this info on redit which mentions > As mentioned below, it seems on Android 11 ALL file access to external storage goes through SAF, even when targeting API 29 and using standard file access functions. This is a change from Android 10. This explains the difference in speed. I hence wonder (with regarding to the causes I seek in this question here) if this this still the reason? Also with this new information I cannot still see why the two folders still have 5-10 times different directory lookup speeds.
Asked by humanityANDpeace (807 rep)
Nov 28, 2024, 07:39 AM
Last activity: Nov 28, 2024, 05:42 PM