What can cause "listing a directory" on Android to be 50-250 times slower than expected?
2
votes
1
answer
82
views
I ended up with my Android device (Google Pixel 7a, on AOSP 15) having accumulated about **~14k** videos/photos files in
/sdcard/DCIM/Camera
. As a consequence I have noticed some very *unexpected* delay when listing this directory in this particular folder, *but also* in general.
I do run the following tests
[user@archlinux-box ~] $ ## having the Pixle 7a connected via USB-3 port USB-debugging
[user@archlinux-box ~] $ adb shell
lynx:/ $ ## time the listing of /sdcard/DCIM/Camera
lynx:/ $ (cd /sdcard/DCIM/Camera; time ls | wc)
13980 13980 398700
0m04.55s real 0m00.07s user 0m00.31s system
lynx:/ $ ## exclude that it is a fault ot ls
binary I check via find
too
lynx:/ $ (cd /sdcard/DCIM/Camera; time find | wc)
13981 13981 426662
0m05.25s real 0m00.07s user 0m00.41s system
Being suspicious that listing ~14k files may very well take some time, yet not ~5seconds (and less so upon 2nd time directory listing reads, given I would assume some caching to occur) I tested if this is related to the specific folder /sdcard/DCIM/Camera
doing this
lynx:/ $ ## create and enter a test directory to hold 14k files
lynx:/ $ mkdir /sdcard/14k-files-dir
lynx:/ $ cd /sdcard/14k-files-dir
lynx:/sdcard/14k-files-dir $ ## create 14k files somewhat efficiently
lynx:/sdcard/14k-files-dir $ ## have filenams be comparable length as photo files
lynx:/sdcard/14k-files-dir $ PREFIX='filename-similar-to-photo-files-file-no-'
lynx:/sdcard/14k-files-dir $ seq 1 14000 | sed 's/^/'"$PREFIX"'/' | xargs sh -c 'touch "$@"' _
lynx:/sdcard/14k-files-dir $ ## rerun tests
130|lynx:/sdcard/14k-files-dir $ time ls | wc
14000 14000 618894
0m01.92s real 0m00.10s user 0m00.28s system
lynx:/sdcard/14k-files-dir $ time find | wc
14001 14001 646896
0m00.72s real 0m00.03s user 0m00.13s system
lynx:/sdcard/14k-files-dir $ time ls | wc
14000 14000 618894
0m00.80s real 0m00.10s user 0m00.13s system
as can be seen this time, the directory listing is far faster (particularly once it appears to be in cache).
Lastly I wanted to test - given Android being based on linux kernel - how well my 10+ year laptop would compare.
[user@archlinux-box ~] $ ## running test on a 10 year old laptop
[user@archlinux-box ~] $ mkdir 14k-files-dir
[user@archlinux-box ~] $ cd 14k-files-dir
[user@archlinux-box 14k-files-dir ] $ ## create 14k files somewhat efficiently
[user@archlinux-box 14k-files-dir ] $ ## have filenams be comparable length as photo files
[user@archlinux-box 14k-files-dir ] $ PREFIX='filename-similar-to-photo-files-file-no-'
[user@archlinux-box 14k-files-dir ] $ seq 1 14000 | sed 's/^/'"$PREFIX"'/' | xargs sh -c 'touch "$@"' _
[user@archlinux-box 14k-files-dir ] $ ## run tests on laptop
[user@archlinux-box 14k-files-dir]$ time ls | wc
14000 14000 632894
real 0m0.025s
user 0m0.015s
sys 0m0.014s
[user@archlinux-box 14k-files-dir]$ time find | wc
14001 14001 660896
real 0m0.030s
user 0m0.019s
sys 0m0.020s
[user@archlinux-box 14k-files-dir]$ time ls | wc
14000 14000 632894
real 0m0.021s
user 0m0.009s
sys 0m0.015s
As can be seen the speed listing directories differ vastly. I am puzzled to determine the reasons.
1. It is mostly unclear why the direct "on Android device" comparison of directories listing times is so huge (1-0.7seconds vs 4-5seconds for /sdcard/DCIM/Camera
)
2. It is puzzling as I would expect both folders residing on the same block-device and filesystem being both below /sdcard
?
3. In a bigger picture however I cannot wrap my haed around (and seek with this question the causes that a 10 year old linux laptop can list 14k files at 0.02seconds while a rather recent Google Pixel 7a needs 0.7 to 5+ seconds, so plainly 250 times as long?
#Update#
I have found this info on redit which mentions
> As mentioned below, it seems on Android 11 ALL file access to external storage goes through SAF, even when targeting API 29 and using standard file access functions. This is a change from Android 10. This explains the difference in speed.
I hence wonder (with regarding to the causes I seek in this question here) if this this still the reason?
Also with this new information I cannot still see why the two folders still have 5-10 times different directory lookup speeds.
Asked by humanityANDpeace
(807 rep)
Nov 28, 2024, 07:39 AM
Last activity: Nov 28, 2024, 05:42 PM
Last activity: Nov 28, 2024, 05:42 PM