Sample Header Ad - 728x90

How to count the number of bytes in a very large file, grouping the same bytes?

1 vote
4 answers
1569 views
I am searching for a way to get a statistics on a very large (multiple times larger than the available RAM) the outputs what byte values in the files are present and how often: A0 01 00 FF 77 01 77 01 A0 I need to know how many A0 bytes there are in this file, how many 01, and so on. The result could be: A0: 2 01: 3 00: 1 FF: 1 77: 2 Therefore this question is very close to the question [How to count the number of bytes in a file, grouping the same bytes?](https://unix.stackexchange.com/questions/527521/how-to-count-the-number-of-bytes-in-a-file-grouping-the-same-bytes) but non of the existing answers works for larger files. From my understanding all answers require a minimum RAM equal to the size of the file to be tested (up to multiple times). Hence the answers don't work on systems with small RAM, e.g. a Raspberry for processing a multi-GB file. Is there a simple solution that works on any file size even if we have for example only 512MB RAM available?
Asked by Robert (163 rep)
Mar 30, 2020, 12:31 PM
Last activity: Mar 31, 2020, 12:51 AM