Optimize ext4 for always full operation

Our application writes data to disk as a huge ring buffer (30 to 150TB); writing new files while deleting old files. As such, by definition, the disk is always "near full". The *writer* process creates various files at a net input speed of about 100-150 Mbits/s. Data files are a mixture of 1GB 'data' files and several smaller meta data files. (The input speed is constant, but note new file sets are created only once per two minutes). There is a separate *deleter* process which deletes the "oldest" files every 30s. It keeps deleting until there it reaches 15GB free space headroom on the disk. So in stable operation, all data partitions have only 15GB free space. On this SO question relating to file system slowdown, DepressedDaniel commented: > Sync hanging just means the filesystem is working hard to save the > latest operations consistently. It is most certainly trying to shuffle > data around on the disk in that time. I don't know the details, but > I'm pretty sure if your filesystem is heavily fragmented, ext4 will > try to do something about that. And that can't be good if the > filesystem is nearly 100% full. The only reasonable way to utilize a > filesystem at near 100% of capacity is to statically initialize it > with some files and then overwrite those same files in place (to avoid > fragmenting). Probably works best with ext2/3. Is ext4 a bad choice for this application? Since we are running live, what tuning can be done to ext4 to avoid fragmentation, slow downs, or other performance limitations? Changing from ext4 would be quite difficult... (and re-writing statically created files means rewriting the entire application) Thanks! **EDIT I** The server has 50 to 100 TB of disks attached (24 drives). The Areca RAID controller manages the 24 drives as a RAID-6 raid set. From there we divide into several partitions/volumes, with each volume being 5 to 10TB. So the size of any one volume is not huge. The "writer" process finds the first volume with "enough" space and writes a file there. After the file is written the process is repeated. For a brand new machine, the volumes are filled up in order. If all volumes are "full" then the "deleter" process starts deleting the oldest files until "enough" space is available. Over a long time, because of the action of other processes, the time sequence of files becomes randomly distributed across all volumes. **EDIT II** Running fsck shows very low fragmentation: 1 - 2%. However, in the meantime, slow filesystem access has been traced to various system calls like fclose(), fwrite(), ftello() etc taking a very long time to execute (5 to 60 seconds!). So far no solution on this problem. See more details at this SO question: How to debug very slow (200 sec) fwrite()/ftello()/fclose()? I've disabled sysstat and raid-check to see if any improvement.