How to Free Up Space on Full overlay and /dev/vda1 Filesystems in a Linux Environment?

I am facing an issue where the overlay and /dev/vda1 filesystems on my server are both completely utilized, showing 100% usage. This scenario has led to operational challenges, and I am seeking advice on how to effectively free up space or manage storage to mitigate this issue. Here is the output of df -h for context: Filesystem Size Used Avail Use% Mounted on overlay 97G 97G 0 100% / tmpfs 64M 0 64M 0% /dev tmpfs 111G 0 111G 0% /sys/fs/cgroup /dev/vdb 393G 65G 329G 17% /root/easymaker tmpfs 222G 12K 222G 1% /dev/shm 198.18.32.36:/GJ_SHARE_FS11/8683766f-1a8c-4531-87ba-2370c3ff2ad3 20T 0 20T 0% /root/backup 198.18.32.7:/GJ_SHARE_FS9/5470219b-58ed-4a37-9de0-1788451ad4b4 49T 16M 49T 1% /root/data /dev/vda1 97G 97G 0 100% /etc/hosts tmpfs 222G 12K 222G 1% /run/secrets/kubernetes.io/serviceaccount As you can see, both the overlay and /dev/vda1 filesystems are at full capacity. I am currently running a cloud environment, which may be contributing to the rapid space utilization on these filesystems. I have looked into common solutions such as pruning unused Docker images and containers, but I am cautious and seeking guidance on best practices, especially considering the operational importance of the data and applications running on this server. Could anyone provide insights or recommended strategies for safely freeing up space or preventing such issues in Docker environments? Are there specific logs, temporary files, or data that can be safely removed or moved to other storage without disrupting ongoing operations? 1. Attempted to identify large files or directories consuming disk space using du -ah / | sort -rh | head -20. - Expected: To list the top 20 largest files or directories. - Actual: Received an error message "sort: write failed: /tmp/sortVOhsjQ: No space left on device." 2. Tried to remove large files or directories to free up disk space using rm -rf /overlay/*. - Expected: To delete files or directories within the specified path. - Actual: Received an error message indicating that the path does not exist. 3. Attempted to remove the /etc/hosts file to free up space on the /dev/vda1 filesystem. - Expected: To successfully remove the file. - Actual: Received an error message "rm: cannot remove '/etc/hosts': Device or resource busy." 4. Tried to update the package repositories using apt-get update. - Expected: To update the package repositories successfully. - Actual: Received errors due to lack of disk space, such as "Splitting up /var/lib/apt/lists/developer.download.nvidia.com_compute_cuda_repos_ubuntu2204_x86%5f64_InRelease into data and signature failed." 5. Attempted to clean up unused Docker resources using docker network rm cw-net. - Expected: To remove the specified Docker network. - Actual: Received an error message "bash: docker: command not found." 6. Tried to identify large files or directories within the /overlay directory using du -ah /overlay | sort -rh | head -n 20. - Expected: To list the top 20 largest files or directories within the specified path. - Actual: Received an error message "du: cannot access '/overlay': No such file or directory." Overall, despite various attempts to free up disk space and identify large files or directories, encountered limitations due to lack of available disk space and inability to execute certain commands. Additional solutions or assistance may be required to address the underlying issue effectively.