Sample Header Ad - 728x90

Fetch data from a large file based on time and date recursively

1 vote
3 answers
137 views
Fetch data based on date(column 1) and time (column 2). Each date has time in column 2. Based on each date in column1 two files are to be generated which will have all the fields based on time 6:00 to 21:50:08 (day) and 22:00:00 to 5:50:00 (night). Trying to get two files for each date as date_day and date_night based on the times specified. Input file:
Date       Time     R1      R2      R3
03/10/2023 19:00:08	19.06	39.870	5.12
03/10/2023 19:10:08	18.87	39.970	4.98
03/10/2023 19:20:08	18.68	39.940	4.80
03/10/2023 19:30:08	18.84	40.110	5.01
03/10/2023 19:40:08	18.89	38.960	4.64
03/10/2023 19:50:08	18.60	39.100	4.43
03/10/2023 23:30:08	18.03	34.200	2.03
03/10/2023 23:40:08	17.94	33.930	1.84
03/10/2023 23:50:08	17.87	33.840	1.74
03/11/2023 00:00:08	17.75	33.790	1.61
03/11/2023 00:10:08	17.96	34.060	1.91
03/11/2023 00:20:08	18.13	33.690	1.91
03/11/2023 00:30:08	17.91	33.620	1.68
Output files: 03/10/2023_day
Date        Time    R1      R2      R3
03/10/2023 19:00:08	19.06	39.870	5.12
03/10/2023 19:10:08	18.87	39.970	4.98
03/10/2023 19:20:08	18.68	39.940	4.80
03/10/2023 19:30:08	18.84	40.110	5.01
03/10/2023 19:40:08	18.89	38.960	4.64
03/10/2023_night
Date        Time     R1      R2      R3
03/10/2023 19:50:08	18.60	39.100	4.43
03/10/2023 23:30:08	18.03	34.200	2.03
03/10/2023 23:40:08	17.94	33.930	1.84
03/10/2023 23:50:08	17.87	33.840	1.74
03/11/2023_night:
Date       Time      R1     R2      R3
03/11/2023 00:00:08	17.75	33.790	1.61
03/11/2023 00:10:08	17.96	34.060	1.91
03/11/2023 00:20:08	18.13	33.690	1.91
03/11/2023 00:30:08	17.91	33.620	1.68
I tried the following to fetch day and night files. This I will have to do for each date again and again. I removed the ':' from the time column in my code to get a number without :. Can anyone please help to put it in a loop and the separate day and night files for each date? awk '$1 ~ /03\/10\/2023/ && $2 >= 060000 && $2 03_10_23_day.csv awk '$1 ~ /03\/10\/2023/ && $2 > 215000' data |sed 's/\t/,/g' > 03_10_23_night.csv
Asked by Gavin (91 rep)
Feb 12, 2024, 12:59 AM
Last activity: Feb 12, 2024, 05:26 PM