I'm creating a small script that will delete indexes on an Elasticsearch cluster to prevent it for fill up all the storage with logstash data.
I have a list of records, and I would like to keep the latest n records (for example 7) and delete all the others.
I can get the list of the indexes with curl:
drakaris:~/ # curl -sL localhost:9200/_cat/indices/logstash-* | awk '{print $3;}' | sort
logstash-2022.12.30
logstash-2022.12.31
logstash-2023.01.01
logstash-2023.01.02
logstash-2023.01.03
logstash-2023.01.04
logstash-2023.01.05
logstash-2023.01.06
logstash-2023.01.07
logstash-2023.01.08
logstash-2023.01.09
In my script I would like to keep only the latest 7th indexes and delete all the others (logstash-2022.12.30, logstash-2022.12.31. logstash-2023.01.01, logstash-2023.01.02) using "curl -XDELETE localhost:9200/index".
How can I get these records from an array like that in bash?
Thanks
---
[EDIT]
I solved in this way, just in case someone find it useful
RETENTION=7
nbk=$(curl -sL localhost:9200/_cat/indices/logstash-* | awk '{print $3;}' | wc -l)
if [ $nbk -gt $RETENTION ]; then
echo -e "======== Delete obsolete indexes (retention: $RETENTION)"
let ntodel=$nbk-$RETENTION
for efile in $(curl -sL localhost:9200/_cat/indices/logstash-* | awk '{print $3;}' | sort -r | /usr/bin/tail -$ntodel); do
curl -XDELETE localhost:9200/$efile
sleep 10
done
fi
Asked by Tasslehoff Burrfoot
(1 rep)
Jan 9, 2023, 02:58 PM
Last activity: Jan 10, 2023, 09:27 AM
Last activity: Jan 10, 2023, 09:27 AM