Sample Header Ad - 728x90

Need to unzip files and then move to AWS S3 bucket automatically

0 votes
1 answer
2123 views
I have a directory in ubuntu example:
/home/ubuntu/mainfiles/
where only zip files get uploaded daily. I need to unzip these files and move the extracted files to S3 bucket. Each unzipped file's files should be moved to s3 at 5 minutes interval. I also need to make sure duplicate files doesn't get uploaded to s3. How do I write a script for this ? What I am currently doing is * Copy the oldest file from /home/ubuntu/mainfiles/ directory each 5 minute withs cron and then storing it on a temp1 directory. * Then I unzip all the files from temp1 directory and move the extracted files to temp2 directory. * Finally I move all files on temp2 directory to s3. But I think above approach might fail because I have to clean the temp1 folder each time. Also duplicate files can get moved to s3 because I don't know how to get the file name and rename it with random name and then move to s3. Can someone provide a demo shell script for this with right approach ?
Asked by Anirban Dutta (1 rep)
May 28, 2020, 09:57 AM
Last activity: Jul 10, 2025, 09:02 AM