Sample Header Ad - 728x90

Using cURL, jq, and declaration and for loop condition, I tried to download multiple files from a GitLab private repo, but it downloaded only one

1 vote
1 answer
1374 views
I learned from the following sources: - **curl -O**: [Download a file with curl on Linux / Unix command line](https://www.cyberciti.biz/faq/download-a-file-with-curl-on-linux-unix-command-line/) - **jq:** [How to urlencode data for curl command?](https://stackoverflow.com/questions/296536/how-to-urlencode-data-for-curl-command/34407620#34407620) - **Multiple files and curl -J:** https://unix.stackexchange.com/questions/91798/download-pdf-files-from-using-curl - **Condition for loop:** https://unix.stackexchange.com/questions/377446/shell-how-to-use-2-variables-with-for-condition and https://unix.stackexchange.com/questions/291049/unable-to-download-data-using-curl-for-loop Description of the script: 1. Variables, required by GitLab's [Repository files API](https://docs.gitlab.com/ee/api/repository_files.html) :
branch="master"
      repo="my-dotfiles"
      private_token="XXY_wwwwwx-qQQQRRSSS"
      username="gusbemacbe"
2. I used a declaration for multiple files:
declare -a context_dirs=(
        "home/.config/Code - Insiders/Preferences"
        "home/.config/Code - Insiders/languagepacks.json"
        "home/.config/Code - Insiders/rapid_render.json"
        "home/.config/Code - Insiders/storage.json"
      )
3. I used the condition for loop with jq to convert all files from the declaration context_dirs to encoded URLs:
for urlencode in "${context_dirs[@]}"; do
        paths=$(jq -nr --arg v "$urlencode" '$v|@uri')
      done
4. I used the condition for loop to download with curl multiple files taken from paths converted by jq. It is important that I used -0 and -J to output the file name, and -H for "PRIVATE-TOKEN: $private_token":
for file in "${paths[@]}"; do 
          curl -sLOJH "PRIVATE-TOKEN: $private_token" "https://gitlab.com/api/v4/projects/$username%2F$repo/repository/files/$file/raw?ref=$branch "
      done
Complete source code:
branch="master"
id="1911000X"
repo="my-dotfiles"
private_token="XXY_wwwwwx-qQQQRRSSS"
username="gusbemacbe"

declare -a context_dirs=(
  "home/.config/Code - Insiders/Preferences"
  "home/.config/Code - Insiders/languagepacks.json"
  "home/.config/Code - Insiders/rapid_render.json"
  "home/.config/Code - Insiders/storage.json"
)

for urlencode in "${context_dirs[@]}"; do
  paths=$(jq -nr --arg v "$urlencode" '$v|@uri')
done

for file in "${paths[@]}"; do 
    curl -sLOJH "PRIVATE-TOKEN: $private_token" "https://gitlab.com/api/v4/projects/$username%2F$repo/repository/files/$file/raw?ref=$branch "
done
But the two conditions for loop output only an encoded path and downloaded only a file.
Asked by Oo'- (255 rep)
Jun 1, 2020, 06:20 AM
Last activity: Jun 1, 2020, 06:50 AM