Unix & Linux Stack Exchange
Q&A for users of Linux, FreeBSD and other Unix-like operating systems
Latest Questions
1
votes
1
answers
104
views
Use axel to get source instead curl for fetchurl in NixOS
I'm trying to start a shell with Cuda. I have following shell.nix file ```nix { pkgs ? import { } }: pkgs.mkShell { name = "cuda-env-shell"; buildInputs = with pkgs; [ unzip # other packages cudatoolkit # other packages binutils ]; } ``` But this fails due to following error. I have faced this befor...
I'm trying to start a shell with Cuda. I have following shell.nix file
{ pkgs ? import { } }:
pkgs.mkShell {
name = "cuda-env-shell";
buildInputs = with pkgs; [
unzip
# other packages
cudatoolkit
# other packages
binutils
];
}
But this fails due to following error. I have faced this before with large files like davinci-resolve for example. For davinci-resolve, my solution was to create a new flake using axel with 10 connections and it worked nicely every single time. Right now I'm facing this issue with Cuda.
[1/0/3 built, 4 copied (183.8/183.8 MiB), 42.8 MiB DL] building cuda_11.8.0_520.61.05_linux.run: Dload Upload Total Spent Left
error: builder for '/nix/store/idz14mzbsv4k5v5ims095jcjk2r50glr-cuda_11.8.0_520.61.05_linux.run.drv' failed with exit code 1;
last 7 log lines:
>
> trying https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 49 4135M 49 2044M 0 0 772k 0 1:31:20 0:45:08 0:46:12 0
> curl: (56) Recv failure: Connection reset by peer
I want to get Cuda but can I retrieve the src using axel instead of curl? I tried something like following but it doesn't seem to be valid syntax?
cudatoolkit.overrideAttrs
(finalAttrs: previousAttrs: {
pname = previousAttrs.pname + "-bar";
src = runCommandLocal "${previousAttrs.pname}.run" rec { } ''
axel \
--num-connections=10 \
--verbose \
--insecure \
--output $out \
"https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run "
'';
})
Error:
at /nix/store/0g53xyh39z3y90p4d8r341wbqyjy1zhl-source/pkgs/stdenv/generic/make-derivation.nix:395:7:
394| depsHostHost = elemAt (elemAt dependencies 1) 0;
395| buildInputs = elemAt (elemAt dependencies 1) 1;
| ^
396| depsTargetTarget = elemAt (elemAt dependencies 2) 0;
error: Dependency is not of a valid type: element 13 of buildInputs for cuda-env-shell
s1n7ax
(437 rep)
Jan 12, 2024, 12:33 PM
• Last activity: Jan 13, 2024, 09:39 AM
10
votes
3
answers
11919
views
Download only a part of a file
I am trying to find if it is possible to download a part of a file (any file) via `axel` or `aria2` or any other tools available. For example there is a movie file (because doing this kind of thing make sense in A/V files). How could I download only the last 50MB or the third 60Mb from the first of...
I am trying to find if it is possible to download a part of a file (any file) via
axel
or aria2
or any other tools available.
For example there is a movie file (because doing this kind of thing make sense in A/V files). How could I download only the last 50MB or the third 60Mb from the first of the file?
The files are on HTTP servers mostly; but could be on FTP servers. If it is possible to download them via other protocols, this would also be an option.
r004
(3549 rep)
Mar 25, 2014, 06:12 AM
• Last activity: Nov 1, 2023, 10:19 PM
1
votes
1
answers
2995
views
Download error with youtube-dl and axel
I am trying to download following video: https://www.youtube.com/watch?v=pQWj2Fgxdrc by combining 'youtube-dl' and 'axel'. I am trying to download this video in such a way that I want these two softwares (youtube-dl and axel) to not only save this video as "interview.mp4" but also limit download spe...
I am trying to download following video:
https://www.youtube.com/watch?v=pQWj2Fgxdrc
by combining 'youtube-dl' and 'axel'. I am trying to download this video in such a way that I want these two softwares (youtube-dl and axel) to not only save this video as "interview.mp4" but also limit download speed to 200KB. For this purpose, I used following command:
youtube-dl -f 22 -c -i --external-downloader /usr/bin/axel -o 'interview.mp4' -s 200000 https://www.youtube.com/watch?v=pQWj2Fgxdrc
Here I am using following options:
> -f 22 for video format
>
> -c for resume download
>
> -i for ignore errors
>
> -o for output file name
>
> -s 200000 for limit download speed to 200KB
However, instead of downloading the video at 200KB and saving it as "interview.mp4", it is giving following error:
ERROR: fixed output name but more than one file to download
I have also tried following variation of command:
youtube-dl -f 22 -c -i --external-downloader /usr/bin/axel -s 200000 -o 'interview.mp4' https://www.youtube.com/watch?v=pQWj2Fgxdrc
and
youtube-dl -f 22 -c -i --external-downloader /usr/bin/axel -o 'interview.mp4' https://www.youtube.com/watch?v=pQWj2Fgxdrc -s 200000
and
youtube-dl -f 22 -c -i -o 'interview.mp4' --external-downloader /usr/bin/axel -s 200000 https://www.youtube.com/watch?v=pQWj2Fgxdrc
But still same error is appearing. However, if I remove the option (-s 200000), the error disappear and the video starts to download. But I want to use (-s 200000) option so that it should not eat all the bandwidth.
What mistake I am doing here?
Zetro Track
(21 rep)
Feb 18, 2019, 02:10 PM
• Last activity: Jul 15, 2022, 07:04 PM
3
votes
1
answers
3276
views
HTTPS Authentication with Axel Downloader
Is there a way to use https authentication with Axel to download files? I have gone through [this answer][1] and tried what was written, but to no extent. I got `HTTP/1.1 401 Unauthorized` as the output for this input command: `axel -n 60 -k -H "Authorisation: Basic [some hash string]" "https://xxx....
Is there a way to use https authentication with Axel to download files?
I have gone through this answer and tried what was written, but to no extent. I got
as the output for this input command:
for which the hash string was generated using :
HTTP/1.1 401 Unauthorized
as the output for this input command:
axel -n 60 -k -H "Authorisation: Basic [some hash string]" "https://xxx.rar "
for which the hash string was generated using :
echo -n "username:password" | openssl base64
So ultimately, I want to know, what do I change in the input command to be able to download the file?
Pratyush Manocha
(133 rep)
Nov 29, 2017, 06:50 PM
• Last activity: Jan 6, 2019, 12:44 PM
1
votes
1
answers
2478
views
Download attachments in a URL redirect via Axel
[Axel][1] is not resolving URLs that redirect to another URL that has the file as an attachment. As you see, I'm getting 404 when I try to download this URL. (original site is replaced with example.com) ~ ▶ axel https://eusa.example.com/zipstream/1581777.zip?tunnel=1&token=b7385bb62e3111e3ace1002481...
Axel is not resolving URLs that redirect to another URL that has the file as an attachment.
As you see, I'm getting 404 when I try to download this URL. (original site is replaced with example.com)
~ ▶ axel https://eusa.example.com/zipstream/1581777.zip?tunnel=1&token=b7385bb62e3111e3ace1002481265109&storage=s09
767
768
Initializing download: https://eusa.example.com/zipstream/1581777.zip?tunnel=1
[1] Exit 1 axel https://eusa.example.com/zipstream/1581777.zip?tunnel=1
+ Done token=b7385bb62e3111e3ace1002481265109
~ ▶ HTTP/1.1 404 Not Found
The website documentations describe URL redirect as following:
> First response is always a redirect. Location of the file is in
> ‘Location’ header. The client must send a new GET request to the new
> location. Only headers are shown.
HTTP/1.1 302 FOUND
Server: nginx
Date: Thu, 09 Feb 2012 15:58:53 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Keep-Alive: timeout=60
Content-Length: 385
Location: http://s01.example.com/v2/files/26205116/download?oauth_token=4a63387841ce11e1a047001018321b64
goes to:
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 09 Feb 2012 15:58:53 GMT
Content-Type: application/x-rar
Content-Length: 3190802
Last-Modified: Fri, 16 Sep 2011 11:42:13 GMT
Connection: keep-alive
Keep-Alive: timeout=60
Content-Disposition: attachment; filename="file.rar"
Accept-Ranges: bytes
curl
is capable of downloading the URL. But axel
gives me 404. I want to have Axel's multi-connection feature. How can I use curl
to just resolve the URL and give downloading task to axel
?
Mohsen
(2705 rep)
Dec 14, 2013, 04:11 AM
• Last activity: Dec 15, 2013, 03:04 AM
Showing page 1 of 5 total questions