Unix & Linux Stack Exchange
Q&A for users of Linux, FreeBSD and other Unix-like operating systems
Latest Questions
0
votes
2
answers
1999
views
Problem Unzipping huge zip file
I have issues unzipping a huge zip files containing around 1M files. The zip file is 15GB and uncompresses to ~60GB. When I run `unzip file.zip -d /directory/to/unzip/at` it uncompresses halfway and gives up at around 700K files. No error messages. Any tips?
I have issues unzipping a huge zip files containing around 1M files. The zip file is 15GB and uncompresses to ~60GB. When I run
unzip file.zip -d /directory/to/unzip/at
it uncompresses halfway and gives up at around 700K files. No error messages.
Any tips?
Anon
(9 rep)
Jan 24, 2018, 07:22 PM
• Last activity: Jul 27, 2025, 04:05 PM
2
votes
1
answers
2724
views
libzip Linux cross-compile configuration
I am trying to cross compile **libzip-1.5.1** having previously successfully complied **zlib-1.2.11** (these are required by libzip and I have installed in a local home path), but I am experiencing problems with the cmake's process. The script I use to build and compile (from a build directory into...
I am trying to cross compile **libzip-1.5.1** having previously successfully complied **zlib-1.2.11** (these are required by libzip and I have installed in a local home path), but I am experiencing problems with the cmake's process.
The script I use to build and compile (from a build directory into the libzip's sources folder) is the following:
#!/bin/sh
PREFIX=${PWD}/install
CCPATH=/opt/arm64/gcc-linaro-6.3.1-2017.02-rc2-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf
ZLIBPATH=${PWD}/../../zlib-1.2.11/
# here is where zlib is installed
# ZLIBINSTALLEDPATH contains include, lib, share directories
ZLIBINSTALLEDPATH=${ZLIBPATH}/build_armhf64/install/
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:${ZLIBINSTALLEDPATH}/lib/pkgconfig/"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${ZLIBINSTALLEDPATH}/lib/"
export CMAKE_AR=${CCPATH}-ar
export CC=${CCPATH}-gcc
export CXX=${CCPATH}-g++
export CMAKE_LINKER=${CCPATH}-ld
export CMAKE_RANLIB=${CCPATH}-ranlib
export CMAKE_OBJDUMP=${CCPATH}-objdump
export CMAKE_OBJCOPY=${CCPATH}-objcopy
export CMAKE_STRIP=${CCPATH}-strip
#export CMAKE_READELF=${CCPREFIX}-readelf
export CMAKE_NM=${CCPATH}-nm
cmake -DCMAKE_INSTALL_PREFIX=${PREFIX} \
-DZLIB_INCLUDE_DIR=${ZLIBINSTALLEDPATH}/include/ \
-DZLIB_LIBRARY=${ZLIBINSTALLEDPATH}/lib/ \
../
make -j 8
But at 94 % of building process, I get the following errors:
[ 92%] Building C object regress/CMakeFiles/tryopen.dir/tryopen.c.o
[ 94%] Building C object src/CMakeFiles/zipcmp.dir/zipcmp.c.o
[ 94%] Linking C executable add_from_filep
[ 94%] Linking C executable fopen_unchanged
[ 94%] Linking C executable tryopen
[ 94%] Building C object regress/CMakeFiles/hole.dir/source_hole.c.o
[ 94%] Linking C executable zipmerge
../lib/libzip.so.5.0: undefined reference to `inflate'
../lib/libzip.so.5.0: undefined reference to `crc32'
../lib/libzip.so.5.0: undefined reference to `zError'
../lib/libzip.so.5.0: undefined reference to `deflate'
../lib/libzip.so.5.0: undefined reference to `deflateInit2_'
../lib/libzip.so.5.0: undefined reference to `inflateEnd'
../lib/libzip.so.5.0: undefined reference to `deflateEnd'
../lib/libzip.so.5.0: undefined reference to `inflateInit2_'
collect2: error: ld returned 1 exit status
regress/CMakeFiles/add_from_filep.dir/build.make:95: recipe for target 'regress/add_from_filep' failed
make: *** [regress/add_from_filep] Error 1
CMakeFiles/Makefile2:726: recipe for target 'regress/CMakeFiles/add_from_filep.dir/all' failed
make: *** [regress/CMakeFiles/add_from_filep.dir/all] Error 2
make: *** Waiting for unfinished jobs....
../lib/libzip.so.5.0: undefined reference to `inflate'
../lib/libzip.so.5.0: undefined reference to `crc32'
../lib/libzip.so.5.0: undefined reference to `zError'
../lib/libzip.so.5.0: undefined reference to `deflate'
../lib/libzip.so.5.0: undefined reference to `deflateInit2_'
../lib/libzip.so.5.0: undefined reference to `inflateEnd'
../lib/libzip.so.5.0: undefined reference to `deflateEnd'
../lib/libzip.so.5.0: undefined reference to `inflateInit2_'
collect2: error: ld returned 1 exit status
regress/CMakeFiles/tryopen.dir/build.make:95: recipe for target 'regress/tryopen' failed
make: *** [regress/tryopen] Error 1
CMakeFiles/Makefile2:800: recipe for target 'regress/CMakeFiles/tryopen.dir/all' failed
make: *** [regress/CMakeFiles/tryopen.dir/all] Error 2
../lib/libzip.so.5.0: undefined reference to `inflate'
../lib/libzip.so.5.0: undefined reference to `crc32'
../lib/libzip.so.5.0: undefined reference to `zError'
../lib/libzip.so.5.0: undefined reference to `deflate'
../lib/libzip.so.5.0: undefined reference to `deflateInit2_'
../lib/libzip.so.5.0: undefined reference to `inflateEnd'
../lib/libzip.so.5.0: undefined reference to `deflateEnd'
../lib/libzip.so.5.0: undefined reference to `inflateInit2_'
collect2: error: ld returned 1 exit status
regress/CMakeFiles/fopen_unchanged.dir/build.make:95: recipe for target 'regress/fopen_unchanged' failed
make: *** [regress/fopen_unchanged] Error 1
CMakeFiles/Makefile2:615: recipe for target 'regress/CMakeFiles/fopen_unchanged.dir/all' failed
make: *** [regress/CMakeFiles/fopen_unchanged.dir/all] Error 2
[ 94%] Building C object regress/CMakeFiles/ziptool_regress.dir/source_hole.c.o
../lib/libzip.so.5.0: undefined reference to `inflate'
../lib/libzip.so.5.0: undefined reference to `crc32'
../lib/libzip.so.5.0: undefined reference to `zError'
../lib/libzip.so.5.0: undefined reference to `deflate'
../lib/libzip.so.5.0: undefined reference to `deflateInit2_'
../lib/libzip.so.5.0: undefined reference to `inflateEnd'
../lib/libzip.so.5.0: undefined reference to `deflateEnd'
../lib/libzip.so.5.0: undefined reference to `inflateInit2_'
collect2: error: ld returned 1 exit status
src/CMakeFiles/zipmerge.dir/build.make:95: recipe for target 'src/zipmerge' failed
make: *** [src/zipmerge] Error 1
CMakeFiles/Makefile2:432: recipe for target 'src/CMakeFiles/zipmerge.dir/all' failed
make: *** [src/CMakeFiles/zipmerge.dir/all] Error 2
[ 94%] Linking C executable zipcmp
[ 94%] Linking C executable ziptool
[ 96%] Linking C executable hole
CMakeFiles/zipcmp.dir/zipcmp.c.o.:. /Inlib /functionlibzip.so.5.0 :` compute_crcundefined' :reference
zipcmp.cto: (`.inflatetext'+
0x4a0.).:/ libundefined/ libzip.so.5.0reference: toundefined `referencecrc32 'to
zipcmp.c`:crc32('.
text.+.0x53e/)lib:/ libzip.so.5.0undefined: referenceundefined toreference
tocrc32 '
zErrorCMakeFiles'/
zipcmp.dir./.zipcmp.c.o/:lib /Inlibzip.so.5.0 :function undefined` test_filereference':
zipcmp.cto: (`.deflatetext'+
0x1494.).:/ libundefined/ libzip.so.5.0reference: toundefined `referencecrc32'
zipcmp.c:(.text+0x14fe): undefined reference to `crc32'
.. /tolib /`libzip.so.5.0deflateInit2_:'
undefined. .reference/ libto/ libzip.so.5.0`:inflate 'undefined
.reference. /tolib /`libzip.so.5.0inflateEnd:'
undefined. .reference/ libto/ libzip.so.5.0`:zError 'undefined
.reference. /tolib /libzip.so.5.0deflateEnd:' undefined reference to
deflate'
.
../.lib//liblibzip.so.5.0/:libzip.so.5.0 :undefined undefinedreference referenceto to deflateInit2_
'inflateInit2_
'.
./lib/libzip.so.5.0: undefined reference to `inflateEnd'
../lib/libzip.so.5.0: undefined reference to `deflateEnd'
../lib/libzip.so.5.0: undefined reference to `collect2: error: ld returned 1 exit status
inflateInit2_'
collect2: error: ld returned 1 exit status
src/CMakeFiles/ziptool.dir/build.make:95: recipe for target 'src/ziptool' failed
make: *** [src/ziptool] Error 1
src/CMakeFiles/zipcmp.dir/build.make:95: recipe for target 'src/zipcmp' failed
make: *** [src/zipcmp] Error 1
CMakeFiles/Makefile2:469: recipe for target 'src/CMakeFiles/ziptool.dir/all' failed
make: *** [src/CMakeFiles/ziptool.dir/all] Error 2
CMakeFiles/Makefile2:395: recipe for target 'src/CMakeFiles/zipcmp.dir/all' failed
make: *** [src/CMakeFiles/zipcmp.dir/all] Error 2
../lib/libzip.so.5.0: undefined reference to `inflate'
../lib/libzip.so.5.0: undefined reference to `crc32'
../lib/libzip.so.5.0: undefined reference to `zError'
../lib/libzip.so.5.0: undefined reference to `deflate'
../lib/libzip.so.5.0: undefined reference to `deflateInit2_'
../lib/libzip.so.5.0: undefined reference to `inflateEnd'
../lib/libzip.so.5.0: undefined reference to `deflateEnd'
../lib/libzip.so.5.0: undefined reference to `inflateInit2_'
collect2: error: ld returned 1 exit status
[ 98%] Linking C executable ziptool_regress
regress/CMakeFiles/hole.dir/build.make:121: recipe for target 'regress/hole' failed
make: *** [regress/hole] Error 1
CMakeFiles/Makefile2:652: recipe for target 'regress/CMakeFiles/hole.dir/all' failed
make: *** [regress/CMakeFiles/hole.dir/all] Error 2
../lib/libzip.so.5.0: undefined reference to `inflate'
../lib/libzip.so.5.0: undefined reference to `crc32'
../lib/libzip.so.5.0: undefined reference to `zError'
../lib/libzip.so.5.0: undefined reference to `deflate'
../lib/libzip.so.5.0: undefined reference to `deflateInit2_'
../lib/libzip.so.5.0: undefined reference to `inflateEnd'
../lib/libzip.so.5.0: undefined reference to `deflateEnd'
../lib/libzip.so.5.0: undefined reference to `inflateInit2_'
collect2: error: ld returned 1 exit status
regress/CMakeFiles/ziptool_regress.dir/build.make:121: recipe for target 'regress/ziptool_regress' failed
make: *** [regress/ziptool_regress] Error 1
CMakeFiles/Makefile2:689: recipe for target 'regress/CMakeFiles/ziptool_regress.dir/all' failed
make: *** [regress/CMakeFiles/ziptool_regress.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
It seems that cmake cannot find the compiled zlib libraries, even if the paths seems to be well configured (or not ?).
Do some has experienced similar issues with libzip ?
Is there any addition cmake's macro that would solve this issue ?
**---- UPDATE ----**
----------
I actually get rid of this error suppressing building task for **zipcmp**, **zipmerge** and **ziptool** (since I don't need them) commenting the lines:
#ADD_EXECUTABLE(zipcmp zipcmp.c ${SRC_EXTRA_FILES})
#TARGET_LINK_LIBRARIES(zipcmp zip)
#INSTALL(TARGETS zipcmp RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
#ADD_EXECUTABLE(zipmerge zipmerge.c ${SRC_EXTRA_FILES})
#TARGET_LINK_LIBRARIES(zipmerge zip)
#INSTALL(TARGETS zipmerge RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
#ADD_EXECUTABLE(ziptool ziptool.c ${SRC_EXTRA_FILES})
#TARGET_LINK_LIBRARIES(ziptool zip)
#INSTALL(TARGETS ziptool RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
into src/CMakeLists.txt
.
Then I compiled the libs using make -j 8 zip
and then make install/fast
.
It still returns the error
Install the project...
-- Install configuration: ""
-- Installing: /home/simo/adtdev/Var_libs/libzip-1.5.1/build_armhf64/install/lib/pkgconfig/libzip.pc
-- Installing: /home/simo/adtdev/Var_libs/libzip-1.5.1/build_armhf64/install/include/zipconf.h
-- Installing: /home/simo/adtdev/Var_libs/libzip-1.5.1/build_armhf64/install/include/zip.h
-- Installing: /home/simo/adtdev/Var_libs/libzip-1.5.1/build_armhf64/install/lib/libzip.a
CMake Error at man/cmake_install.cmake:36 (file):
file INSTALL cannot find
"/home/simo/adtdev/Var_libs/libzip-1.5.1/build_armhf64/man/ZIP_SOURCE_GET_ARGS.3".
Call Stack (most recent call first):
cmake_install.cmake:46 (include)
Makefile:89: recipe for target 'install/fast' failed
make: *** [install/fast] Error 1
but I don't care about it since I have the libs and headers installed in the install dir, and this is what I need.
xyx
(797 rep)
Nov 8, 2018, 01:41 PM
• Last activity: May 23, 2025, 11:06 AM
0
votes
1
answers
651
views
how to skip "is this a single-disk" archive message
after running this code terminal asks me "is this a single-disk archive : y/n". where I need to type y and press enter but I want to do this from the code I am running. In other words I don't want to manually press y and then enter key. ``` rm -rf fireDataSet1 mkdir fireDataSet1 cd fireDataSet1 cp '...
after running this code terminal asks me "is this a single-disk archive : y/n". where I need to type y and press enter but I want to do this from the code I am running. In other words I don't want to manually press y and then enter key.
rm -rf fireDataSet1
mkdir fireDataSet1
cd fireDataSet1
cp '/content/fire_dataset/fire image dataset2/fire image dataset.z80' dataset.zip
zip -r -FF dataset.zip --out dataset.zip.fixed
unzip dataset.zip.fixed
Emam Kownine
(1 rep)
Sep 16, 2022, 08:52 AM
• Last activity: May 20, 2025, 04:43 PM
11
votes
1
answers
632
views
How to get zip to treat any missing file as an error?
To reproduce: ``` ❯ cd "$(mktemp --directory)" ❯ zip result.zip foo no-such-file zip warning: name not matched: no-such-file adding: foo (stored 0%) ❯ echo $? 0 ``` `zip` should return a non-zero exit code when *any* of the files are missing, but it only does so if *all* the files are missing. How d...
To reproduce:
❯ cd "$(mktemp --directory)"
❯ zip result.zip foo no-such-file
zip warning: name not matched: no-such-file
adding: foo (stored 0%)
❯ echo $?
0
zip
should return a non-zero exit code when *any* of the files are missing, but it only does so if *all* the files are missing. How do I change this behaviour?
l0b0
(53368 rep)
May 1, 2025, 12:14 PM
• Last activity: May 1, 2025, 12:20 PM
1
votes
1
answers
4373
views
Add files to zip archive skipping existing files with zip command
Is it possible to skip existing files when adding files to existing archive with zip command? `zip -u` adds only changed or new files, but I need to add only new files, skipping 'changed'. Background (although it should be irrelevant): I have zip archive produced by other process, and I want to add...
Is it possible to skip existing files when adding files to existing archive with zip command?
zip -u
adds only changed or new files, but I need to add only new files, skipping 'changed'.
Background (although it should be irrelevant): I have zip archive produced by other process, and I want to add default files, if they are missing (do not override changed files with defaults).
user224076
Sep 28, 2018, 09:29 AM
• Last activity: Apr 16, 2025, 05:01 AM
1
votes
0
answers
356
views
How can I import lxc containers back from the snapshots the lxc create just before the `lxc snap` is removed?
I have an Ubuntu 22.04 node, which used to have [lxd v5][1] installed through snap. It used to have 3 containers, at the moment of remove the lxd using the `snap remove lxd` [it did take a snapshot of all the containers][2], but when I try to import those snapshoted containers I got some errors. Fir...
I have an Ubuntu 22.04 node, which used to have lxd v5 installed through snap. It used to have 3 containers, at the moment of remove the lxd using the
snap remove lxd
it did take a snapshot of all the containers , but when I try to import those snapshoted containers I got some errors.
First, the snapshots were deliver to me in .zip format, which when I try to import using the lxc import file_name.zip
I get the *Importing instance: 100% (108.71MB/s)Error: Unsupported compression* error, weird because as I pointed I got the snapshots in .zip format.
So I needed to extract it and then compress it again using tar (tar -czvf containers.tar.gz file_contains_extracteds/
)
Once I got the correct format, then I try to import it, but I get the following error:
lxc import containers.tar.gz
Importing instance: 100% (701.66MB/s)Error: Backup is missing at "backup/index.yaml"
When I look into the extracted .zip file I get:
ls
25112 archive.tgz common meta.json meta.sha3_384 user
And inside the common
there are all the container's folder:
ls common/lxd/storage-pools/default/containers/
monitor orch1 vrouter
How can I import lxc containers back from the snapshots the lxc create just before the lxc snap is removed?
k.Cyborg
(527 rep)
Aug 25, 2023, 08:56 PM
• Last activity: Apr 14, 2025, 07:33 AM
0
votes
1
answers
39
views
Help me understand how Apple IOS does compression
There is zip, and info-zip, correct? Am I missing something? In debian when I create a zip using -Z store, or the -0 option, on a folder with -r, it shows that zip version 1.0 is needed using the file tool. The archive type is stored. Zip version on debian is infozip v3.0, dependency is libbz2-1.0....
There is zip, and info-zip, correct?
Am I missing something? In debian when I create a zip using -Z store, or the -0 option, on a folder with -r, it shows that zip version 1.0 is needed using the file tool. The archive type is stored. Zip version on debian is infozip v3.0, dependency is libbz2-1.0.
On an iPhone, I compress a file, and when I transfer it to a pc, and run file on it in a terminal it shows that version 2.0 is needed. Further information shows that the archive type is STORED. Not deflate, or compressed.
What am I missing.
shrykullgod
(39 rep)
Mar 24, 2025, 05:38 AM
• Last activity: Mar 29, 2025, 09:23 PM
3
votes
1
answers
269
views
unzip a windows archive - some subdirectories are "permission denied"
I have a zip-file created on Windows, so - no specific permissions flags were assigned to the files inside it. When I unzip the archive: unzip filename.zip -d ~/path/to/destination Some (only some, random) subdirectories 2 levels deep (and more) are not accessible to me. I can't seem to find any pat...
I have a zip-file created on Windows, so - no specific permissions flags were assigned to the files inside it.
When I unzip the archive:
unzip filename.zip -d ~/path/to/destination
Some (only some, random) subdirectories 2 levels deep (and more) are not accessible to me. I can't seem to find any pattern... Seems like if a folder has only subfolders (and no files) in it - then it will be created inaccessible, with restrictive permissions flags.
I know I can just do
chmod -R 777 ~/path/to/destination
and see everything, but this really puzzles me. What is causing this?
And how can I create a zip file (in particular, on Windows) so that it can be extracted properly on linux?
Alex
(131 rep)
Dec 5, 2022, 11:21 PM
• Last activity: Mar 29, 2025, 05:55 PM
19
votes
9
answers
25038
views
How do I recursively grep through compressed archives?
I'm trying to find out what modules [`use Test::Version`](http://search.cpan.org/dist/Test-Version/lib/Version.pm) in cpan. So I've used [`minicpan`](http://search.cpan.org/dist/CPAN-Mini/bin/minicpan) to mirror it. My problem is that I need to iterate through the archives that are downloaded, and g...
I'm trying to find out what modules [
use Test::Version
](http://search.cpan.org/dist/Test-Version/lib/Version.pm) in cpan. So I've used [minicpan
](http://search.cpan.org/dist/CPAN-Mini/bin/minicpan) to mirror it. My problem is that I need to iterate through the archives that are downloaded, and grep the files that are in the archives. Can anyone tell me how I might do this? preferably in a way that tells me which file in the archive and what line it's on.
*(note: they aren't all tarballs some are zip files)*
xenoterracide
(61203 rep)
May 25, 2011, 12:29 PM
• Last activity: Mar 20, 2025, 11:10 AM
1
votes
1
answers
119
views
unzip returns exit code 1 on empty archive
I have an valid, yet empty zip archive: ``` $ file 22.zip 22.zip: Zip archive data (empty) $ xxd 22.zip 00000000: 504b 0506 0000 0000 0000 0000 0000 0000 PK.............. 00000010: 0000 0000 0000 ``` Trying to list the archive content or unzipping the file gives both a warning and returns exit code...
I have an valid, yet empty zip archive:
$ file 22.zip
22.zip: Zip archive data (empty)
$ xxd 22.zip
00000000: 504b 0506 0000 0000 0000 0000 0000 0000 PK..............
00000010: 0000 0000 0000
Trying to list the archive content or unzipping the file gives both a warning and returns exit code 1.
$ unzip -l 22.zip; echo $?
Archive: 22.zip
warning [22.zip]: zipfile is empty
1
$ unzip 22.zip; echo $?
Archive: 22.zip
warning [22.zip]: zipfile is empty
1
I understand the warning, but the exit code seems wrong. There is nothing wrong with an empty file; for example running cat
on an empty file yields exit code 0.
What is the reasoning behind unzip
returning an exit code 1 on a valid, yet empty file?
miku
(683 rep)
Mar 13, 2025, 10:26 AM
• Last activity: Mar 13, 2025, 10:35 AM
2
votes
1
answers
296
views
When trying to unzip file from script, I get Segmentation fault (core dumped)
I'm trying to unzip a file in a bash script, but I get a 'Segmentation fault (core dumped)' error following a repeated output message of 'Unzip agent files.' ``` Unzip agent files. Unzip agent files. Unzip agent files. Unzip agent files. Unzip agent files. Unzip agent files. Unzip agent files. Unzip...
I'm trying to unzip a file in a bash script, but I get a 'Segmentation fault (core dumped)' error following a repeated output message of 'Unzip agent files.'
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Unzip agent files.
Segmentation fault (core dumped)
Please see script below:
cat /usr/local/bin/release-agent.sh
#! /bin/bash -e
source functions.sh
arg1=$1
check_arg (){
if [[ -z ${arg1} ]]; then
output "ERROR! Usage: Agent-unzip.sh [local zip file]..." red
exit 0
else
arg=${arg1}
if [[ ! -f ${arg} ]]; then
output "No local file found!" blue
exit 0
else
file=${arg}
ext=${file##*.}
if [[ ${ext} != "zip" ]]; then
output "File specified does not appear to be a zip." red
confirm "Would you like to proceed." #yes no question
if [[ $? == "1" ]]; then #if no
exit 0
fi
else
output "Local file, ${file} found." blue
fi
fi
zip="${file##*/}"
fi
}
select_version (){
prompt="Enter agent version number: "
read -p "${prompt}" version
while [ -z ${version} ]; do
output "No input entered" red
read -p "${prompt}" version
done
output "You entered: ${version}" blue
}
create_dir (){
path="Agent/agent-${version}"
output "Creating working directory ${path}" blue
mkdir -p "${path}"
}
unzip (){
output "Unzip agent files." blue
unzip -uq "$zip" -d "${path}"
}
conf_details (){
output "See details entered below:" blue
output "VERSION = ${version}" green
output "FILE PATH = ${zip}" green
confirm "Would you like to proceed? press 'Y' to initiate the release of the agent image, press 'N' to edit any details:" #yes no question
if [[ $? != "0" ]]; then #if anything but yes is returned
$0 ${arg}
exit 0
fi
}
check_arg
select_version
conf_details
create_dir
unzip
## If script has't crashed out ##
output "Success! All operations completed" green
exit 0
The output function is just something I've written to make output easier, this is below for reference:
output() {
#
# Useage output "string" [colour] {flash}
#
if [[ $3 == "flash" ]]; then
_blink=";5"
else
_blink=""
fi
case $2 in
grey)
echo -e "\e[30;1${_blink}m${1}\e[0m"
;;
red)
echo -e "\e[31;1${_blink}m${1}\e[0m"
;;
green)
echo -e "\e[32;1${_blink}m$1\e[0m"
;;
yellow)
echo -e "\e[33;1${_blink}m$1\e[0m"
;;
blue)
echo -e "\e[34;1${_blink}m$1\e[0m"
;;
magenta)
echo -e "\e[35;1${_blink}m$1\e[0m"
;;
lightblue)
echo -e "\e[36;1${_blink}m$1\e[0m"
;;
white)
echo -e "\e[37;1${_blink}m$1\e[0m"
;;
*)
echo -e "\e[0mNo formatting option!"
;;
esac
}
Chris Pavey
(115 rep)
Feb 1, 2025, 11:02 AM
• Last activity: Feb 1, 2025, 01:38 PM
1
votes
1
answers
116
views
In ImageMagick, how to create a PDF file from an image with the best Flate compression ratio?
Assume you have a PNM or PNG image file, gray or color. With ImageMagick, you wish to generate a possibly small PDF file from it without losing information. So far I though it is simply convert infile.pnm -quality 100 -compress Zip outfile.pdf or convert infile.png -quality 100 -compress Zip outfile...
Assume you have a PNM or PNG image file, gray or color. With ImageMagick, you wish to generate a possibly small PDF file from it without losing information. So far I though it is simply
convert infile.pnm -quality 100 -compress Zip outfile.pdf
or
convert infile.png -quality 100 -compress Zip outfile.pdf
Is that it? After testing, I'm no longer sure that the compression is lossless.
AlMa1r
(1 rep)
Dec 19, 2024, 06:50 PM
• Last activity: Dec 20, 2024, 03:53 PM
1
votes
1
answers
178
views
Linux extract .zip and preserve all original files timestamps (created/modified/accessed)
It's possible to restore original file timestamps during extraction on linux? I'm tested .zip extraction using 7z, unzip, unar, but only modification date is restored. Environment: - Linux Mint 22 - Filesystem: ext4 - tested.zip: it's really keeps all original timestamps creation/modification/access...
It's possible to restore original file timestamps during extraction on linux? I'm tested .zip extraction using 7z, unzip, unar, but only modification date is restored.
Environment:
- Linux Mint 22
- Filesystem: ext4
- tested.zip: it's really keeps all original timestamps creation/modification/access (it's visible e.g. in PeaZip), archive created at Win via 7z.
osb
(13 rep)
Dec 2, 2024, 08:51 AM
• Last activity: Dec 2, 2024, 09:27 AM
39
votes
1
answers
17322
views
The zip format's external file attribute
This is a slightly exotic question, but there doesn't seem to be much information on the net about this. I just added [an answer to a question about the zip format's external file attribute](https://stackoverflow.com/questions/434641/how-do-i-set-permissions-attributes-on-a-file-in-a-zip-file-using-...
This is a slightly exotic question, but there doesn't seem to be much information on the net about this. I just added [an answer to a question about the zip format's external file attribute](https://stackoverflow.com/questions/434641/how-do-i-set-permissions-attributes-on-a-file-in-a-zip-file-using-pythons-zipf/6297838#6297838) . As you can see from my answer, I conclude that only the second byte (of 4 bytes) is actually used for Unix. Apparently this contains enough information when unzipping to deduce whether the object is a file or a directory, and also has space for other permission and attribute information. My question is, how does this map to the usual Unix permissions? Do the usual Unix permissions (e.g. below) that
ls
gives fit into exactly one byte, and if so, can someone describe the layout or give a reference, please?
$ ls -la
total 36
drwxr-xr-x 3 faheem faheem 4096 Jun 10 01:11 .
drwxrwxrwt 136 root root 28672 Jun 10 01:07 ..
-rw-r--r-- 1 faheem faheem 0 Jun 10 01:07 a
drwxr-xr-x 2 faheem faheem 4096 Jun 10 01:07 b
lrwxrwxrwx 1 faheem faheem 1 Jun 10 01:11 c -> b
Let me make this more concrete by asking a specific question. Per the [Trac patch](http://trac.edgewall.org/attachment/ticket/8919/ZipDownload.patch) quoted in my answer above, you can create a zip file with the snippet of Python below.
The 040755 << 16L
value corresponds to the creation of an empty directory with the permissions drwxr-xr-x
. (I tested it). I recognize 0755
corresponds to the rwxr-xr-x
pattern, but what about the 04
, and how does the whole value correspond to a byte? I also recognize << 16L
corresponds to a bitwise left shift of 16 places, which would make it end up as the second from top byte.
def makezip1():
import zipfile
z = zipfile.ZipFile("foo.zip", mode = 'w')
zfi = zipfile.ZipInfo("foo/empty/")
zfi.external_attr = 040755 << 16L # permissions drwxr-xr-x
z.writestr(zfi, "")
print z.namelist()
z.close()
EDIT: On rereading this, I think that my conclusion that the Unix permissions only correspond to one byte may be incorrect, but I'll let the above stand for the present, since I'm not sure what the correct answer is.
EDIT2: I was indeed incorrect about the Unix values only corresponding to 1 byte. As @Random832 explained, it uses both of the top two bytes. Per @Random832's answer, we can construct the desired 040755
value from the tables he gives below. Namely:
0040000 # __S_IFDIR
+ 0400 # S_IRUSR
+ 0200 # S_IWUSR
+ 0100 # S_IXUSR
+ 0040 # S_IRGRP
# S_IWGRP skipped
+ 0010 # S_IXGRP
+ 0004 # S_IROTH
# S_IWOTH skipped
+ 0001 # S_IXOTH
== 40755
The addition here is in [base 8](http://en.wikipedia.org/wiki/Octal) .
Faheem Mitha
(36008 rep)
Jun 9, 2011, 07:52 PM
• Last activity: Nov 14, 2024, 04:56 AM
234
votes
8
answers
352192
views
Zip the contents of a folder without including the folder itself
I have a directory called `folder` that looks like this: folder -> root_folder -> some files I want to zip this directory into `zipped_dir`, I tried: zip -r zipped_dir.zip folder/* But this generates a ZIP that looks like this: zipped_dir -> folder -> root_folder -> some files in other words, it's i...
I have a directory called
folder
that looks like this:
folder
-> root_folder
-> some files
I want to zip this directory into zipped_dir
, I tried:
zip -r zipped_dir.zip folder/*
But this generates a ZIP that looks like this:
zipped_dir
-> folder
-> root_folder
-> some files
in other words, it's including the directory whose contents I want to zip. How can I exclude this parent directory from the ZIP, without moving anything?
IE I would like this end result:
zipped_dir
-> root_folder
-> some files
Juicy
(4125 rep)
Jan 30, 2015, 02:52 PM
• Last activity: Oct 31, 2024, 09:07 AM
1
votes
2
answers
194
views
Does unzip have a way to extract only a specific subfolder without creating that folder's parent dirs?
I want to extract a specific subfolder (and all its contents) from a .zip using the standard `unzip` tool. I am not interested in its parent directories and don't want them to be created. For example: `unzip /path/to/archive.zip "var/www/html/example.com/*" -d ~/website` I would like the `~/website`...
I want to extract a specific subfolder (and all its contents) from a .zip using the standard
unzip
tool. I am not interested in its parent directories and don't want them to be created.
For example:
unzip /path/to/archive.zip "var/www/html/example.com/*" -d ~/website
I would like the ~/website
folder to contain **only** the contents of example.com
from the archive but in reality, unzip recreates the folder's parent directory path so I actually end up with ~/website/var/www/html/example.com
.
I saw this question about extracting a specific folder but it did not answer this question.
Clanape
(13 rep)
Oct 23, 2024, 01:47 AM
• Last activity: Oct 23, 2024, 04:55 PM
35
votes
3
answers
35720
views
How to unzip and dd a disk image to an SD Card with a single command?
I am under the following restrictions: * I have a 1.0 GB `.zip` file on my computer which contains one file, a disk image of `raspbian`. When uncompressed, this file is 3.2 GB large and named `2015-11-21-raspbian-jessie.img`. * After having downloaded the zip file, I have just under 1.0 GB of storag...
I am under the following restrictions:
* I have a 1.0 GB
.zip
file on my computer which contains one file, a disk image of raspbian
. When uncompressed, this file is 3.2 GB large and named 2015-11-21-raspbian-jessie.img
.
* After having downloaded the zip file, I have just under 1.0 GB of storage space on my computer, not enough space to extract the image to my computer.
* This file needs to be uncompressed and written to an SD card using plain old dd
.
Is it possible for me to write the image to the SD card under these restrictions?
I know it's possible to pipe data through tar
and then pipe that data elsewhere, however, will this still work for the zip
file format, or does the entire archive need to be uncompressed before any files are accessible?
IQAndreas
(10635 rep)
Aug 29, 2015, 05:13 AM
• Last activity: Oct 9, 2024, 10:05 AM
3
votes
2
answers
1841
views
How to extract and delete contents of a zip archive simultaneously?
I want to download and extract a large zip archive (>180 GB) containing multiple small files of a single file format onto an SSD, but I don't have enough storage for both the zip archive and the extracted contents. I know that it would be possible to extract and delete individual files from an archi...
I want to download and extract a large zip archive (>180 GB) containing multiple small files of a single file format onto an SSD, but I don't have enough storage for both the zip archive and the extracted contents. I know that it would be possible to extract and delete individual files from an archive using the zip command as mentioned in the answers [here](https://unix.stackexchange.com/questions/14120/extract-only-a-specific-file-from-a-zipped-archive-to-a-given-directory) and [here](https://superuser.com/questions/600385/remove-single-file-from-zip-archive-on-linux) . I could also get the names of all the files in an archive using the
-l
command, store the results in an array as mentioned [here](https://www.baeldung.com/linux/reading-output-into-array) , filter out the unnecessary values using the method given [here](https://stackoverflow.com/questions/9762413/bash-array-leave-elements-containing-string) , and iterate over them in BASH as mentioned [here](https://www.cyberciti.biz/faq/bash-for-loop-array/) . So, the final logic would look something like this:
1. List the zip file's contents using -l
and store the filenames in a bash array, using regular expressions to match the single file extension present in the archive.
2. Iterate over the array of filenames and successively extract and delete individual files using the -j -d
and -d
commands.
How feasible is this method in terms of time required, logic complexity, and computational resources? I am worried about the efficiency of deleting and extracting single files, especially with such a large archive. If you have any feedback or comments about this approach, I would love to hear them. Thank you all in advance for your help.
**Edit 1:**
It seems this question has become a bit popular. Just in case anyone is interested, here is a BASH script following the logic I have outlined earlier, with batching for the extraction and deletion of files to reduce the number of operations. I have used DICOM files in this example but this would work for any other file type or for any files whose file names can be described by a regular expression. Here is the code:
#!/bin/bash
# Check if a zip file is provided as an argument
if [ -z "$1" ]; then
echo "Usage: $0 "
exit 1
fi
zipfile=$1
# List the contents of the zip file and store .dcm files in an array
mapfile -t dcm_files < <(unzip -Z1 "$zipfile" | grep '\.dcm$')
# Define the batch size
batch_size=10000
total_files=${#dcm_files[@]}
# Process files in batches
for ((i=0; i
The file would have to be saved with a name like .sh
with a .sh
extension and marked as executable. If the script and archive are in the same folder and the name of the archive is .zip
, the method to run the script would be ./inplace_extractor.sh archive.zip
. Feel free to adjust the batch size or the regular expression or account for any subfolders in your archive.
I tried it with my large archive and the performance was absolutely abysmal while the disk space rapidly shrunk, so I would still recommend going with the approaches suggested in other answers.
Kumaresh Balaji Sundararajan
(51 rep)
Nov 12, 2023, 12:24 PM
• Last activity: Oct 5, 2024, 11:37 AM
31
votes
5
answers
43955
views
Is there a way to convert a zip to a tar without extracting it to the filesystem?
Is there a way to convert a `zip` archive to a `tar` archive without extracting to a temporary directory first? (and without writing my own implementation of `tar` or `unzip`)
Is there a way to convert a
zip
archive to a tar
archive without extracting to a temporary directory first? (and without writing my own implementation of tar
or unzip
)
Stack Exchange Broke The Law
(2294 rep)
Jul 24, 2014, 05:11 AM
• Last activity: Oct 2, 2024, 11:54 AM
1
votes
0
answers
61
views
Error while unziping a file from a pendrive from windows on linux mint
I recently (today) installed Linux Mint for the first time. I made my backup on a pendrive, but when I tried unzipping the backup file I made, it gave me the following error: ``` End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-pa...
I recently (today) installed Linux Mint for the first time. I made my backup on a pendrive, but when I tried unzipping the backup file I made, it gave me the following error:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of backup.zip or
backup.zip.zip, and cannot find backup.zip.ZIP, period.
I tried seeing what I could do on the forums. Apparently, it means my file is corrupted. The thing is I can normally access the files from my pendrive without taking it to my PC.
OBS.: A detail I judged to be useful is that when I installed Linux, I forgot to take the USB stick out. I don't think this is the cause of the problem, but who knows.
UPDATE: I tried taking the file on a windows computer, it was not a .zip, it was a .rar, sorry for the inconvenience.
octo
(11 rep)
Oct 1, 2024, 01:22 AM
• Last activity: Oct 1, 2024, 01:23 PM
Showing page 1 of 20 total questions