I have a file with the lines like below.

title1:A1
title2:A2
title3:A3
title4:A4
title5:A5

title1:B1
title2:B2
title3:B3
title4:B4
title5:B5

title1:C1
title2:C2
title3:C3
title4:C4
title5:C5

title1:D1
title2:D2
title3:D3
title4:D4
title5:D5

How can I achieve this?

title1    title2     title3    title4
A1         A2         A3         A4
B1         B2         B3         B4
C1         C2         C3         C4
D1         D2         D3         D4

I want to apply grep on particular column in column command. myfile.txt 3,ST,ST01,3,3,856 3,ST,ST02,4,9,0234 6,N1,N101,2,3,ST 6,N1,N102,1,60,Comcast 6,N1,N103,1,2,92 My Command: column -s, -t < myfile.txt | grep -w "ST" Here I want to only grep the pattern ST in 2nd column. How to do this ? **Expected Result:** 3 ST ST01 3 3 856 3 ST ST02 4 9 0234

OK, I will try and explain what I need to do as best as possible. Basically I have two CSV files, as per the examples below: File 1: Column 1, Column 2 abc , 123 def , 234 adf , 567 File 2 Column 1, Column 2 abc , 123 def , 234 adf , 578 I need to write either a shell script or simple command that will do the following: 1. Sort both files by column 1 2. Row by row, do the following: - Using column 1 in file 1, search for this value in column 1 in file 2. 1. if found, compare the value in column 2 in file 1 against the value in column 2 of file 2 1. if it matches, write column 1, column 2 and "Validated" in column 3 to a separate file 1. if it does not match, write column 1, column 2 and "Failed" to a separate file This results in two output files: the first with everything that was found in column 1 and column 2 matches, and a second file containing either column 1 lookups that failed or where column 1 was found, where column 2 did not match, so, essentially, using column 1 as the key to check column 2.

I am looking for a command-line tool to calculate the sum of the values in a specified column of a CSV file. (**Update**: The CSV file might have quoted fields, so a simple solution just to break on a delimiter (',') does not work.) Given the following sample CSV file:

description A,description B,data 1, data 2
fruit,"banana,apple",3,17
veggie,cauliflower,7,18
animal,"fish,meat",9,22

I want to build the sum, for example, over the column data 1 with the result **19**. I have tried to use [csvkit] for this but didn't get very far. Are there other command-lien tools specialised in this CSV operation?

Editing /etc/fstab with my text editor makes all the spacing go out of alignment. I could go through the file and insert/delete spaces to line everything up, but I'm looking for a more automated solution. Ideally, there would be an online javascript page that I could put my fstab into, and have it "pretty" align everything. Then I could copy/paste the final result back into the file. Is there a similar or better solution to this available? EDIT: I use Linux on my desktop, and I'm not seeking to join the religion of vi or emacs just to edit my fstab. Emacs might be a better solution for some people, but for some other people it's not a better solution. EDIT2: Here is an example snippet of my fstab using tabs. The columns are not in alignment. proc /proc proc nodev,noexec,nosuid 0 0 /dev/disk/by-label/Linux / ext4 errors=remount-ro 0 1 /dev/disk/by-label/Home /home ext4 defaults 0 0 I want it to be automatically formatted with spaces and look more like the following. proc /proc proc nodev,noexec,nosuid 0 0 /dev/disk/by-label/Linux / ext4 errors=remount-ro 0 1 /dev/disk/by-label/Home /home ext4 defaults 0 0

I need some help to implement awk command in a bash script for implementing below logic. Tried sort/uniq combination, but no luck. I have a long list with time stamp and some number in front of it like below. [2020-09-15 09:03:21.835335] 1021 [2020-09-15 09:03:21.935335] 1021 [2020-09-15 09:03:22.835335] 1022 [2020-09-15 09:03:25.835335] 1022 [2020-09-15 09:04:21.835335] 1023 [2020-09-15 09:05:21.835335] 1023 [2020-09-15 09:04:22.835335] 1023 [2020-09-15 09:05:23.835335] 1023 In the final outcome, I want to process the list with respect to column 2 above with numbers. In the final processed data, I wanted to have count of unique numbers (even if the occurrence is just once) from column 2 and add it in a new column in front of number. What I really needed was to have time stamp from last occurrence of duplicate/unique entries from number column. [2020-09-15 09:03:21.935335] 1021 2 [2020-09-15 09:03:25.835335] 1022 2 [2020-09-15 09:05:23.835335] 1023 4

I'm dealing with a series of bed files, which look like this:

chr1 100 110 0.5
chr1 150 175 0.2
chr1 200 300 1.5

With the columns being chromosome, start, end, score. I have multiple different files with different scores in each one, and I'd like to combine them like this:

> cat a.bed
chr1 100 110 0.5
chr1 150 175 0.2
chr1 200 300 1.5

> cat b.bed
chr1 100 110 0.4
chr1 150 175 0.7
chr1 200 300 0.9

> cat c.bed
chr1 100 110 1.5
chr1 150 175 1.2
chr1 200 300 0.1

> cat combined.bed
chr1 100 110 0.5 0.4 1.5
chr1 150 175 0.2 0.7 1.2
chr1 200 300 1.5 0.9 0.1

All the score columns (last column of the file) are added to a single file. I found [this answer](https://unix.stackexchange.com/a/167290/387150) , which can combine a column from one additional file into an existing file, but I would like a command which can add a *variable* number of columns together. So if I have 10 bed files to combine, I'd like a command that can process them all together and create a single file with 10 score columns. Each file should have the same number of lines, and each entry should have the same coordinates in all the files, so there should be no conflicts there. However there can be a lot of entries in each of the files (100K or more generally), so I'd like to avoid processing each one multiple times. Is there a way to handle this cleanly? This will be in a script so no need to be a one liner.

I have a program that I'd like to run, deluding it to believe that my terminal has less columns than it does, because that enables its native truncation, which I need. I tried resize from xterm as suggested [on askubuntu](https://askubuntu.com/questions/852105/is-there-a-terminal-command-that-changes-the-terminal-window-dimensions) , however, that only works properly within xterm and can hardly be scripted. I need something that I can use within a script to get the output of a [taskwarrior](https://taskwarrior.org/) command. Exporting a different value for COLUMNS doesn't seem to change anything unfortunately.

I can get column to use tab as a field separator:

$ printf "a\tb\tc\n" | column -s $'\t' -t
a  b  c

But using null separator doesn't seem to work:

$ printf "a\0b\0c\n" | column -s $'\0' -t
column: line too long

To confirm, the null separators are really there:

$ printf "a\0b\0c\n" | xxd
00000000: 6100 6200 630a                           a.b.c.

What could be going wrong here?

I have 2 files, one having 2 columns, another having 1 column. The second file is sorted using sort -u. Now the task is I need to join this column with the first column of the first file, which is not sorted. So what will be the syntax? Will join -j 1 file2.txt sort -s -n -k 1 file1.txt work? The output I want is actually the 2nd column of file 2 after joining and the unique entries in it. File 2 ------ 1 2 3 File 1 ------ 2 500 1 5000 1 300 3 3000 3 300 4 450 Output ------ 5000 300 500 3000

I have a list of strings:

python2-distutils-extra|==>|python2-distutils-extra|2.39-5|Enhancements to the Python build system
python2-fuse|==>|python2-fuse|1.0.5-1|This is a Python 2 interface to libfuse (https://github.com/libfuse/libfuse) , a simple interface for userspace programs to export a virtual filesystem to the Linux kernel                                               
python2-gobject2|==>|python2-gobject2|2.28.7-7|Legacy Python 2 bindings for GObject                                                                                                                                                                                     
python2-numpy|==>|python2-numpy|1.16.6-3|Scientific tools for Python 2

I want to get:

python2-distutils-extra ==> python2-distutils-extra 2.39-5
    Enhancements to the Python build system
python2-fuse            ==> python2-fuse 1.0.5-1
    This is a Python 2 interface to libfuse (https://github.com/libfuse/libfuse) , a simple interface for userspace programs to export a virtual filesystem to the Linux kernel                                               
python2-gobject2        ==> python2-gobject2 2.28.7-7
    Legacy Python 2 bindings for GObject                                                                                                                                                                                     
python2-numpy           ==> python2-numpy 1.16.6-3
    Scientific tools for Python 2

edit: aligned to the length of the first column means '==>' is always at the same position (just like in above code). I've already tried to get it with column, but doesn't matter which options of column I use the result is:

python2-distutils-extra                                                                                                                                                         ==>  python2-distutils-extra    2.39-5        
    Enhancements to the Python build system                                                                                                                                                                                  
python2-fuse                                                                                                                                                                    ==>  python2-fuse               1.0.5-1       
    This is a Python 2 interface to libfuse (https://github.com/libfuse/libfuse) , a simple interface for userspace programs to export a virtual filesystem to the Linux kernel                                               
python2-gobject2                                                                                                                                                                ==>  python2-gobject2           2.28.7-7      
    Legacy Python 2 bindings for GObject                                                                                                                                                                                     
python2-numpy                                                                                                                                                                   ==>  python2-numpy              1.16.6-3      
    Scientific tools for Python 2

Please, help...

I have this input, which is displayed in columns. I would like to get the second last column with the numbers of this sample: [ 3] 1.0- 2.0 sec 1.00 MBytes 8.39 Mbits/sec [ 3] 2.0- 3.0 sec 768 KBytes 6.29 Mbits/sec [ 3] 3.0- 4.0 sec 512 KBytes 4.19 Mbits/sec [ 3] 4.0- 5.0 sec 256 KBytes 2.10 Mbits/sec ... If I use cut -d\ -f 13 I get Mbits/sec 6.29 4.19 2.10 because sometimes there are additional spaces in between.

I've got a text file containing two columns, like so: 26 0.000342231 27 0.000342231 28 0.000684463 29 0.00136893 30 0.00102669 31 0.00308008 32 0.00308008 33 0.00444901 34 0.00718686 35 0.00718686 36 0.0109514 37 0.0123203 ... I'd like to loop over the text file and store each columns value to an associative array - something like a dictionary. If possible I'd like to keep data types of values in each column (int and float). For a calculation I then need to sum up 2nd column's values for a particular interval till the end of the file, e.g. "sum up 29's (1st column) associated value (0.00136893) down to the last associated value" What's the best way to do so? Bash and python solutions are most welcome!

I'm looking for a pipeable one-liner to reorder a large number of columns (where manually entering the column numbers in e.g. an awk command such as awk '{print $3,$2,$1}' is not feasible). The order could be given by a sorting scheme (alphabetical, numerical - so like 'sort' but acting on columns rather than rows.) or be arbitrarily given in a text file.

I'm colorizing the header of a table formatted with column -ts $'\t' Works well without color codes, but when I add color codes to the first line column doesn't properly align the output. Without colored output it works **as expected**: printf "1\t2\t3\nasdasdasdasdasdasdasd\tqwe\tqweqwe\n" | column -ts $'\t' But when adding color on the first line column **doesn't align** the text of the colored row: printf "\e[7m1\t2\t3\e[0m\nasdasdasdasdasdasdasd\tqwe\tqweqwe\n" | column -ts $'\t' Observed this behaviour both on Ubuntu Linux and Mac OS X.

I'm working on a commandline that retrieve some data (*curl*), extract the relevant fields (*awk*) and format it (*column*). It work nice, although it's very ugly (but all my script begin from a *"too long and ugly"* one-liner) but when I try to have some colour column goes wrong. This is the plain (reduced) version that **work**: curl "http://webservices.rm.ingv.it/fdsnws/event/1/query?lat=42.35&lon=13.4&maxradius=5.0&starttime=2016-01-01T00:00:00&endtime=2016-12-31T23:59:59&minmag=5&format=text&orderby=time-asc " 2>/dev/null \ | awk 'BEGIN { FS= "|"; OFS= "|" } {print $1, $2, $5, $10, $11, $13}' \ | column -t -s '|' Now, I want to underline some fields, and then add some **ANSI escape code** in awk: curl "http://webservices.rm.ingv.it/fdsnws/event/1/query?lat=42.35&lon=13.4&maxradius=5.0&starttime=2016-01-01T00:00:00&endtime=2016-12-31T23:59:59&minmag=5&format=text&orderby=time-asc " 2>/dev/null \ | awk 'BEGIN { FS= "|" ; OFS= "|" } \ $13~/Rieti/||/Perugia/ {$13="\033[1;31m"$13"\033[0m"} \ $11~/[0-9]+/ && $11 > 5.8 {$11="\0331;33m"$11"\033[0m"} {print $1, $2, $5, $10, $11, $13 }' \ | column -t -s '|' Now, the alignment is wrong (see the picture). Why? And how can I fix it? ## UPDATE ## I already saw the question [Issue with column command and color escape codes but does not solve my problem because his answers are applied and work in the case of a fully colored line. In my case I can't apply or adapt the answers (or I'm not able to) because: 1. The problem is circumscribed to the case where the column $11 is colored, regardless of the subsequent column. 2. I can't see a good or elegant way to add color code *after* column. If I send column's output to awk for the test I don't know how to instruct awk to separate the fields correctly (if the fields were separated by more space I could use a regex but in some cases the separation is by a single space, and awk would not know how to recognize spaces between words and spaces as Fields Separators). The only thing I can see is that if I move the reset color code from the assignment to the print block the **first** row was better spaced, like the plain output version (see below, the \033[0m underlined in second commandline): ### *Then, how can we fix it? There is another way, more elegant, to colorize as I did?* *(I know, I can do it better with some lines of perl, but I'm curious about this problem)*

I have input containing ANSI color codes that I'd like to tablify. I want the output to remain colored, so the tablified output should maintain the ANSI color codes. Hence, naively stripping them away does not meet my requirements. For example, for this input, \033[0;32;1mgreen_apple\033[0m 1 100 orange 20 19 pineapple 1000 87 avocado 4 30 The output I expect would be similar to green_apple 1 100 orange 20 19 pineapple 1000 87 avocado 4 30 In the above output, "green_apple" should be colored according to the color codes from the input i.e. green. I'd like to know how this can be accomplished. I've tried column, but it doesn't handle ANSI codes. The output of echo '\033[0;32;1mgreen_apple\033[0m 1 100 orange 20 19 pineapple 1000 87 avocado 4 30' | column -t is unfortunately green_apple 1 100 orange 20 19 pineapple 1000 87 avocado 4 30 Notice the non-tablification.

What is the purpose of the column with 0 after the numeric timestamp in .zsh_history? : 1568128379:0;cp -a ~/.zshrc.pre-oh-my-zsh ~/.zshrc : 1568128381:0;exit Is it part of the timestamp or does it serve a different purpose altogether?

Can we get specific column in a top command, for eg I'm interested in only the memory utilization and CPU usage column. How do I reduce the displayed columns of the top command to only the above two columns?

My script: date echo -e "${YELLOW}Network check${NC}\n\n" while read hostname do ping -c 1 "$hostname" > /dev/null 2>&1 && echo -e "Network $hostname : ${GREEN}Online${NC}" || echo -e "${GRAY}Network $hostname${NC} : ${RED}Offline${NC}" done < list.txt sleep 30 clear done Is outputting info like this: Network 10.x.xx.xxx : Online Network 10.x.xx.xxx : Offline Network 10.x.xx.xxx : Offline Network 10.x.xx.xxx : Offline Network 10.x.xx.x : Online Network 139.xxx.x.x : Online Network 208.xx.xxx.xxx : Online Network 193.xxx.xxx.x : Online which I'd like to clean up to get something like this: Network 10.x.xx.xxx : Online Network 10.x.xx.xxx : Offline Network 10.x.xx.xxx : Offline Network 10.x.xx.x : Online Network 139.xxx.x.x : Online Network 208.xx.xxx.xxx : Online Network 193.xxx.xxx.x : Online Network 193.xxx.xxx.xxx : Offline