I'm using a bash script that needs to read the JSON output and parse a value from different JSON variables or strings. Here's the sample output. It needs to read the value next to the Content or from any other variable. Such as, Lookup Content and be able to print Value1. Lookup DeviceType and be able to print Value4. Sample Input:

{"Content":"Value1","CreationMethod":"Value2","database":"Value3","DeviceType":"Value4"}

I tried the combination of sed and awk:

sed 's/["]/ /g' | awk '{print $4}'

... but only if the position of Content remains the same in the output. Otherwise, in the different JSON output, the positioning of Content changes that puts the value out of scope thus, awk '{print $4}' picks up the wrong value.

The default functionality of jq is to send each object from an array one at a time, though the join operator can merge those values. My problem is in trying to print all the values from multiple arrays at once. Taking this example:

{
    "key1": {
        "list1": [
            "val1",
            "val2",
            "val3"
        ]
    },
    "key2": {
        "list1": [
            "val4",
            "val5"
        ]
    },
    "key3": {
        "list1": [
            "val6"
        ]
    }
}

I'd like to print:

val1 val2 val3 val4 val5 val6

And so far have this:

jq -r 'to_entries[] | { list: .value.list1 } | .list | join(" ")' test.json

*(More verbose than necessary to help reviewers.)* which gives:

val1 val2 val3
val4 val5
val6

Is there a way to gather all the values together in one command?

I want to format SQLite output in JSON format from the command line. Currently, I have CSV output that looks like this: label1,value1 label2,value2 label3,value3 ... Now I'd like to have it formatted like this: {'label1' : 'value1', 'label2': 'value2', ... } Thanks!

I want to get JSON with curl command, so with below command I am getting output: curl -GET http://localhost:9200/oldindex/_mapping?pretty { "gl-events_1" : { "mappings" : { "message" : { "dynamic" : "false", "dynamic_templates" : [ { "fields" : { "path_match" : "fields.*", "mapping" : { "doc_values" : true, "index" : true, "type" : "keyword" } } } ], "properties" : { "alert" : { "type" : "boolean" }, "event_definition_id" : { "type" : "keyword" }, "event_definition_type" : { "type" : "keyword" }, "fields" : { "type" : "object", "dynamic" : "true" }, "id" : { "type" : "keyword" }, "key" : { "type" : "keyword" }, "key_tuple" : { "type" : "keyword" }, "message" : { "type" : "text", "norms" : false, "fields" : { "keyword" : { "type" : "keyword" } }, "analyzer" : "standard" }, "origin_context" : { "type" : "keyword" }, "priority" : { "type" : "long" }, "source" : { "type" : "keyword" }, "source_streams" : { "type" : "keyword" }, "streams" : { "type" : "keyword" }, "timerange_end" : { "type" : "date", "format" : "yyyy-MM-dd HH:mm:ss.SSS" }, "timerange_start" : { "type" : "date", "format" : "yyyy-MM-dd HH:mm:ss.SSS" }, "timestamp" : { "type" : "date", "format" : "yyyy-MM-dd HH:mm:ss.SSS" }, "timestamp_processing" : { "type" : "date", "format" : "yyyy-MM-dd HH:mm:ss.SSS" }, "triggered_jobs" : { "type" : "keyword" } } } } } } Now i want to store this output as json file so I copied it in file and gave extension as .json But when i try to put with curl I am getting below error curl -X PUT http://localhost:9200/new_good -H 'Content-Type: application/json' -d sampl.json {"error":{"root_cause":[{"type":"not_x_content_exception","reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}],"type":"not_x_content_exception","reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"},"status":500} But when i run below command with same json format directly it works, curl -X PUT \ http://localhost:9200/new_good \ -H 'Content-Type: application/json' \ -d '{"mappings" : { "message" : { "dynamic_templates" : [ { "internal_fields" : { "match" : "gl2_*", "match_mapping_type" : "string", "mapping" : { "type" : "keyword" } } }, { "store_generic" : { "match_mapping_type" : "string", "mapping" : { "type" : "keyword" } } } ], "properties" : { "LoggerName" : { "type" : "keyword" }, "MessageParam0" : { "type" : "keyword" }, "MessageParam1" : { "type" : "long" }, "MessageParam2" : { "type" : "keyword" }, "MessageParam3" : { "type" : "keyword" }, "MessageParam4" : { "type" : "keyword" }, "MessageParam5" : { "type" : "keyword" }, "MessageParam6" : { "type" : "keyword" }, "MessageParam7" : { "type" : "keyword" }, "MessageParam8" : { "type" : "keyword" }, "Severity" : { "type" : "keyword" }, "SourceClassName" : { "type" : "keyword" }, "SourceMethodName" : { "type" : "keyword" }, "SourceSimpleClassName" : { "type" : "keyword" }, "StackTrace" : { "type" : "keyword" }, "Thread" : { "type" : "keyword" }, "Time" : { "type" : "keyword" }, "facility" : { "type" : "keyword" }, "full_message" : { "type" : "text", "analyzer" : "standard" }, "gl2_accounted_message_size" : { "type" : "long" }, "gl2_message_id" : { "type" : "keyword" }, "gl2_processing_timestamp" : { "type" : "date", "format" : "yyyy-MM-dd HH:mm:ss.SSS" }, "gl2_receive_timestamp" : { "type" : "date", "format" : "yyyy-MM-dd HH:mm:ss.SSS" }, "gl2_remote_ip" : { "type" : "keyword" }, "gl2_remote_port" : { "type" : "long" }, "gl2_source_input" : { "type" : "keyword" }, "gl2_source_node" : { "type" : "keyword" }, "level" : { "type" : "long" }, "message" : { "type" : "text", "analyzer" : "standard" }, "source" : { "type" : "text", "analyzer" : "analyzer_keyword", "fielddata" : true }, "streams" : { "type" : "keyword" }, "timestamp" : { "type" : "date", "format" : "yyyy-MM-dd HH:mm:ss.SSS" } } } } } }' What I want is store curl GET command output as valid json which I can use in curl PUT, curl get > some.json curl put -d some.json I am new to this and i tried several options with jq as well but that also didn't workd for me. Please guide me here. Regards SAM

We have the following example file ( very long file , this is short example ) "request_status" : "FAILED" { "href" : "http://localhost:8080/api/v1/clusters/sys41/requests/333", "Requests" : { "cluster_name" : "sys41", "id" : 333, "request_status" : "COMPLETED" } }, { "href" : "http://localhost:8080/api/v1/clusters/sys41/requests/334", "Requests" : { "cluster_name" : "sys41", "id" : 334, "request_status" : "FAILED" } }, { "href" : "http://localhost:8080/api/v1/clusters/sys41/requests/335", "Requests" : { "cluster_name" : "sys41", "id" : 335, "request_status" : "FAILED" } }, { "href" : "http://localhost:8080/api/v1/clusters/sys41/requests/336", "Requests" : { "cluster_name" : "sys41", "id" : 336, "request_status" : "COMPLETED" } } how to print the line after the line that matches "id" : $num e.g. for num=335 how to get the line after "id" : $num Expected output "request_status" : "FAILED"

Let's say I have a file as below { "fruit": "Apple", } I want to remove the comma at the end of the line, if and only if the next line contains "}". So, the output will be : { "fruit": "Apple" } However, if the file is as below. I do not want to do any change. Since the ,s are not followed by a } { "fruit": "Apple", "size": "Large", "color": "Red" } Anything with sed would be fantastic.

I have a newline delimited JSON file with entries like this: {"id":"eprints.ulster.ac.uk/view/year/2015.html","title":"Items where Year is 2015 - Ulster Institutional Repository","url":"eprints.ulster.ac.uk/view/year/2015.html"} {"id":"eprints.ulster.ac.uk/view/year/2016.html","title":"Items where Year is 2016 - Ulster Institutional Repository","url":"eprints.ulster.ac.uk/view/year/2016.html"} {"id":"eprints.ulster.ac.uk/view/year/2017.html","title":"Items where Year is 2017 - Ulster Institutional Repository","url":"eprints.ulster.ac.uk/view/year/2017.html"} {"id":"eprints.ulster.ac.uk/10386/","title":"Structural performance of rotationally restrained steel columns in fire - Ulster Institutional Repos","url":"eprints.ulster.ac.uk/10386/"} {"id":"eprints.ulster.ac.uk/10387/","title":"Determining the Effective Length of Fixed End Steel Columns in Fire - Ulster Institutional Repositor","url":"eprints.ulster.ac.uk/10387/"} I only want blocks where the .id does not begin with "eprints.ulster.ac.uk/view/" So if the script was run on the above code snippet, the first 3 blocks would be deleted and the only blocks remaining would be: {"id":"eprints.ulster.ac.uk/10386/","title":"Structural performance of rotationally restrained steel columns in fire - Ulster Institutional Repos","url":"eprints.ulster.ac.uk/10386/"} {"id":"eprints.ulster.ac.uk/10387/","title":"Determining the Effective Length of Fixed End Steel Columns in Fire - Ulster Institutional Repositor","url":"eprints.ulster.ac.uk/10387/"} Can anybody help write an awk script to do this?

When parsing *json*, the command-line tool jshon converts numbers to scientific notation, and sometimes tries to round them . To avoid these problems, I want jshon to consider these numbers as strings. For that, I have found that I need to place quotes around all numbers in the json file. After some unsuccessful googling, I have tried writing a sed command to quote the numbers, but I found it pretty unsafe, and have run into lots of issues already: sed -r 's/(" ?[:,] ?)"?([0-9]+(\.[0-9]+)?)"?([,}]|$)/\1"\2"\4/g' $file I would like to know if there is some stable **parser** that can give me the desired result. I'm not including an example json file in the question, because I need this code for some little risky operations, and I will be parsing json from random websites.

Consider a contrived example using a JSON object such as this, where I want to extract the related id, firstname, and lastname fields for each of many array objects into shell variables for further (non-JSON) processing.

-json
{
  "customers": [
    {
      "id": 1234,
      "firstname": "John",
      "lastname": "Smith",
      "other": "fields",
      "are": "present",
      "here": "etc."
    },
    {
      "id": 2468,
      "firstname": "Janet",
      "lastname": "Green",
      "other": "values",
      "are": "probably",
      "here": "maybe"
    }
  ]
}

For simple data I can use this,

-bash
jq -r '.customers[] | (.id + " " + .firstname + " " + .lastname)' 

Output

-none
1234 -- John -- Smith
2468 -- Janet -- Green

but of course this will fail with double-barrelled firstname values such as Anne Marie. Changing the separator to another character such as # feels more like a fudge than a solution but could be acceptable. For more complex situations I might pick out the list of id values and then trade speed for accuracy by going back to extract the corresponding firstname and lastname elements. Something like this:

-bash
jq -r '.customers[].id' 

Output

-none
1234 -- John -- Smith
2468 -- Janet -- Green

However, neither of these is both correct and efficient. While I'm not going to be running the real code at a high frequency, I'd like to understand if there is a more appropriate way of getting multiple data elements safely and efficiently out of a JSON object structure and into shell variables?