Remove trailing commas from invalid json (to make it valid)

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP

up vote
2
down vote

favorite

Let’s say I have a file as below

{
    "fruit": "Apple",
}

I want to remove the comma at the end of the line, if and only if the next line contains “}”. So, the output will be :

{
    "fruit": "Apple"
}

However, if the file is as below. I do not want to do any change. Since the ,s are not followed by a }

{
    "fruit": "Apple",
    "size": "Large",
    "color": "Red"
}

Anything with sed would be fantastic.

share|improve this question

  • Sorry but I don’t get what fruit has to do with it. Are you saying that if you have the line "colour": "Red", followed by }, then to leave it alone?
    – ctrl-alt-delor
    Nov 29 at 19:52

  • Thanks for the question. All I am saying is, if the pattern fruit occurs and the immediate next line does not contain “}” (close parenthesis) I do not want to remove the comma at the end of the line where the pattern “fruit” exists. However, if “}” is present in the line immediately following the line where the pattern “fruit” exists, I want to remove the comma at the end of that line (where the pattern fruit is present). Let me know if you still have question.
    – Somy
    Nov 29 at 19:55

  • 1

    This is an x-y problem. I’m fairly certain you are not concerned with the fruit value at all, you are concerned with making valid json objects.
    – Jesse_b
    Nov 29 at 19:56

  • Same question. You just repeated your self. So I am thinking that you don’t want valid JSON, if the line with color has a comma at the end, even though the next line is a }.
    – ctrl-alt-delor
    Nov 29 at 19:57

  • I am just concerned with the”fruit” pattern. I already have the code in place to make the json valid except for the missing piece I just posted. I would say lets not think about json at all. Just think of the problem piece in general and how we can solve in unix.
    – Somy
    Nov 29 at 20:01

up vote
2
down vote

favorite

Let’s say I have a file as below

{
    "fruit": "Apple",
}

I want to remove the comma at the end of the line, if and only if the next line contains “}”. So, the output will be :

{
    "fruit": "Apple"
}

However, if the file is as below. I do not want to do any change. Since the ,s are not followed by a }

{
    "fruit": "Apple",
    "size": "Large",
    "color": "Red"
}

Anything with sed would be fantastic.

share|improve this question

  • Sorry but I don’t get what fruit has to do with it. Are you saying that if you have the line "colour": "Red", followed by }, then to leave it alone?
    – ctrl-alt-delor
    Nov 29 at 19:52

  • Thanks for the question. All I am saying is, if the pattern fruit occurs and the immediate next line does not contain “}” (close parenthesis) I do not want to remove the comma at the end of the line where the pattern “fruit” exists. However, if “}” is present in the line immediately following the line where the pattern “fruit” exists, I want to remove the comma at the end of that line (where the pattern fruit is present). Let me know if you still have question.
    – Somy
    Nov 29 at 19:55

  • 1

    This is an x-y problem. I’m fairly certain you are not concerned with the fruit value at all, you are concerned with making valid json objects.
    – Jesse_b
    Nov 29 at 19:56

  • Same question. You just repeated your self. So I am thinking that you don’t want valid JSON, if the line with color has a comma at the end, even though the next line is a }.
    – ctrl-alt-delor
    Nov 29 at 19:57

  • I am just concerned with the”fruit” pattern. I already have the code in place to make the json valid except for the missing piece I just posted. I would say lets not think about json at all. Just think of the problem piece in general and how we can solve in unix.
    – Somy
    Nov 29 at 20:01

up vote
2
down vote

favorite

up vote
2
down vote

favorite

Let’s say I have a file as below

{
    "fruit": "Apple",
}

I want to remove the comma at the end of the line, if and only if the next line contains “}”. So, the output will be :

{
    "fruit": "Apple"
}

However, if the file is as below. I do not want to do any change. Since the ,s are not followed by a }

{
    "fruit": "Apple",
    "size": "Large",
    "color": "Red"
}

Anything with sed would be fantastic.

share|improve this question

Let’s say I have a file as below

{
    "fruit": "Apple",
}

I want to remove the comma at the end of the line, if and only if the next line contains “}”. So, the output will be :

{
    "fruit": "Apple"
}

However, if the file is as below. I do not want to do any change. Since the ,s are not followed by a }

{
    "fruit": "Apple",
    "size": "Large",
    "color": "Red"
}

Anything with sed would be fantastic.

text-processing json

share|improve this question

share|improve this question

share|improve this question

share|improve this question

edited Nov 29 at 20:23

Rui F Ribeiro

38.4k1479128

38.4k1479128

asked Nov 29 at 19:45

Somy

215

215

  • Sorry but I don’t get what fruit has to do with it. Are you saying that if you have the line "colour": "Red", followed by }, then to leave it alone?
    – ctrl-alt-delor
    Nov 29 at 19:52

  • Thanks for the question. All I am saying is, if the pattern fruit occurs and the immediate next line does not contain “}” (close parenthesis) I do not want to remove the comma at the end of the line where the pattern “fruit” exists. However, if “}” is present in the line immediately following the line where the pattern “fruit” exists, I want to remove the comma at the end of that line (where the pattern fruit is present). Let me know if you still have question.
    – Somy
    Nov 29 at 19:55

  • 1

    This is an x-y problem. I’m fairly certain you are not concerned with the fruit value at all, you are concerned with making valid json objects.
    – Jesse_b
    Nov 29 at 19:56

  • Same question. You just repeated your self. So I am thinking that you don’t want valid JSON, if the line with color has a comma at the end, even though the next line is a }.
    – ctrl-alt-delor
    Nov 29 at 19:57

  • I am just concerned with the”fruit” pattern. I already have the code in place to make the json valid except for the missing piece I just posted. I would say lets not think about json at all. Just think of the problem piece in general and how we can solve in unix.
    – Somy
    Nov 29 at 20:01

  • Sorry but I don’t get what fruit has to do with it. Are you saying that if you have the line "colour": "Red", followed by }, then to leave it alone?
    – ctrl-alt-delor
    Nov 29 at 19:52

  • Thanks for the question. All I am saying is, if the pattern fruit occurs and the immediate next line does not contain “}” (close parenthesis) I do not want to remove the comma at the end of the line where the pattern “fruit” exists. However, if “}” is present in the line immediately following the line where the pattern “fruit” exists, I want to remove the comma at the end of that line (where the pattern fruit is present). Let me know if you still have question.
    – Somy
    Nov 29 at 19:55

  • 1

    This is an x-y problem. I’m fairly certain you are not concerned with the fruit value at all, you are concerned with making valid json objects.
    – Jesse_b
    Nov 29 at 19:56

  • Same question. You just repeated your self. So I am thinking that you don’t want valid JSON, if the line with color has a comma at the end, even though the next line is a }.
    – ctrl-alt-delor
    Nov 29 at 19:57

  • I am just concerned with the”fruit” pattern. I already have the code in place to make the json valid except for the missing piece I just posted. I would say lets not think about json at all. Just think of the problem piece in general and how we can solve in unix.
    – Somy
    Nov 29 at 20:01

Sorry but I don’t get what fruit has to do with it. Are you saying that if you have the line "colour": "Red", followed by }, then to leave it alone?
– ctrl-alt-delor
Nov 29 at 19:52

Sorry but I don’t get what fruit has to do with it. Are you saying that if you have the line "colour": "Red", followed by }, then to leave it alone?
– ctrl-alt-delor
Nov 29 at 19:52

Thanks for the question. All I am saying is, if the pattern fruit occurs and the immediate next line does not contain “}” (close parenthesis) I do not want to remove the comma at the end of the line where the pattern “fruit” exists. However, if “}” is present in the line immediately following the line where the pattern “fruit” exists, I want to remove the comma at the end of that line (where the pattern fruit is present). Let me know if you still have question.
– Somy
Nov 29 at 19:55

Thanks for the question. All I am saying is, if the pattern fruit occurs and the immediate next line does not contain “}” (close parenthesis) I do not want to remove the comma at the end of the line where the pattern “fruit” exists. However, if “}” is present in the line immediately following the line where the pattern “fruit” exists, I want to remove the comma at the end of that line (where the pattern fruit is present). Let me know if you still have question.
– Somy
Nov 29 at 19:55

1

1

This is an x-y problem. I’m fairly certain you are not concerned with the fruit value at all, you are concerned with making valid json objects.
– Jesse_b
Nov 29 at 19:56

This is an x-y problem. I’m fairly certain you are not concerned with the fruit value at all, you are concerned with making valid json objects.
– Jesse_b
Nov 29 at 19:56

Same question. You just repeated your self. So I am thinking that you don’t want valid JSON, if the line with color has a comma at the end, even though the next line is a }.
– ctrl-alt-delor
Nov 29 at 19:57

Same question. You just repeated your self. So I am thinking that you don’t want valid JSON, if the line with color has a comma at the end, even though the next line is a }.
– ctrl-alt-delor
Nov 29 at 19:57

I am just concerned with the”fruit” pattern. I already have the code in place to make the json valid except for the missing piece I just posted. I would say lets not think about json at all. Just think of the problem piece in general and how we can solve in unix.
– Somy
Nov 29 at 20:01

I am just concerned with the”fruit” pattern. I already have the code in place to make the json valid except for the missing piece I just posted. I would say lets not think about json at all. Just think of the problem piece in general and how we can solve in unix.
– Somy
Nov 29 at 20:01

2 Answers
2

active

oldest

votes

up vote
2
down vote

accepted

sed -i.bak ':begin;$!N;s/,n}/n}/g;tbegin;P;D' FILE

sed -i.bak = creates a backup of the original file, then applies changes to the file

':begin;$!N;s/,n}/n}/g;tbegin;P;D' = anything ending with , followed by new line and }. Remove the , on the previous line

FILE = the file you want to make the change to

share|improve this answer

  • 1

    Fantastic. Works like Magic! Thanks a lot.
    – Somy
    Nov 29 at 21:21

up vote
4
down vote

What makes this a non-trivial problem is that the JSON format does not care about whitespace that does not occur inside keys or data. Therefore,

{ "key": "data" }

is the same as

{ "key":
"data"
}

If you add the possibility of a “broken” JSON file, such as

{ "key":
"data", }

it becomes really difficult to properly parse the document with anything other than a JSON parser that knows how to relax the restrictions of the JSON format when parsing the data.

The Perl JSON module can do that, and also pretty-print the result:

$ cat file.json
{
    "fruit": "Apple",
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple"
}

Here, we read in the whole text document into the array @text. We then decode this while relaxing the parsing (this enables the JSON document to have commas before } and ] and also to include # comments). We then immediately encode the resulting Perl data structure into JSON again and print it.

Another example:

$ cat file.json
{
    "fruit": "Apple",   # a comment
    "stuff": [1, 2, 3,],
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple",
   "stuff" : [
      1,
      2,
      3
   ]
}

Without pretty printing:

$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}))' file.json
{"fruit":"Apple","stuff":[1,2,3]}

(no newline at the end of the output)

For really large documents, you would want to use the module’s incremental parsing capability and write a proper script for the conversion.

share|improve this answer

  • Thanks for sharing the solution. I went ahead with the sed solution so did’nt try the one you shared. But thanks anyway.
    – Somy
    Nov 29 at 22:11

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: “”.split(” “),
id: “106”
};
initTagRenderer(“”.split(” “), “”.split(” “), channelOptions);

StackExchange.using(“externalEditor”, function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using(“snippets”, function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: ‘answer’,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: “”,
imageUploader: {
brandingHtml: “Powered by u003ca class=”icon-imgur-white” href=”https://imgur.com/”u003eu003c/au003e”,
contentPolicyHtml: “User contributions licensed under u003ca href=”https://creativecommons.org/licenses/by-sa/3.0/”u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href=”https://stackoverflow.com/legal/content-policy”u003e(content policy)u003c/au003e”,
allowUrls: true
},
onDemand: true,
discardSelector: “.discard-answer”
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved
draft discarded

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin(‘.new-post-login’, ‘https%3a%2f%2funix.stackexchange.com%2fquestions%2f485004%2fremove-trailing-commas-from-invalid-json-to-make-it-valid%23new-answer’, ‘question_page’);
}
);

Post as a guest

Required, but never shown

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

active

oldest

votes

active

oldest

votes

up vote
2
down vote

accepted

sed -i.bak ':begin;$!N;s/,n}/n}/g;tbegin;P;D' FILE

sed -i.bak = creates a backup of the original file, then applies changes to the file

':begin;$!N;s/,n}/n}/g;tbegin;P;D' = anything ending with , followed by new line and }. Remove the , on the previous line

FILE = the file you want to make the change to

share|improve this answer

  • 1

    Fantastic. Works like Magic! Thanks a lot.
    – Somy
    Nov 29 at 21:21

up vote
2
down vote

accepted

sed -i.bak ':begin;$!N;s/,n}/n}/g;tbegin;P;D' FILE

sed -i.bak = creates a backup of the original file, then applies changes to the file

':begin;$!N;s/,n}/n}/g;tbegin;P;D' = anything ending with , followed by new line and }. Remove the , on the previous line

FILE = the file you want to make the change to

share|improve this answer

  • 1

    Fantastic. Works like Magic! Thanks a lot.
    – Somy
    Nov 29 at 21:21

up vote
2
down vote

accepted

up vote
2
down vote

accepted

sed -i.bak ':begin;$!N;s/,n}/n}/g;tbegin;P;D' FILE

sed -i.bak = creates a backup of the original file, then applies changes to the file

':begin;$!N;s/,n}/n}/g;tbegin;P;D' = anything ending with , followed by new line and }. Remove the , on the previous line

FILE = the file you want to make the change to

share|improve this answer

sed -i.bak ':begin;$!N;s/,n}/n}/g;tbegin;P;D' FILE

sed -i.bak = creates a backup of the original file, then applies changes to the file

':begin;$!N;s/,n}/n}/g;tbegin;P;D' = anything ending with , followed by new line and }. Remove the , on the previous line

FILE = the file you want to make the change to

share|improve this answer

share|improve this answer

share|improve this answer

edited Nov 29 at 21:28

kemotep

1,9493618

1,9493618

answered Nov 29 at 20:25

Wayne

443

443

  • 1

    Fantastic. Works like Magic! Thanks a lot.
    – Somy
    Nov 29 at 21:21

  • 1

    Fantastic. Works like Magic! Thanks a lot.
    – Somy
    Nov 29 at 21:21

1

1

Fantastic. Works like Magic! Thanks a lot.
– Somy
Nov 29 at 21:21

Fantastic. Works like Magic! Thanks a lot.
– Somy
Nov 29 at 21:21

up vote
4
down vote

What makes this a non-trivial problem is that the JSON format does not care about whitespace that does not occur inside keys or data. Therefore,

{ "key": "data" }

is the same as

{ "key":
"data"
}

If you add the possibility of a “broken” JSON file, such as

{ "key":
"data", }

it becomes really difficult to properly parse the document with anything other than a JSON parser that knows how to relax the restrictions of the JSON format when parsing the data.

The Perl JSON module can do that, and also pretty-print the result:

$ cat file.json
{
    "fruit": "Apple",
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple"
}

Here, we read in the whole text document into the array @text. We then decode this while relaxing the parsing (this enables the JSON document to have commas before } and ] and also to include # comments). We then immediately encode the resulting Perl data structure into JSON again and print it.

Another example:

$ cat file.json
{
    "fruit": "Apple",   # a comment
    "stuff": [1, 2, 3,],
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple",
   "stuff" : [
      1,
      2,
      3
   ]
}

Without pretty printing:

$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}))' file.json
{"fruit":"Apple","stuff":[1,2,3]}

(no newline at the end of the output)

For really large documents, you would want to use the module’s incremental parsing capability and write a proper script for the conversion.

share|improve this answer

  • Thanks for sharing the solution. I went ahead with the sed solution so did’nt try the one you shared. But thanks anyway.
    – Somy
    Nov 29 at 22:11

up vote
4
down vote

What makes this a non-trivial problem is that the JSON format does not care about whitespace that does not occur inside keys or data. Therefore,

{ "key": "data" }

is the same as

{ "key":
"data"
}

If you add the possibility of a “broken” JSON file, such as

{ "key":
"data", }

it becomes really difficult to properly parse the document with anything other than a JSON parser that knows how to relax the restrictions of the JSON format when parsing the data.

The Perl JSON module can do that, and also pretty-print the result:

$ cat file.json
{
    "fruit": "Apple",
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple"
}

Here, we read in the whole text document into the array @text. We then decode this while relaxing the parsing (this enables the JSON document to have commas before } and ] and also to include # comments). We then immediately encode the resulting Perl data structure into JSON again and print it.

Another example:

$ cat file.json
{
    "fruit": "Apple",   # a comment
    "stuff": [1, 2, 3,],
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple",
   "stuff" : [
      1,
      2,
      3
   ]
}

Without pretty printing:

$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}))' file.json
{"fruit":"Apple","stuff":[1,2,3]}

(no newline at the end of the output)

For really large documents, you would want to use the module’s incremental parsing capability and write a proper script for the conversion.

share|improve this answer

  • Thanks for sharing the solution. I went ahead with the sed solution so did’nt try the one you shared. But thanks anyway.
    – Somy
    Nov 29 at 22:11

up vote
4
down vote

up vote
4
down vote

What makes this a non-trivial problem is that the JSON format does not care about whitespace that does not occur inside keys or data. Therefore,

{ "key": "data" }

is the same as

{ "key":
"data"
}

If you add the possibility of a “broken” JSON file, such as

{ "key":
"data", }

it becomes really difficult to properly parse the document with anything other than a JSON parser that knows how to relax the restrictions of the JSON format when parsing the data.

The Perl JSON module can do that, and also pretty-print the result:

$ cat file.json
{
    "fruit": "Apple",
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple"
}

Here, we read in the whole text document into the array @text. We then decode this while relaxing the parsing (this enables the JSON document to have commas before } and ] and also to include # comments). We then immediately encode the resulting Perl data structure into JSON again and print it.

Another example:

$ cat file.json
{
    "fruit": "Apple",   # a comment
    "stuff": [1, 2, 3,],
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple",
   "stuff" : [
      1,
      2,
      3
   ]
}

Without pretty printing:

$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}))' file.json
{"fruit":"Apple","stuff":[1,2,3]}

(no newline at the end of the output)

For really large documents, you would want to use the module’s incremental parsing capability and write a proper script for the conversion.

share|improve this answer

What makes this a non-trivial problem is that the JSON format does not care about whitespace that does not occur inside keys or data. Therefore,

{ "key": "data" }

is the same as

{ "key":
"data"
}

If you add the possibility of a “broken” JSON file, such as

{ "key":
"data", }

it becomes really difficult to properly parse the document with anything other than a JSON parser that knows how to relax the restrictions of the JSON format when parsing the data.

The Perl JSON module can do that, and also pretty-print the result:

$ cat file.json
{
    "fruit": "Apple",
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple"
}

Here, we read in the whole text document into the array @text. We then decode this while relaxing the parsing (this enables the JSON document to have commas before } and ] and also to include # comments). We then immediately encode the resulting Perl data structure into JSON again and print it.

Another example:

$ cat file.json
{
    "fruit": "Apple",   # a comment
    "stuff": [1, 2, 3,],
}
$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}), {pretty=>1})' file.json
{
   "fruit" : "Apple",
   "stuff" : [
      1,
      2,
      3
   ]
}

Without pretty printing:

$ perl -MJSON -e '@text=(<>);print to_json(from_json("@text", {relaxed=>1}))' file.json
{"fruit":"Apple","stuff":[1,2,3]}

(no newline at the end of the output)

For really large documents, you would want to use the module’s incremental parsing capability and write a proper script for the conversion.

share|improve this answer

share|improve this answer

share|improve this answer

edited Nov 29 at 21:01

answered Nov 29 at 20:25

Kusalananda

119k16223364

119k16223364

  • Thanks for sharing the solution. I went ahead with the sed solution so did’nt try the one you shared. But thanks anyway.
    – Somy
    Nov 29 at 22:11

  • Thanks for sharing the solution. I went ahead with the sed solution so did’nt try the one you shared. But thanks anyway.
    – Somy
    Nov 29 at 22:11

Thanks for sharing the solution. I went ahead with the sed solution so did’nt try the one you shared. But thanks anyway.
– Somy
Nov 29 at 22:11

Thanks for sharing the solution. I went ahead with the sed solution so did’nt try the one you shared. But thanks anyway.
– Somy
Nov 29 at 22:11

draft saved
draft discarded

Thanks for contributing an answer to Unix & Linux Stack Exchange!

  • Please be sure to answer the question. Provide details and share your research!

But avoid

  • Asking for help, clarification, or responding to other answers.
  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Some of your past answers have not been well-received, and you’re in danger of being blocked from answering.

Please pay close attention to the following guidance:

  • Please be sure to answer the question. Provide details and share your research!

But avoid

  • Asking for help, clarification, or responding to other answers.
  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin(‘.new-post-login’, ‘https%3a%2f%2funix.stackexchange.com%2fquestions%2f485004%2fremove-trailing-commas-from-invalid-json-to-make-it-valid%23new-answer’, ‘question_page’);
}
);

Post as a guest

Required, but never shown

Required, but never shown

Required, but never shown

Required, but never shown

Required, but never shown

Required, but never shown

Required, but never shown

Required, but never shown

Required, but never shown

Related Post

Leave a Reply

Your email address will not be published. Required fields are marked *