Remove Duplicate Records in A csv file [closed]

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP

up vote
-1
down vote

favorite

I have a CSV file with pipe separated 40 column , In this file there are so many duplicate records.
I want to remove all duplicate records on the basis of 20th column records

share|improve this question

closed as unclear what you’re asking by Rui F Ribeiro, tink, George Vasiliou, Jeff Schaller, Stephen Harris Dec 4 at 3:59

Please clarify your specific problem or add additional details to highlight exactly what you need. As it’s currently written, it’s hard to tell exactly what you’re asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1

    Hi, welcome to Unix&Linux. Can you please provide some sample data? As it stands (given the utter lack of usable information) this is next to impossible to answer reliably. Your idea of what a CSV file looks like may differ from other people’s views.
    – tink
    Dec 3 at 20:51

  • File formate> 19543|29/11/2018|ABCDEF|CUSTOMER|THOMAS FLYNN|||||||||||||||1235919769|MR.|XYZ|FLYNN|4|||VILLA COURT||PUNE|PUNE|CB4 2TX|INDIA||BP1|THOMAS M FLYNN|47654060|908708|1235919769-1001||||||||||||||||||||||
    – Deep
    Dec 3 at 20:52

  • 3

    Please edit you question, don’t slap that in a comment.
    – tink
    Dec 3 at 20:52

  • For the record – that’s actually not CSV, that’s “pipe delimited” … en.wikipedia.org/wiki/Comma-separated_values
    – tink
    Dec 3 at 21:13

  • You also need to specify which row you’d like to survive the culling if there’s more than one match. First one? Last one? Some other row altogether?
    – tink
    Dec 3 at 21:19

up vote
-1
down vote

favorite

I have a CSV file with pipe separated 40 column , In this file there are so many duplicate records.
I want to remove all duplicate records on the basis of 20th column records

share|improve this question

closed as unclear what you’re asking by Rui F Ribeiro, tink, George Vasiliou, Jeff Schaller, Stephen Harris Dec 4 at 3:59

Please clarify your specific problem or add additional details to highlight exactly what you need. As it’s currently written, it’s hard to tell exactly what you’re asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1

    Hi, welcome to Unix&Linux. Can you please provide some sample data? As it stands (given the utter lack of usable information) this is next to impossible to answer reliably. Your idea of what a CSV file looks like may differ from other people’s views.
    – tink
    Dec 3 at 20:51

  • File formate> 19543|29/11/2018|ABCDEF|CUSTOMER|THOMAS FLYNN|||||||||||||||1235919769|MR.|XYZ|FLYNN|4|||VILLA COURT||PUNE|PUNE|CB4 2TX|INDIA||BP1|THOMAS M FLYNN|47654060|908708|1235919769-1001||||||||||||||||||||||
    – Deep
    Dec 3 at 20:52

  • 3

    Please edit you question, don’t slap that in a comment.
    – tink
    Dec 3 at 20:52

  • For the record – that’s actually not CSV, that’s “pipe delimited” … en.wikipedia.org/wiki/Comma-separated_values
    – tink
    Dec 3 at 21:13

  • You also need to specify which row you’d like to survive the culling if there’s more than one match. First one? Last one? Some other row altogether?
    – tink
    Dec 3 at 21:19

up vote
-1
down vote

favorite

up vote
-1
down vote

favorite

I have a CSV file with pipe separated 40 column , In this file there are so many duplicate records.
I want to remove all duplicate records on the basis of 20th column records

share|improve this question

I have a CSV file with pipe separated 40 column , In this file there are so many duplicate records.
I want to remove all duplicate records on the basis of 20th column records

shell-script shell

share|improve this question

share|improve this question

share|improve this question

share|improve this question

asked Dec 3 at 20:49

Deep

1

1

closed as unclear what you’re asking by Rui F Ribeiro, tink, George Vasiliou, Jeff Schaller, Stephen Harris Dec 4 at 3:59

Please clarify your specific problem or add additional details to highlight exactly what you need. As it’s currently written, it’s hard to tell exactly what you’re asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

closed as unclear what you’re asking by Rui F Ribeiro, tink, George Vasiliou, Jeff Schaller, Stephen Harris Dec 4 at 3:59

Please clarify your specific problem or add additional details to highlight exactly what you need. As it’s currently written, it’s hard to tell exactly what you’re asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1

    Hi, welcome to Unix&Linux. Can you please provide some sample data? As it stands (given the utter lack of usable information) this is next to impossible to answer reliably. Your idea of what a CSV file looks like may differ from other people’s views.
    – tink
    Dec 3 at 20:51

  • File formate> 19543|29/11/2018|ABCDEF|CUSTOMER|THOMAS FLYNN|||||||||||||||1235919769|MR.|XYZ|FLYNN|4|||VILLA COURT||PUNE|PUNE|CB4 2TX|INDIA||BP1|THOMAS M FLYNN|47654060|908708|1235919769-1001||||||||||||||||||||||
    – Deep
    Dec 3 at 20:52

  • 3

    Please edit you question, don’t slap that in a comment.
    – tink
    Dec 3 at 20:52

  • For the record – that’s actually not CSV, that’s “pipe delimited” … en.wikipedia.org/wiki/Comma-separated_values
    – tink
    Dec 3 at 21:13

  • You also need to specify which row you’d like to survive the culling if there’s more than one match. First one? Last one? Some other row altogether?
    – tink
    Dec 3 at 21:19

  • 1

    Hi, welcome to Unix&Linux. Can you please provide some sample data? As it stands (given the utter lack of usable information) this is next to impossible to answer reliably. Your idea of what a CSV file looks like may differ from other people’s views.
    – tink
    Dec 3 at 20:51

  • File formate> 19543|29/11/2018|ABCDEF|CUSTOMER|THOMAS FLYNN|||||||||||||||1235919769|MR.|XYZ|FLYNN|4|||VILLA COURT||PUNE|PUNE|CB4 2TX|INDIA||BP1|THOMAS M FLYNN|47654060|908708|1235919769-1001||||||||||||||||||||||
    – Deep
    Dec 3 at 20:52

  • 3

    Please edit you question, don’t slap that in a comment.
    – tink
    Dec 3 at 20:52

  • For the record – that’s actually not CSV, that’s “pipe delimited” … en.wikipedia.org/wiki/Comma-separated_values
    – tink
    Dec 3 at 21:13

  • You also need to specify which row you’d like to survive the culling if there’s more than one match. First one? Last one? Some other row altogether?
    – tink
    Dec 3 at 21:19

1

1

Hi, welcome to Unix&Linux. Can you please provide some sample data? As it stands (given the utter lack of usable information) this is next to impossible to answer reliably. Your idea of what a CSV file looks like may differ from other people’s views.
– tink
Dec 3 at 20:51

Hi, welcome to Unix&Linux. Can you please provide some sample data? As it stands (given the utter lack of usable information) this is next to impossible to answer reliably. Your idea of what a CSV file looks like may differ from other people’s views.
– tink
Dec 3 at 20:51

File formate> 19543|29/11/2018|ABCDEF|CUSTOMER|THOMAS FLYNN|||||||||||||||1235919769|MR.|XYZ|FLYNN|4|||VILLA COURT||PUNE|PUNE|CB4 2TX|INDIA||BP1|THOMAS M FLYNN|47654060|908708|1235919769-1001||||||||||||||||||||||
– Deep
Dec 3 at 20:52

File formate> 19543|29/11/2018|ABCDEF|CUSTOMER|THOMAS FLYNN|||||||||||||||1235919769|MR.|XYZ|FLYNN|4|||VILLA COURT||PUNE|PUNE|CB4 2TX|INDIA||BP1|THOMAS M FLYNN|47654060|908708|1235919769-1001||||||||||||||||||||||
– Deep
Dec 3 at 20:52

3

3

Please edit you question, don’t slap that in a comment.
– tink
Dec 3 at 20:52

Please edit you question, don’t slap that in a comment.
– tink
Dec 3 at 20:52

For the record – that’s actually not CSV, that’s “pipe delimited” … en.wikipedia.org/wiki/Comma-separated_values
– tink
Dec 3 at 21:13

For the record – that’s actually not CSV, that’s “pipe delimited” … en.wikipedia.org/wiki/Comma-separated_values
– tink
Dec 3 at 21:13

You also need to specify which row you’d like to survive the culling if there’s more than one match. First one? Last one? Some other row altogether?
– tink
Dec 3 at 21:19

You also need to specify which row you’d like to survive the culling if there’s more than one match. First one? Last one? Some other row altogether?
– tink
Dec 3 at 21:19

active

oldest

votes

active

oldest

votes

active

oldest

votes

active

oldest

votes

active

oldest

votes

Related Post

Leave a Reply

Your email address will not be published. Required fields are marked *