About Remove duplicate CSV rows
Remove duplicate CSV rows after combining reports or exporting overlapping records. You can deduplicate CSV by selected columns such as an ID, or match every field in the row. Quoted commas and multiline cells are parsed as CSV records rather than treated as separate plain-text lines.
How to remove duplicate records from a CSV
- Paste or import UTF-8 CSV content with a unique, nonempty header for every column.
- Choose match columns and whether to keep the first or last occurrence. Set case, surrounding-space, and formula-cell protection options.
- Select Process, review the number of removed rows, and download unique-rows.csv or continue to Select CSV columns.
A practical example
- Input
id,name 1,Ada 2,Lin 1,Ada updated
- Result
id,name 1,Ada 2,Lin
Only id is compared, so the first record for id 1 is kept. Other values remain unchanged.
Supported input and limitations
- Input is limited to 5 MB, 100,000 data rows, 1,000 columns, and 1,000,000 data cells. A unique nonempty header and the same field count in each record are required; blank records are skipped.
- Selected keys are compared as strings. Empty key cells match other empty key cells. Case and surrounding-space options affect matching only; they do not rewrite retained cells. Formula protection can add an apostrophe to risky cell values.
Processing happens on your device. Read about local processing and optional workspace recovery.
Frequently asked questions
Can I keep the last duplicate CSV record?
Yes. Choose Last under Keep occurrence. The retained records remain in their original relative order; they are not sorted by the key.
Is CSV deduplication different from removing duplicate lines?
Yes. A CSV field can contain quoted line breaks or commas. This tool parses records and compares the chosen fields, while Remove duplicate lines compares plain-text lines.