Remove Duplicates In An Array Javascript

When working with data in JavaScript, you often need to remove duplicates in an array JavaScript to ensure each value appears only once. Plus, duplicate entries can skew calculations, clutter UI lists, or cause unexpected bugs in algorithms. Because of that, fortunately, JavaScript provides several elegant ways to deduplicate an array, ranging from modern built‑in objects to classic loops. This guide walks through the most common techniques, explains how they work under the hood, compares their performance, and offers practical tips for choosing the right approach in different scenarios.

Why Deduplication Matters

Arrays frequently come from user input, API responses, or generated loops where the same value may appear multiple times. Keeping duplicates can:

• Waste memory and processing time.
• Produce incorrect results in set‑based operations like unions or intersections.
• Cause visual noise in UI components such as dropdowns or tables.

By removing duplicates, you create a clean, predictable dataset that simplifies further logic And that's really what it comes down to. That alone is useful..

Core Techniques to Remove Duplicates

Using a Set (ES6+)

The simplest and most readable method leverages the Set object, which stores only unique values The details matter here..

function dedupeWithSet(arr) {
  return [...new Set(arr)];
}

// Example
const numbers = [1, 2, 2, 3, 4, 4, 5];
console.log(dedupeWithSet(numbers)); // [1, 2, 3, 4, 5]

How it works:
new Set(arr) iterates over the array and adds each element to the set. Because a set cannot contain duplicate keys, repeated values are ignored. Spreading the set back into an array ([...set]) yields a deduplicated list.

Pros:

• Concise, one‑liner solution.
• Preserves original order of first occurrence.
• Works for primitive types (string, number, boolean) and objects when you rely on reference equality.

Cons:

• For objects, two distinct objects with identical content are considered different (since Set uses ===).
• Slight overhead of creating a temporary set.

Filtering with indexOf or lastIndexOf

A classic approach uses filter combined with indexOf to keep only the first occurrence of each value Practical, not theoretical..

function dedupeWithFilter(arr) {
  return arr.filter((item, index) => arr.indexOf(item) === index);
}

// Example
const words = ['apple', 'banana', 'apple', 'orange', 'banana'];
console.log(dedupeWithFilter(words)); // ['apple', 'banana', 'orange']

How it works:
For each element, indexOf(item) returns the position of its first appearance. If the current index matches that position, the element is the first occurrence and is kept; otherwise, it’s a duplicate and filtered out.

Pros:

• No extra data structures beyond the temporary array returned by filter.
• Easy to understand for beginners.

Cons:

• indexOf runs in O(n) time for each element, leading to O(n²) overall complexity—inefficient for large arrays.
• Only works reliably with primitives that can be compared with ===.

Using reduce to Build a Unique Array

reduce lets you accumulate a result while checking for duplicates on the fly But it adds up..

function dedupeWithReduce(arr) {
  return arr.reduce((acc, item) => {
    return acc.includes(item) ? acc : [...acc, item];
  }, []);
}

// Example
const mixed = [1, '1', 1, '1', true, true];
console.log(dedupeWithReduce(mixed)); // [1, '1', true]

How it works:
The accumulator (acc) starts as an empty array. For each item, we test whether it already exists in acc using includes. If not, we concatenate it; otherwise, we skip it.

Pros:

• Maintains order of first appearance.
• Works with any data type as long as the equality check (includes) fits your needs.

Cons:

• Similar to the filter method, includes is O(n) per iteration, resulting in O(n²) time.
• Creates a new array at each step when using spread ([...acc, item]), which can be costly for large datasets.

Object‑Based Hashing (ES5‑Friendly)

Before Set, developers often used an object (or a Map) as a hash table to track seen values.

function dedupeWithObject(arr) {
  const seen = {};
  return arr.filter(item => {
    const key = typeof item + JSON.stringify(item);
    return seen[key] ? false : (seen[key] = true);
  });
}

// Example
const objArray = [
  { id: 1, name: 'Ali' },
  { id: 2, name: 'Budi' },
  { id: 1, name: 'Ali' }, // duplicate content
];
console.log(dedupeWithObject(objArray));
// Returns only the first two objects (reference‑based duplicates are still distinct)

How it works:
We create a lookup object (seen). For each item we generate a string key that uniquely represents its value (using typeof plus JSON.stringify). If the key already exists, we filter the item out; otherwise, we mark it as seen and keep it Easy to understand, harder to ignore..

Pros:

• O(n) average time because object property lookup is constant time.
• Works for primitives and can be adapted for deep equality of objects by customizing the key function.

Cons:

• Requires a reliable serialization method; JSON.stringify fails for functions, undefined, or circular structures.
• Slightly more verbose than the Set approach.

Using a Map for Object Deduplication

When you need to deduplicate an array of objects based on a specific property (e.g., id), a Map offers a clean solution Small thing, real impact..

function dedupeByKey(arr, keyFn) {
  const map = new Map();
  return arr.filter(item => {
    const key = keyFn(item);
    return map.has(key) ? false : map.set(key, true);
  });
}

// Example: keep first object with each unique id
const users = [
  { id: 1, name: 'Siti' },
  { id: 2, name: 'Rina' },
  { id: 1, name: 'Siti Duplicate' },
  { id: 3, name: 'Dewi' },
];
const uniqueUsers = dedupeByKey(users,

```javascript
function dedupeByKey(arr, keyFn) {
  const map = new Map();               // O(1) look‑up for each element
  return arr.filter(item => {
    const key = keyFn(item);           // compute a stable identifier
    if (map.has(key))                  // already seen → drop it
      return false;
    map.set(key, true);                // remember this key
    return true;                       // keep the current item
  });
}

Example

const users = [
  { id: 1, name: 'Siti' },
  { id: 2, name: 'Rina' },
  { id: 1, name: 'Siti Duplicate' },
  { id: 3, name: 'Dewi' },
];

const uniqueUsers = dedupeByKey(users, Object.prototype.getId);
console.

This version shines when you care about **specific attributes** rather than raw values. By passing a lightweight predicate (`keyFn`) you can deduplicate based on any field—e.g., email address, role, or even a composite string built from multiple properties.

---

## Summary of the three techniques

| Technique | When it shines | Typical performance |
|-----------|----------------|---------------------|
| **`reduce` + `includes`** | Quick one‑liner for primitive arrays where you want to preserve insertion order with minimal code. Even so, | O(n²) – each `includes` scans the growing accumulator. Think about it: |
| **Object‑based hashing** | Need a quick, reference‑oriented dedup for heterogeneous primitives; also easy to extend with custom keys. In practice, | O(n) average, but relies on `JSON. Now, stringify` and may break for non‑serializable types. Think about it: |
| **`Map`‑based deduplication** | You have many items and want fast look‑ups, or you want to deduplicate by a particular property without mutating original references. | O(n) worst‑case, no extra copies of elements during filtering. 

All three approaches respect the original order of first appearance, and they each trade off simplicity versus efficiency according to the nature of your data.

In practice, start with the simplest tool that meets your requirements. If you later discover that the basic `reduce` version becomes too slow with large lists, switch to the `Map` implementation. Should you work mainly with plain objects and need to ignore reference identity, the object‑hashing method gives you a concise way to keep only the first occurrence of each unique structure.

**Conclusion**  
Choosing between `reduce`, an object‑based hash, or a `Map` hinges on the data shape and size constraints. The `Map`‑driven function presented here provides a balanced blend of speed and flexibility, making it the most versatile choice for typical real‑world deduplication tasks while remaining easy to read and maintain.