Originally posted on my-blog. Check it out for more up to date content

Let's begin by defining a simple array:

const cars = [
    'Mazda', 
    'Ford', 
    'Renault', 
    'Opel', 
    'Mazda'
]

As you can see, the first and the last item are the same. Finding this duplicate is straightforward considering that we have an array of items that are a primitive type. To achieve this, we can simply use the method filter along with the indexOf method within the provided callback.

const unique = cars.filter((car, idx) => cars.indexOf(car) === idx);
console.log(unique); // outputs ['Mazda', 'Ford', 'Renault', 'Opel']

Note that the indexOf method will return the first occurrence of an item within the array. This is why we can compare the index returned by the indexOf method with the current index in each iteration to see if the current item is a duplicate.

Finding Object Duplicates

This is the tricky part. Objects are compared via reference rather than the value or structure. This means that if we compare two objects that are exactly the same they won't match. We cannot simply do something like obj1 === obj2 because of the way they are compared.

const obj1 = {
   name: 'John',
   surname: 'Doe'
}

const obj2 = {
   name: 'John',
   surname: 'Doe'
}

const match = obj1 === obj2;
console.log(match) // outputs false

Now, what if we have an array of object duplicates? How are we going to filter out those? Considering what we've just read, it's not possible to use something simple like indexOf.

Example array:

const names = [{
   name: 'John',
   surname: 'Doe'
}, {
   name: 'Muhamed',
   surname: 'Ali'
}, {
   name: 'Mike',
   surname: 'Tyson'
}, {
   name: 'John',
   surname: 'Doe'
}, {
   name: 'John',
   surname: 'Doe'
}, {
   name: 'Mike',
   surname: 'Tyson'
}, {
   name: 'Mike',
   surname: 'Tyson'
}];

As you can see, we have a couple of duplicate items. Let's implement the function that will find the duplicates.

The Longer Version

In this approach, we will manually loop through the source array (forEach method) and check if each item exists in the resulting array by using the find method.
Considering that we have an array of objects, we must compare each property of the current object in order to make sure that the items are the same. Broken down into steps the process looks like this:

1. Get the object properties
2. Define the resulting arrays (unique and duplicates)
3. Loop through the source array
4. Try to locate the current item within the unique array
5. If the item is found, push it into the duplicates otherwise into the unique array

const findDuplicates = (source) => {
    const keys = Object.keys(source[0]);
    let unique = [], duplicates = [];

    source.forEach((item, idx) => {

        if(idx == 0) {
            unique.push(item);
            return;
        };

        const resultItem = unique.find(resultItem => {
            let notFound = true;

            keys.forEach(key => {
                notFound = notFound && 
                    item[key] != resultItem[key];
            });

            return !notFound;
        });

        (!resultItem ? unique : duplicates).push(item);

    });

    return { unique: unique, duplicates: duplicates };
};

const result = findDuplicates(names);
console.log(result.unique, result.duplicates);

// expected output

// unique items

// 0: {name: "John", surname: "Doe"}
// 1: {name: "Muhamed", surname: "Ali"}
// 2: {name: "Mike", surname: "Tyson"}

// duplicate items

// 0: {name: "John", surname: "Doe"}
// 1: {name: "John", surname: "Doe"}
// 2: {name: "Mike", surname: "Tyson"}
// 3: {name: "Mike", surname: "Tyson"}

A Bit Shorter Version

We could use the reduce method in order to achieve the same thing. This is a very powerful method and it can be used to transform the array into the desired result. It accepts a callback as a parameter which is executed for each item in the array. The return value of the callback is the given accumulator modified within each iteration. Considering that this is not an article about the reduce method, check out the official MDN documentation

Ok, back to our code. The modified version of the findDuplicates method looks like this:

const findDuplicates = (source) => {
    const keys = Object.keys(source[0]);
    return source.reduce((acc, item) => {
        const resultItem = acc.unique.find(x => {
            let notFound = true;

            keys.forEach(key => {
                notFound = notFound && 
                    item[key] != x[key];
            });

            return !notFound;
        });

        (!resultItem ? acc.unique : acc.duplicates).push(item);
        return acc;
    }, {
        unique: [],
        duplicates: []
    })
};

The modified version should return the same resulting arrays as it did before.

// unique items

// 0: {name: "John", surname: "Doe"}
// 1: {name: "Muhamed", surname: "Ali"}
// 2: {name: "Mike", surname: "Tyson"}

// duplicate items

// 0: {name: "John", surname: "Doe"}
// 1: {name: "John", surname: "Doe"}
// 2: {name: "Mike", surname: "Tyson"}
// 3: {name: "Mike", surname: "Tyson"}

That's it. Thank you for reading and see you in the next article.

Further Reading

See this cheat sheet that will guide you through the most common use cases when it comes to the array manipulation.