I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.
So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?
Array.prototype.getUnique = function() {
var o = {}, a = [], i, e;
for (i = 0; e = this[i]; i++) {o[e] = 1};
for (e in o) {a.push (e)};
return a;
}
o
= object
, a
= array
, i
= index
and e
= umm, something :P - anyone With JavaScript 1.6 / ECMAScript 5 you can use the native filter
method of an Array in the following way to get an array with unique values:
function onlyUnique(value, index, array) {
return array.indexOf(value) === index;
}
// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);
console.log(unique); // ['a', 1, 2, '1']
The native method filter
will loop through the array and leave only those entries that pass the given callback function onlyUnique
.
onlyUnique
checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.
This solution works without any extra library like jQuery or prototype.js.
It works for arrays with mixed value types too.
For old Browsers (<ie9), that do not support the native methods filter
and indexOf
you can find work arounds in the MDN documentation for filter and indexOf.
If you want to keep the last occurrence of a value, simply replace indexOf
with lastIndexOf
.
With ES6 this can be shorten to:
// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((value, index, array) => array.indexOf(value) === index);
console.log(unique); // unique is ['a', 1, 2, '1']
Thanks to Camilo Martin for hint in comment.
ES6 has a native object Set
to store unique values. To get an array with unique values you could now do this:
var myArray = ['a', 1, 'a', 2, '1'];
let unique = [...new Set(myArray)];
console.log(unique); // unique is ['a', 1, 2, '1']
The constructor of Set
takes an iterable object, like an Array, and the spread operator ...
transform the set back into an Array. Thanks to Lukas Liese for hint in comment.
Answered 2023-09-20 20:25:10
.filter((v,i,a)=>a.indexOf(v)==i)
(fat arrow notation). - anyone let unique_values = [...new Set(random_array)];
developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… - anyone Updated answer for ES6/ES2015: Using the Set and the spread operator (thanks le-m), the single line solution is:
let uniqueItems = [...new Set(items)]
Which returns
[4, 5, 6, 3, 2, 23, 1]
Answered 2023-09-20 20:25:10
Array.from(new Set([[1,2],[1,2],[1,2,3]]))
- anyone Set
and add objects instead of primitive values it will contain unique references to the objects. Thus the set s
in let s = new Set([{Foo:"Bar"}, {Foo:"Bar"}]);
will return this: Set { { Foo: 'Bar' }, { Foo: 'Bar' } }
which is a Set
with unique object references to objects that contain the same values. If you write let o = {Foo:"Bar"};
and then create a set with two references like so: let s2 = new Set([o,o]);
, then s2 will be Set { { Foo: 'Bar' } }
- anyone Array.from( new Set( items ) )
- anyone I split all answers to 4 possible solutions:
{ }
to prevent duplicates[ ]
filter + indexOf
Sets
method.Here's sample codes found in answers:
{ }
to prevent duplicatesfunction uniqueArray1( ar ) {
var j = {};
ar.forEach( function(v) {
j[v+ '::' + typeof v] = v;
});
return Object.keys(j).map(function(v){
return j[v];
});
}
[ ]
function uniqueArray2(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
filter + indexOf
function uniqueArray3(a) {
function onlyUnique(value, index, self) {
return self.indexOf(value) === index;
}
// usage
var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']
return unique;
}
[...new Set(a)]
function uniqueArray4(a) {
return [...new Set(a)];
}
And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.
I expected to see that code using object { }
will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to @rab for the code.
Google Script enabled ES6 Engine. Now I tested the last code with Sets
and it appeared faster than the object method.
Answered 2023-09-20 20:25:10
uniqueItems = [...new Set(items)]
appears to be the fastest and the most succinct of all the approaches? - anyone uniqueItems = [...new Set(items)]
seems the fastest - anyone Object.values(j);
- anyone You can also use underscore.js.
console.log(_.uniq([1, 2, 1, 3, 1, 4]));
<script src="http://underscorejs.org/underscore-min.js"></script>
which will return:
[1, 2, 3, 4]
Answered 2023-09-20 20:25:10
array = [...new Set(array)]
- anyone _.uniqWith(arrayOfObjects, _.isEqual)
. _.uniqWith([{a: 2}, {b: 3}, {a: 2}], _.isEqual)
gives [{a: 2}, {b: 3}]
. - anyone uniq()
funciton. So, even if it is not the same lib, thanks for the heads-up! - anyone With ES6 syntax
list = list.filter((x, i, a) => a.indexOf(x) == i)
x --> item in array
i --> index of item
a --> array reference, (in this case "list")
With ES5 syntax
list = list.filter(function (x, i, a) {
return a.indexOf(x) == i;
});
Browser Compatibility: IE9+
Answered 2023-09-20 20:25:10
Remove duplicates using Set
.
Array
with duplicates
const withDuplicates = [2, 2, 5, 5, 1, 1, 2, 2, 3, 3];
Get a new array without duplicates by using Set
const withoutDuplicates = Array.from(new Set(withDuplicates));
A shorter version
const withoutDuplicates = [...new Set(withDuplicates)];
Result: [2, 5, 1, 3]
Answered 2023-09-20 20:25:10
Many of the answers here may not be useful to beginners. If de-duping an array is difficult, will they really know about the prototype chain, or even jQuery?
In modern browsers, a clean and simple solution is to store data in a Set, which is designed to be a list of unique values.
const cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
const uniqueCars = Array.from(new Set(cars));
console.log(uniqueCars);
The Array.from
is useful to convert the Set back to an Array so that you have easy access to all of the awesome methods (features) that arrays have. There are also other ways of doing the same thing. But you may not need Array.from
at all, as Sets have plenty of useful features like forEach.
If you need to support old Internet Explorer, and thus cannot use Set, then a simple technique is to copy items over to a new array while checking beforehand if they are already in the new array.
// Create a list of cars, with duplicates.
var cars = ['Volvo', 'Jeep', 'Volvo', 'Lincoln', 'Lincoln', 'Ford'];
// Create a list of unique cars, to put a car in if we haven't already.
var uniqueCars = [];
// Go through each car, one at a time.
cars.forEach(function (car) {
// The code within the following block runs only if the
// current car does NOT exist in the uniqueCars list
// - a.k.a. prevent duplicates
if (uniqueCars.indexOf(car) === -1) {
// Since we now know we haven't seen this car before,
// copy it to the end of the uniqueCars list.
uniqueCars.push(car);
}
});
To make this instantly reusable, let's put it in a function.
function deduplicate(data) {
if (data.length > 0) {
var result = [];
data.forEach(function (elem) {
if (result.indexOf(elem) === -1) {
result.push(elem);
}
});
return result;
}
}
So to get rid of the duplicates, we would now do this.
var uniqueCars = deduplicate(cars);
The deduplicate(cars)
part becomes the thing we named result when the function completes.
Just pass it the name of any array you like.
Answered 2023-09-20 20:25:10
["volvo","lincoln"]
- anyone Map
to store previously seen items and an array to store the duplicate items. Then loop through the cars
array and check if the Map has the current item, if it does then push it to the duplicates array, if not then add it to the Map. I'd be happy to create a code example for you if you create a new question and we can continue the discussion there. - anyone set
method as an extension of array.prototype - anyone Array.prototype
and would strongly advise against doing so. Among other reasons, it can lead to future bugs if new methods are added to Array as part of the language. In fact, if your site is popular, it could even prevent new methods from being added to the language. That has actually happened. See: 2ality.com/2022/03/naming-conflicts.html - anyone Using ES6 new Set
var array = [3,7,5,3,2,5,2,7];
var unique_array = [...new Set(array)];
console.log(unique_array); // output = [3,7,5,2]
Using For Loop
var array = [3,7,5,3,2,5,2,7];
for(var i=0;i<array.length;i++) {
for(var j=i+1;j<array.length;j++) {
if(array[i]===array[j]) {
array.splice(j,1);
}
}
}
console.log(array); // output = [3,7,5,2]
Answered 2023-09-20 20:25:10
I have since found a nice method that uses jQuery
arr = $.grep(arr, function(v, k){
return $.inArray(v ,arr) === k;
});
Note: This code was pulled from Paul Irish's duck punching post - I forgot to give credit :P
Answered 2023-09-20 20:25:10
a.filter(e=>!(t[e]=e in t))
O(n) performance - we assume your array is in a
and t={}
. Explanation here (+Jeppe impr.)
let unique = (a,t={}) => a.filter(e=>!(t[e]=e in t));
// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));
// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];
// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))
Answered 2023-09-20 20:25:10
in
operator outside the other construction than for
loop :P) - Thank you - I appreciate it and will give +2 to your other good answers. - anyone unique(['2', 2]) // ['2'];
unique([[1, 7], [1, '7'], ['1', 7], ['1', '7']]) // [1, 7]
So be careful using this - anyone The simplest, and fastest (in Chrome) way of doing this:
Array.prototype.unique = function() {
var a = [];
for (var i=0, l=this.length; i<l; i++)
if (a.indexOf(this[i]) === -1)
a.push(this[i]);
return a;
}
Simply goes through every item in the array, tests if that item is already in the list, and if it's not, pushes to the array that gets returned.
According to JSBench, this function is the fastest of the ones I could find anywhere - feel free to add your own though.
The non-prototype version:
function uniques(arr) {
var a = [];
for (var i=0, l=arr.length; i<l; i++)
if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
a.push(arr[i]);
return a;
}
When also needing to sort the array, the following is the fastest:
Array.prototype.sortUnique = function() {
this.sort();
var last_i;
for (var i=0;i<this.length;i++)
if ((last_i = this.lastIndexOf(this[i])) !== i)
this.splice(i+1, last_i-i);
return this;
}
or non-prototype:
function sortUnique(arr) {
arr.sort();
var last_i;
for (var i=0;i<arr.length;i++)
if ((last_i = arr.lastIndexOf(arr[i])) !== i)
arr.splice(i+1, last_i-i);
return arr;
}
This is also faster than the above method in most non-Chrome browsers.
Answered 2023-09-20 20:25:10
unique
function has O(n^2) complexity while the one in getUnique
is O(n). The first one may be faster on small data sets, but how can you argue with the maths :) You can make sure the latter one is faster if you run it on an array of, say, 1e5 unique items - anyone input_array.length < 200
, otherwise uses the [...new Set(input_array)]
method. expressed as reducer: input_array.reduce((c, v) => {if (!c.includes(v)) c.push(v); return c;}, [])
- anyone We can do this using ES6 sets:
var duplicatesArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = [...new Set(duplicatesArray)];
console.log(uniqueArray); // [1,2,3,4,5]
Answered 2023-09-20 20:25:10
["Defects", "Total", "Days", "City", "Defects"].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
[0,1,2,0,3,2,1,5].reduce(function(prev, cur) {
return (prev.indexOf(cur) < 0) ? prev.concat([cur]) : prev;
}, []);
Answered 2023-09-20 20:25:10
[].reduce((p, c) => (p.some((item) => c.id === item.id) ? p : p.concat(c)), [])
- anyone This has been answered a lot, but it didn't address my particular need.
Many answers are like this:
a.filter((item, pos, self) => self.indexOf(item) === pos);
But this doesn't work for arrays of complex objects.
Say we have an array like this:
const a = [
{ age: 4, name: 'fluffy' },
{ age: 5, name: 'spot' },
{ age: 2, name: 'fluffy' },
{ age: 3, name: 'toby' },
];
If we want the objects with unique names, we should use array.prototype.findIndex
instead of array.prototype.indexOf
:
a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);
Answered 2023-09-20 20:25:10
indexOf
solution, but the findIndex
solution might be useful - anyone After looking into all the 90+ answers here, I saw there is room for one more:
Array.includes has a very handy second-parameter: "fromIndex", so by using it, every iteration of the filter
callback method will search the array, starting from [current index] + 1
which guarantees not to include currently filtered item in the lookup and also saves time.
Note - this solution does not retain the order, as it removed duplicated items from left to right, but it wins the
Set
trick if the Array is a collection of Objects.
// 🚩 🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
list.filter((v,i) => !list.includes(v,i+1))
)
// [0,1,3,"b",4,5,2,"a"]
For example, lets assume the filter
function is currently iterating at index 2
) and the value at that index happens to be 2
. The section of the array that is then scanned for duplicates (includes
method) is everything after index 2 (i+1
):
👇 👇
[0, 1, 2, 2 ,3 ,'a', 'b', 4, 5, 2, 'a']
👆 |---------------------------|
And since the currently filtered item's value 2
is included in the rest of the array, it will be filtered out, because of the leading exclamation mark which negates the filter rule.
// 🚩 🚩 🚩
var list = [0,1,2,2,3,'a','b',4,5,2,'a']
console.log(
// Initialize with empty array and fill with non-duplicates
list.reduce((acc, v) => (!acc.includes(v) && acc.push(v), acc), [])
)
// [0,1,2,3,"a","b",4,5]
Answered 2023-09-20 20:25:10
You can simlply use the built-in functions Array.prototype.filter()
and Array.prototype.indexOf()
array.filter((x, y) => array.indexOf(x) == y)
var arr = [1, 2, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 6, 9];
var newarr = arr.filter((x, y) => arr.indexOf(x) == y);
console.log(newarr);
Answered 2023-09-20 20:25:10
This prototype getUnique
is not totally correct, because if i have a Array like: ["1",1,2,3,4,1,"foo"]
it will return ["1","2","3","4"]
and "1"
is string and 1
is a integer; they are different.
Here is a correct solution:
Array.prototype.unique = function(a){
return function(){ return this.filter(a) }
}(function(a,b,c){ return c.indexOf(a,b+1) < 0 });
using:
var foo;
foo = ["1",1,2,3,4,1,"foo"];
foo.unique();
The above will produce ["1",2,3,4,1,"foo"]
.
Answered 2023-09-20 20:25:10
$foo = 'bar'
is the PHP way of declaring variables. It will work in javascript, but will create an implicit global, and generally shouldn't be done. - anyone $foo
is the way of declaring variables in javascript while actually var foo
is. - anyone [...new Set(duplicates)]
This is the simplest one and referenced from MDN Web Docs.
const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]
Answered 2023-09-20 20:25:10
Without extending Array.prototype (it is said to be a bad practice) or using jquery/underscore, you can simply filter
the array.
By keeping last occurrence:
function arrayLastUnique(array) {
return array.filter(function (a, b, c) {
// keeps last occurrence
return c.indexOf(a, b + 1) < 0;
});
},
or first occurrence:
function arrayFirstUnique(array) {
return array.filter(function (a, b, c) {
// keeps first occurrence
return c.indexOf(a) === b;
});
},
Well, it's only javascript ECMAScript 5+, which means only IE9+, but it's nice for a development in native HTML/JS (Windows Store App, Firefox OS, Sencha, Phonegap, Titanium, ...).
Answered 2023-09-20 20:25:10
filter
. At the MDN page they have an implementation for Internet Explorer, I mean, older browsers. Also: JS 1.6 refers only to Firefox's js engine, but the right thing to say it's that it is ECMAScript 5. - anyone That's because 0
is a falsy value in JavaScript.
this[i]
will be falsy if the value of the array is 0 or any other falsy value.
Answered 2023-09-20 20:25:10
Array.prototype.getUnique = function() {
var o = {}, a = []
for (var i = 0; i < this.length; i++) o[this[i]] = 1
for (var e in o) a.push(e)
return a
}
Answered 2023-09-20 20:25:10
o
instead of just a 1
, although equality comparison would still be stringwise (although, out of all the possible Javascript equalities, it doesn't seem too unreasonable). - anyone Now using sets you can remove duplicates and convert them back to the array.
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
console.log([...new Set(names)])
Another solution is to use sort & filter
var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);
Answered 2023-09-20 20:25:10
If you're using Prototype framework there is no need to do 'for' loops, you can use http://prototypejs.org/doc/latest/language/Array/prototype/uniq/ like this:
var a = Array.uniq();
Which will produce a duplicate array with no duplicates. I came across your question searching a method to count distinct array records so after uniq()
I used size()
and there was my simple result.
p.s. Sorry if i mistyped something
edit: if you want to escape undefined records you may want to add compact()
before, like this:
var a = Array.compact().uniq();
Answered 2023-09-20 20:25:10
I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.
let objArr = [{
id: '123'
}, {
id: '123'
}, {
id: '456'
}];
objArr = objArr.reduce((acc, cur) => [
...acc.filter((obj) => obj.id !== cur.id), cur
], []);
console.log(objArr);
Answered 2023-09-20 20:25:10
If you're okay with extra dependencies, or you already have one of the libraries in your codebase, you can remove duplicates from an array in place using LoDash (or Underscore).
Usage
If you don't have it in your codebase already, install it using npm:
npm install lodash
Then use it as follows:
import _ from 'lodash';
let idArray = _.uniq ([
1,
2,
3,
3,
3
]);
console.dir(idArray);
Out:
[ 1, 2, 3 ]
Answered 2023-09-20 20:25:10
_.uniqWith(objectArray, _.isEqual)
. - anyone Set
(Recommended)var array = ["FreePhoenix888", "FreePhoenix888", "konard", "FreePhoenix888"];
let set = [...new Set(array)];
console.log(set); // ["FreePhoenix888", "konard"]
Set
function filterUniqueObjects(value, index, array) {
return array.indexOf(value) === index;
}
// usage example:
var array = ["FreePhoenix888", "FreePhoenix888", "konard", "FreePhoenix888"];
var arrayOfUniqueItems = array.filter(filterUniqueObjects);
console.log(arrayOfUniqueItems); // ["FreePhoenix888", "konard"]
This example shows how you can filter not just an array of primitive values but an array of objects. I have added comments to make it easier to understand what you can change there depending on your requirements.
let array = [
{ name: '@deep-foundation/core', version: '0.0.2' },
{ name: '@deep-foundation/capacitor-device', version: '10.0.1' },
{ name: '@deep-foundation/capacitor-device', version: '10.0.2' },
];
// Of course you can inline this function as filter argument uniqueArray.filter((item, index, self) => self.findIndex(innerItem => innerItem.name === item.name) === index);
function filterUniqueObjects(value, index, self) {
return (
self.findIndex(
// Modify this function as you desire. You may want to calculate uniqueness depending only on specific fields, not all
(obj) => obj.name === value.name
) === index
);
};
let uniqueArray = array
.reverse() // If you want latest duplicates to remain
.filter(filterUniqueObjects)
.reverse(); // To get back to original order after first reverse
console.log(uniqueArray)
Answered 2023-09-20 20:25:10
I'm not sure why Gabriel Silveira wrote the function that way but a simpler form that works for me just as well and without the minification is:
Array.prototype.unique = function() {
return this.filter(function(value, index, array) {
return array.indexOf(value, index + 1) < 0;
});
};
or in CoffeeScript:
Array.prototype.unique = ->
this.filter( (value, index, array) ->
array.indexOf(value, index + 1) < 0
)
Answered 2023-09-20 20:25:10
Finding unique Array values in simple method
function arrUnique(a){
var t = [];
for(var x = 0; x < a.length; x++){
if(t.indexOf(a[x]) == -1)t.push(a[x]);
}
return t;
}
arrUnique([1,4,2,7,1,5,9,2,4,7,2]) // [1, 4, 2, 7, 5, 9]
Answered 2023-09-20 20:25:10
It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:
Array.prototype.getUnique = function(){
var u = {}, a = [];
for (var i = 0, l = this.length; i < l; ++i) {
if (u.hasOwnProperty(this[i])) {
continue;
}
a.push(this[i]);
u[this[i]] = 1;
}
return a;
}
If you do have a mixed-type array, you can serialize the hash key:
Array.prototype.getUnique = function() {
var hash = {}, result = [], key;
for ( var i = 0, l = this.length; i < l; ++i ) {
key = JSON.stringify(this[i]);
if ( !hash.hasOwnProperty(key) ) {
hash[key] = true;
result.push(this[i]);
}
}
return result;
}
Answered 2023-09-20 20:25:10
strange this hasn't been suggested before.. to remove duplicates by object key (id
below) in an array you can do something like this:
const uniqArray = array.filter((obj, idx, arr) => (
arr.findIndex((o) => o.id === obj.id) === idx
))
Answered 2023-09-20 20:25:10
filter()
and findIndex()
have to iterate through the array? That would make this a double-loop and therefore twice as expensive to run as any other answer here. - anyone new Set()
, or a lookup object similar to the answer by Grozz. - anyone