Dart How to remove duplicates by key-values in object list (Distinct)

eye-catch Dart and Flutter

If you want to run the code, you can download my GitHub repository.

Sponsored links

Extract unique numbers from a list

It’s easy to implement it for String, int, and any other literals. List has toSet method to remove duplicates. Just call toSet method first, and then, convert it to list again.

final list = [1, 2, 3, 4, 1, 2, 3, 4, 5];
final result = list.toSet().toList();
print(result); // [1, 2, 3, 4, 5]

If the data type of the list is such a simple data type, using toSet is the easiest way.

Sponsored links

Calling toSet doesn’t work for object list by default

List can store objects. Let’s create our own class here.

class _Product {
  final name;
  final type;
  final price;
  _Product({
    required this.name,
    required this.type,
    required this.price,
  });

  String toJson() {
    return "{ name: $name, type: $type, price: $price}";
  }
}

Let’s call toSet for this object list.

final products = [
  _Product(name: "USB", type: "A", price: 10), // same
  _Product(name: "USB", type: "A", price: 10), // same
  _Product(name: "USB", type: "B", price: 12),
  _Product(name: "USB", type: "C", price: 11),
  _Product(name: "Mouse", type: "A", price: 10),
  _Product(name: "Mouse", type: "B", price: 12), // same
  _Product(name: "Mouse", type: "B", price: 12), // same
  _Product(name: "Mouse", type: "C", price: 10),
  _Product(name: "Laptop", type: "A", price: 100),
  _Product(name: "Laptop", type: "B", price: 120), // same
  _Product(name: "Laptop", type: "B", price: 120), // same
];

print(products.length); // 11
print(products.toSet().length); // 11

The length is the same even though the list contains the same products. Let’s check the hashCode.

products.forEach((element) {
    print(element.hashCode);
    // 257416280
    // 1045932166
    // 730517725
    // 586146378
    // 215518337
    // 471982393
    // 775971279
    // 335683510
    // 844714385
    // 633234047
    // 1021163746
});

As you can see the result, each hashCode is different. I’m not sure how the default implementation of == operator for a class but I guess that it checks the two hashCodes to check whether the two objects are the same or not.

Override hashCode and == operator to make toSet work

If you want to make toSet work for objects list, you need to override hashCode and == operator.

Add the following two methods.

class _Product {
  ...

  @override
  bool operator ==(Object other) {
    return other is _Product && other.hashCode == hashCode;
  }

  @override
  int get hashCode => Object.hash(name, type, price);
}

This class doesn’t have a list, so Object.hash function is used but if it contains a list variable, use Object.hashAll instead.

Let’s check the hashCode again.

352588711
352588711
319291696
14951189
332108346
166390776
166390776
285527190
69564107
412104106
412104106

The hashCode is the same for the objects that have the same values. Let’s try to call toSet again.

print(products.length); // 11
print(products.toSet().length); // 8
products.toSet().forEach((element) => print(element.toJson()));
// { name: USB, type: A, price: 10}
// { name: USB, type: B, price: 12}
// { name: USB, type: C, price: 11}
// { name: Mouse, type: A, price: 10}
// { name: Mouse, type: B, price: 12}
// { name: Mouse, type: C, price: 10}
// { name: Laptop, type: A, price: 100}
// { name: Laptop, type: B, price: 120}

The duplicates are not shown here.

The following post can also be a good reference to know how to compare two objects.

How to remove objects that have the same key-values

There are some cases where only some keys need to be used to extract the desired objects. toSet can’t be used in this case.

To extract the unique list, the list needs to be iterated and store the element. If the current element is already stored, it doesn’t need to be stored into the list again. To do this, index needs to be used.

You can check how it works in the following post. It is written in TypeScript but you can understand it.

list.where doesn’t pass the index to the callback. Firstly, let’s create an extension for it.

extension ListExtensions<T> on List<T> {
  Iterable<T> whereWithIndex(bool test(T element, int index)) {
    final List<T> result = [];
    for (var i = 0; i < this.length; i++) {
      if (test(this[i], i)) {
        result.add(this[i]);
      }
    }
    return result;
  }
}

The target elements are added only when test function returns true. Let’s write the same logic as the one in TypeScript in the post above.

final list = [1, 2, 3, 4, 1, 2, 3, 4, 5];
final result =
    list.whereWithIndex((element, index) => list.indexOf(element) == index);
print(result); // [1, 2, 3, 4, 5]

It works for the int list. Then, let’s apply it to an object list.

final products = [
  _Product(name: "USB", type: "A", price: 10),
  _Product(name: "USB", type: "A", price: 10),
  _Product(name: "USB", type: "B", price: 12),
  _Product(name: "USB", type: "C", price: 11),
  _Product(name: "Mouse", type: "A", price: 10),
  _Product(name: "Mouse", type: "B", price: 12),
  _Product(name: "Mouse", type: "B", price: 12),
  _Product(name: "Mouse", type: "C", price: 10),
  _Product(name: "Laptop", type: "A", price: 100),
  _Product(name: "Laptop", type: "B", price: 120),
  _Product(name: "Laptop", type: "B", price: 120),
];

final result = products
    .whereWithIndex((element, index) =>
        products.indexWhere((element2) => element2.name == element.name) ==
        index)
    .toList();
result.forEach((element) => print(element.toJson()));
// { name: USB, type: A, price: 10}
// { name: Mouse, type: A, price: 10}
// { name: Laptop, type: A, price: 100}

This example compares the objects by name. Only the first appearance of each product is stored to the result.

Just add other keys to it if necessary. Let’s use type variable here.

final result = products
    .whereWithIndex((element, index) =>
        products.indexWhere((element2) =>
            element2.name == element.name &&
            element2.type == element.type) ==
        index)
    .toList();
result.forEach((element) => print(element.toJson()));
// { name: USB, type: A, price: 10}
// { name: USB, type: B, price: 12}
// { name: USB, type: C, price: 11}
// { name: Mouse, type: A, price: 10}
// { name: Mouse, type: B, price: 12}
// { name: Mouse, type: C, price: 10}
// { name: Laptop, type: A, price: 100}
// { name: Laptop, type: B, price: 120}

Comments

Copied title and URL