Python | Remove consecutive duplicates from list
Removing consecutive duplicates from a list means eliminating repeated elements that appear next to each other in the list. If an element repeats consecutively, only the first occurrence should remain and the duplicates should be removed.
Example:
Input: [‘a’, ‘a’, ‘b’, ‘b’, ‘c’, ‘a’, ‘a’, ‘a’]
Output: [‘a’, ‘b’, ‘c’, ‘a’]
Consecutive duplicates like ‘a’, ‘a’ and ‘b’, ‘b’ are removed, but non-consecutive duplicates like ‘a’ at the end remain because they are not consecutive.
Using list Comprehension
List comprehension in Python allows us to remove consecutive duplicates from list by iterating through it once. This method compares each element to the previous one, achieving time complexity of O(n).
Example:
a = [1, 4, 4, 4, 5, 6, 7, 4, 3, 3, 9]
# Remove consecutive duplicates
res = [a[i] for i in range(len(a)) if i == 0 or a[i] != a[i-1]]
print(res)
Output
[1, 4, 5, 6, 7, 4, 3, 9]
Explanation:
- The expression [a[i] for i in range(len(a)) if i == 0 or a[i] != a[i-1]] iterates through the list a.
- The first part i == 0 ensures the first element is always included. The second part a[i] != a[i-1] ensures that each element is included only if it is not equal to the previous one. This removes consecutive duplicates.
Table of Content
Using itertools.groupby
The groupby() function from the itertools module groups consecutive identical elements together in the list. It returns an iterator of tuples, where each tuple contains:
- key: The element itself (the value).
- group: The group of identical elements.
Example:
from itertools import groupby
a = [1, 4, 4, 4, 5, 6, 7, 4, 3, 3, 9]
res = [key for key, _ in groupby(a)]
print(res)
Output
[1, 4, 5, 6, 7, 4, 3, 9]
Explanation:
- The groupby() function from the itertools module groups consecutive identical elements together in the list. It returns an iterator of tuples, where each tuple contains:
- key: The element itself (the value).
- group: The group of identical elements.
- The list comprehension [key for key, _ in groupby(a)] iterates over the groups returned by groupby(a) and extracts only the key (the unique elements in each group).
Using Loop
Looping through the list, this method manually appends elements that are different from the previous one. It removes consecutive duplicates with a time complexity of O(n).
Example:
a = [1, 4, 4, 4, 5, 6, 7, 4, 3, 3, 9]
# Initialize result with first element
res = [a[0]] if a else []
# Loop through the list and append non-duplicate elements
for i in range(1, len(a)):
if a[i] != a[i - 1]:
res.append(a[i])
print(res)
Output
[1, 4, 5, 6, 7, 4, 3, 9]
Explanation:
- Iterating Through the List: The for loop runs from the second element (i = 1) to the end of the list.
- Skipping Consecutive Duplicates: In each iteration, if a[i] != a[i – 1]: checks if the current element is different from the previous one. If it is, it’s appended to res.
