Removing Duplicates

Deciding what counts as the same row is most of the work.

Exact duplicates

A set removes repeats, but it also loses the order and cannot hold a dictionary. For rows, tracking what you have seen keeps both.

names = ["Ada", "Grace", "Ada"]
print(len(names), len(set(names)))

seen = []
for n in names:
    if n not in seen:
        seen.append(n)
print(seen)

Same row, written differently

"Ada" and "ada" are one student to a human and two to a computer. Normalising before comparing is what makes the duplicate visible.

names = ["Ada", "ada", "GRACE"]
keys = [n.strip().lower() for n in names]
print(keys)
print(len(set(keys)))

Exercise

Try It Yourself

Write unique_names(names) that returns the names with duplicates removed, ignoring case and surrounding spaces, keeping the first spelling of each and the original order.

def unique_names(names):
    # Keep the first spelling of each name, in order.
    return []

print(unique_names(["Ada", " ada ", "Grace"]))

Press Run to see output

Check Your Understanding

← Previous Next