Missing Markers on Load

Files write missing values in a dozen ways; na_values names yours.

What counts as missing

pandas already treats an empty field and NA as missing. Real files also use n/a, -, unknown, or 999, and it has no way to guess which of those you meant.

import pandas as pd

df = pd.read_csv("scores.csv", na_values=["n/a", "unknown", "-"])
print(df["score"].isna().sum())

A missing marker left unnamed poisons the column

One "n/a" in a numeric column makes the whole column text, and every calculation on it then either fails or is wrong. Naming the marker on load is cheaper than converting afterwards.

import pandas as pd

df = pd.read_csv("scores.csv", na_values=["n/a"])
print(df["score"].dtype)
print(df["score"].mean())

Exercise

Try It Yourself

A file scores.csv uses n/a for a missing score. Write known_scores(path) that returns how many rows have a score.

import pandas as pd

def known_scores(path):
    # How many rows actually have a score?
    return 0

print(known_scores("scores.csv"))

Press Run to see output

Check Your Understanding

← Previous Next