Checking What You Loaded

Four questions to ask a new table before you trust a number from it.

The four questions

How many rows, what are the columns, what type is each, and how much is missing. Every import problem this unit covers shows up in one of those four answers.

import pandas as pd

df = pd.read_csv("scores.csv")
print(df.shape)
print(list(df.columns))
print(df.dtypes.to_dict())
print(df.isna().sum().to_dict())

Distinct values catch the rest

For a column that should hold a handful of categories, the set of what is actually in there finds the stray spellings and the values nobody expected.

import pandas as pd

df = pd.DataFrame({"subject": ["maths", "Maths", "computing"]})
print(sorted(df["subject"].unique()))

Exercise

Try It Yourself

Write load_report(path) that reads a CSV and returns a tuple of the row count, the sorted column names, and how many values are missing in total.

import pandas as pd

def load_report(path):
    # (rows, [columns], missing values)
    return None

print(load_report("scores.csv"))

Press Run to see output

Check Your Understanding

← Previous Back to the course