Unit 2 • Lesson 3

Fixing Types

Overview

Everything out of a file is text, and text sorts and adds in the wrong way.

Beginner 20–25 min

← Previous Next: Removing Duplicates

What You Will Learn in This Lesson

By the end of this lesson, you will be able to:

Show how text that looks like a number compares, sorts and adds wrongly.
Convert with int and float, and read the ValueError they raise.
Write a safe converter with try/except ValueError.
Clean common decorations — spaces, thousands separators, percent signs — before converting.

Why This Matters

Every value from a CSV is a string, and strings behave like numbers just often enough to be dangerous: "9" > "10" is True, and a sort puts 100 before 20. Converting types correctly on the way in is the step that makes every later sum, sort and comparison mean what it says. In pandas this is to_numeric and dtype, and the reasoning is identical.

Step 1: Text that looks like a number

The bug is quiet because every operation still works — it just does the text version.

Comparing and sorting as text

Strings compare one character at a time. "9" beats "10" because "9" comes after "1". A sorted text column puts "100" before "20".

print("9" > "10")
print(sorted(["100", "20", "3"]))
print(sorted([100, 20, 3]))

Adding as text

+ joins strings. sum on a list of strings raises, which is at least loud; the + in a loop is not.

print("9" + "10")
print(int("9") + int("10"))

Mini Practice #1: Sort it properly

Try It Yourself

Print the file order, the text sort, and a numeric sort of raw. Which one would put the wrong student first?

raw = ["9", "85", "100", "72"]

print(raw)
print(sorted(raw))
# a numeric sort:

Press Run to see output

Step 2: Converting safely

int() and float() convert text that is a clean number and raise ValueError on anything else.

What converts and what raises

int accepts surrounding spaces and a sign, but not a decimal point. float accepts decimals. Both raise on blanks and words.

print(int(" 42 "), int("-5"), float("3.5"))

for text in ["", "n/a", "3.5"]:
    try:
        print(int(text))
    except ValueError as e:
        print("ValueError:", e)

Catch the failure, do not predict it

Testing first with isdigit() looks tidy and misses "-5" and "3.5". Trying the conversion and catching ValueError lets int itself decide what counts as a number.

def to_int(text, default=0):
    try:
        return int(text)
    except ValueError:
        return default

print(to_int("92"), to_int(""), to_int("-5"), to_int("n/a", None))
print("-5".isdigit())

Key idea: Catch ValueError by name. A bare except: also swallows typos in your own code, so a misspelled variable silently becomes the default.

Step 3: Clean, then convert

Real files decorate numbers for human readers. Strip the decoration first.

Thousands separators and percent signs

"1,250" and "85%" are numbers to a person and errors to float. replace and strip remove what does not belong.

raw = ["1,250", " 85% ", "3.5"]

for text in raw:
    cleaned = text.strip().replace(",", "").rstrip("%")
    print(repr(text), "->", float(cleaned))

Convert the whole column once

Convert on the way in, into a new column of numbers, rather than calling int() at every place the value is used. Then every later line works with real numbers.

rows = [{"name": "Ada", "score": "92"}, {"name": "Grace", "score": "88"}]

scores = [int(r["score"]) for r in rows]
print(scores, sum(scores), max(scores))

Mini Practice #2: Decorated numbers

Try It Yourself

Convert every price to a float by stripping spaces, the pound sign and commas, then print the total.

prices = ["£1,200", " £85 ", "£3,000.50"]

total = 0
# clean and convert each price

print(total)

Press Run to see output

Step 4: When to Use It

Straight after reading a file

Convert every numeric column on the way in, so every later line works with real numbers.

scores = [int(r["score"]) for r in rows]

Columns with decorations or junk

Strip commas, currency and percent signs, then try the conversion and catch ValueError by name.

When Not to Use It

Do not test with isdigit() to decide what is a number — it rejects "-5" and "3.5". And do not use a bare except, which hides your own bugs along with bad data.

End-of-Lesson Exercises

Exercise 1: A safe score converter

Write to_score(text) that returns the number in text, or 0 when it is not a number at all.

Try the conversion and catch ValueError. Testing the string yourself with isdigit() misses the minus sign on negative numbers.

def to_score(text):
    # A whole number, or 0 when text is not one.
    return 0

print(to_score("92"))
print(to_score(""))

Press Run to try it, then Check my work to mark it.

Exercise 2: Parse a decorated amount

Write parse_amount(text) that turns strings like "1,250", " 85% " or "3.5" into a float, by stripping spaces, removing commas and a trailing %. Return None when what is left is not a number.

cleaned = text.strip().replace(",", "").rstrip("%"), then try: return float(cleaned) except ValueError: return None.

def parse_amount(text):
    # Strip spaces, commas and a trailing %, then float(); None if it fails.
    return None

print(parse_amount("1,250"))
print(parse_amount(" 85% "))
print(parse_amount("n/a"))

Press Run to try it, then Check my work to mark it.

← Prev Next