PyPath
Start learning
Unit 6 • Lesson 4

apply, and When Not To

Overview

A general escape hatch that is slower than the thing it usually replaces.

Intermediate 20–25 min
← Previous Next: Binning Values

What You Will Learn in This Lesson

By the end of this lesson, you will be able to:

  • Say what apply does and what it costs.
  • Reach for .str and .dt methods before reaching for apply.
  • Recognise the cases where apply genuinely is the answer.
  • Rewrite an apply as a vectorised expression.

Why This Matters

apply is the first thing people find, because it lets you write ordinary Python over a column. It is also a Python loop wearing a pandas coat, and on a large table it can be a hundred times slower than the built-in it replaced. Knowing what it costs is what makes reaching for it a choice rather than a habit.

Step 1: apply runs your function per value

It is flexible and it is a Python loop underneath, so it costs far more than the vectorised operation it usually replaces.

1

What it does

Your function is called once per element, in Python, and the results are collected into a new Series.

import pandas as pd

scores = pd.Series([92, 88, 79])
print(scores.apply(lambda s: s + 5))
print(scores + 5)          # the same answer, without the loop
2

axis=1 is the expensive one

Applying across rows builds a Series object per row and calls your function with it. On a large table this is the slowest thing in this course.

import pandas as pd

df = pd.DataFrame({"a": [1, 2], "b": [10, 20]})
print(df.apply(lambda r: r["a"] + r["b"], axis=1))
print(df["a"] + df["b"])   # the same answer, vectorised

Mini Practice #1: The same answer, two ways

Try It Yourself

Double this column with apply and then without it, and check the results match.

Press Run to see output

Step 2: Reach for the built-in first

Arithmetic, comparisons, .str methods and .dt methods all have vectorised forms. Almost every apply people write is one of these.

1

The .str accessor

Every string method you know, applied to a whole column, without a loop and without apply.

import pandas as pd

names = pd.Series([" Ada ", "GRACE"])
print(names.str.strip().str.lower())
print(names.str.len())
print(names.str.contains("A"))
2

Choosing between values

np.where and .map cover the two cases people usually reach for apply to solve: a two-way choice, and a lookup.

import pandas as pd
import numpy as np

scores = pd.Series([92, 60])
print(np.where(scores >= 70, "pass", "fail"))

codes = pd.Series(["L", "Y"])
print(codes.map({"L": "Leeds", "Y": "York"}))
3

When apply really is the answer

When the operation genuinely has no vectorised form — parsing an awkward string format, calling an external function, anything with real branching per row — apply is the right tool and there is nothing to feel bad about. It is the unconsidered apply that costs.

import pandas as pd

def band(row):
    if row["score"] > 90 and row["city"] == "L":
        return "local star"
    return "other"

df = pd.DataFrame({"score": [92, 92], "city": ["L", "Y"]})
print(df.apply(band, axis=1).tolist())

Mini Practice #2: Rewrite without apply

Try It Yourself

Each of these uses apply for something with a built-in. Rewrite all three.

Press Run to see output

Step 3: When to Use It

Logic with no vectorised form

Parsing an odd string format or calling a function that only takes one value at a time.

df["initials"] = df["name"].apply(to_initials)

A quick one-off on a small table

When readability matters more than speed and the table is a few hundred rows.

When Not to Use It

Do not use apply for arithmetic, comparisons or string methods that pandas already vectorises (.str, np.where, plain operators). It is a loop in disguise and much slower.

End-of-Lesson Exercises

Exercise 1: Clean a column of names

Write clean_names(names) that takes a Series of names and returns a list with the spaces trimmed and everything lowercased. Use the vectorised string methods rather than apply.

names.str.strip().str.lower() does both steps to the whole column, and .tolist() hands back plain strings.

Press Run to try it, then Check my work to mark it.

Exercise 2: Pass or fail, without a loop

Write verdicts(scores, mark) returning a list of "pass" or "fail" for each score, using a vectorised two-way choice rather than apply.

np.where(scores >= mark, "pass", "fail") makes the whole column at once. Finish with .tolist() so the result is plain Python strings rather than numpy ones.

Press Run to try it, then Check my work to mark it.
← Prev Next