Reading a CSV File
Overview
The csv module turns a file of text into rows you can work with.
What You Will Learn in This Lesson
By the end of this lesson, you will be able to:
- Say why splitting each line on commas breaks on real files.
- Read a file with
csv.readerinside awithblock. - Read named records with
csv.DictReader. - Remember that every value read from a file is a string.
Why This Matters
Almost all data you analyse arrives as a file, and CSV is the most common format there is. Reading it correctly is the “get the data” step for real — and the two traps in it, quoted commas and everything-is-text, are the same two traps pandas.read_csv handles for you in unit 5. Seeing them by hand first is what makes its options make sense.
Step 1: Why not just split on commas
A CSV file looks like lines of values separated by commas, so line.split(",") looks like it should work. It works until the first real file.
A comma inside a value
A name like "Lovelace, Ada" is one field. CSV protects it with quotes; split does not know about quotes and cuts the row in the wrong place, shifting every column after it.
line = '"Lovelace, Ada",92'
print(line.split(",")) # three pieces -- wrongThe csv module knows the rules
csv.reader understands quoting, escaped quotes and the line endings different systems write. Here it reads from an in-memory string, so the example runs without a file; on a real file the loop is identical.
import csv
import io
text = 'name,score\n"Lovelace, Ada",92\nGrace,88\n'
for row in csv.reader(io.StringIO(text)):
print(row)Mini Practice #1: Split versus csv
Try It YourselfPrint what split makes of each line, then what csv.reader makes of the same text. Count the fields in each.
Step 2: Opening the file properly
A file has to be opened and closed. The with statement does both, including when something goes wrong halfway through.
with open(...) as f
The block reads the file; leaving the block closes it. The example writes a small file first so there is something to read.
import csv
with open("scores.csv", "w") as f:
f.write("name,score\nAda,92\nGrace,88\n")
with open("scores.csv") as f:
for row in csv.reader(f):
print(row)The header is just the first row
csv.reader does not treat the header specially. next(reader) takes it off the front, so the loop sees only data.
import csv
with open("scores.csv", "w") as f:
f.write("name,score\nAda,92\nGrace,88\n")
with open("scores.csv") as f:
reader = csv.reader(f)
header = next(reader)
print("columns:", header)
for row in reader:
print(row)Mini Practice #2: Skip the header
Try It YourselfWrite the file, then read it with csv.reader. Take the header off with next and print only the names.
Step 3: DictReader gives you named rows
The last lesson argued for records keyed by column name. csv.DictReader builds exactly those, using the header line for the keys.
One dictionary per row
No next, no zip: the header becomes the keys automatically, and row["score"] works straight away.
import csv
with open("scores.csv", "w") as f:
f.write("name,score\nAda,92\nGrace,88\n")
with open("scores.csv") as f:
for row in csv.DictReader(f):
print(row["name"], row["score"])Everything arrives as text
A file is text, so row["score"] is "92", not 92. Adding two of them concatenates. Convert on the way in, with int or float, before any arithmetic.
import csv
with open("scores.csv", "w") as f:
f.write("name,score\nAda,92\nGrace,88\n")
with open("scores.csv") as f:
rows = list(csv.DictReader(f))
print(rows[0]["score"] + rows[1]["score"]) # "9288"
print(int(rows[0]["score"]) + int(rows[1]["score"])) # 180Key idea: The exercises below use a supplied scores.csv. Pressing Run puts that file in place for you, and Check swaps in different files to make sure your code reads it rather than reciting it.
Predict, then check: Parse first, convert second
The header is deliberately in a different order, and the name contains a comma. Predict the two printed values.
import csv
import io
text = 'score,name\n7,"Hopper, Grace"\n'
row = next(csv.DictReader(io.StringIO(text)))
print(row["name"])
print(int(row["score"]) + 1)Show the output and explanation
Hopper, Grace 8
DictReader uses the header names, so changing the column order does not change the meaning. It handles the quoted comma as part of the name. The score is still text: int converts it before addition. Opening a real CSV with newline="" lets the csv module handle line endings itself.
Try one change: Change the score to 12 and reverse the header and values together. The name should still print correctly.
Go deeper: Parse the file before interpreting its values
CSV is a file format with quoting rules, not just text separated at every comma. DictReader uses the header row as field names and returns string values. Parsing establishes the rows; conversion then gives selected fields the numeric types needed for analysis.
Trace the example
Predict the output before opening the explanation.
import csv
from io import StringIO
text = 'name,score\n"Lee, Jo",8\nAda,10\n'
rows = list(csv.DictReader(StringIO(text)))
print(rows[0]["name"])
print(sum(int(row["score"]) for row in rows))Show the output and trace
Lee, Jo 18
The comma inside the quoted name belongs to that one field. DictReader keeps Lee, Jo together. The score values start as "8" and "10", then int converts them before summing. StringIO lets this small example behave like a text file without requiring an upload.
A mistake to watch for
line.split(",") would split the quoted name incorrectly. Also, a DictReader is consumed as you iterate; a second pass over the same reader will not start again. Save rows in a list for repeated access, or reopen the file for another pass.
Transfer challenge
Add a third CSV row for "Pat, Sam" with a score of 12. Print all three names and the new total. Keep the name quoted inside the CSV text.
Change the example below to solve the challenge. Predict the result, run your code, and compare. This practice is ungraded.
Compare with a worked solution
import csv
from io import StringIO
text = 'name,score\n"Lee, Jo",8\nAda,10\n"Pat, Sam",12\n'
rows = list(csv.DictReader(StringIO(text)))
print([row["name"] for row in rows])
print(sum(int(row["score"]) for row in rows))Run both versions. Explain any difference in their output, then change one input and predict the new result.
Step 4: When to Use It
Any comma-separated file
Spreadsheet exports, logs and downloads. csv handles quoting and line endings that split() gets wrong.
with open(path) as f:
rows = list(csv.DictReader(f))Files with a header row
DictReader turns the header into keys, so the rest of the code reads fields by name.
When Not to Use It
Do not split lines on commas yourself, even for a file that looks simple today. The first quoted name with a comma in it shifts every column after it, with no error.
End-of-Lesson Exercises
Exercise 1: Print every row
A file scores.csv is supplied, with a name and a score column. Print each name and score on its own line, separated by a space.
DictReader reads the header for you, so row["name"] and row["score"] work straight away. Typing the three lines out passes the first check and fails the second.
Exercise 2: Total a column from a file
Write total_score(path) that reads the CSV at path (columns name and score) and returns the sum of the scores as a number. Scores are integer text. A quoted name may contain a comma, columns may be reordered, and a header-only file should return 0. Use the path passed into the function; return the number rather than printing it.
Loop csv.DictReader(f) and add int(row["score"]) each time. Splitting lines on commas breaks on "Lovelace, Ada", and adding the strings without int() joins them instead of summing.