File Operations | Course Notes

Lesson purpose

Everything a program has stored so far disappears when it ends, and all input has come from someone typing. This lesson introduces files so programs can read large amounts of data and save their results permanently. Students learn how files and folders are organized, the difference between absolute and relative paths, what a file is made of, text vs. binary files, linear vs. random access, streams and buffering, how to open files for reading, writing, and appending, how to read a whole file, a line, or a few characters at a time, how to process comma-separated data, why and how to close files (including with/as), and how to use the os module to work with paths, folders, and files.

For students who know C# and JavaScript: Python’s open() returns a file object much like C#’s StreamReader/StreamWriter, and with ... as plays the same role as C#’s using statement. Browser JavaScript cannot touch the local file system this way; Node.js uses the fs module for the same jobs.

Learning objectives

By the end of the lesson, students should be able to:

Explain the purpose of files and file handling, and describe folders and the directory structure.
Distinguish absolute and relative file paths, and use escape characters in file names.
Identify the components of a file: name, contents, delimiter, end of file (EOF), newline, and access point.
Define file input, file output, and file I/O, and compare text files with binary files.
Explain the role of a file handler.
Compare linear (sequential) access with random access.
Explain streams, stream readers and writers, and buffering.
Open a file for reading, writing, or appending, and handle errors when a file cannot be opened.
Read an entire file, one line at a time, or a set number of characters at a time.
Read and process a comma-delimited file into a two-dimensional list.
Close a file with close() or automatically with with and as.
Create a new file, write to it, and append to an existing file.
Use os module functions to build paths, list folder contents, check paths, and remove files and folders.

1. File input and output

Why files?

input() and print() work well for small amounts of data such as a word or sentence. For larger amounts, programs use files.

A file is a digital container that stores data such as text, settings, or program commands. It can be a document, picture, video, or other type of information (for example, a playlist of songs and their ratings).
Typing thousands of values by hand is slow and error prone; reading them from a file is efficient and reliable.
When a program ends, its output is lost unless it is saved; files store output permanently.
File handling is reading large amounts of data into programs and storing the data programs create.

Folders and paths

A folder (directory) organizes files into groups and can hold other folders. The OS stores files in a hierarchy called the directory structure (folder structure), which resembles an upside-down tree. For example, songs.txt might live in C:\Music\Playlists.

A program refers to a file by name and location, described in two ways:

Path type	Starts from	Example (current folder is `C:\Music`)
Absolute file path	The top of the drive: a letter such as `C:` in Windows, or `/` in macOS and Linux	`C:\Music\Playlists\songs.txt`
Relative file path	The current folder	`.\Playlists\songs.txt`

. means the current folder; .. means the folder one level up.
Windows separates folders with a backslash (\); macOS and Linux use a forward slash (/).
Analogy: an absolute path is a home’s street address; a relative path is directions from the grocery store.
Absolute paths assume the file never moves, even on another computer. Relative paths are more flexible because they are not affected when folders above the current one are renamed or moved.

Escape characters in file names

Escape characters give an alternate interpretation to characters that otherwise have a special meaning. Inside a Python string:

Character	Escape sequence
Single quote `'`	`\'`
Double quote `"`	`\"`
Backslash `\`	`\\`

Correction: the book’s table also lists \? for a question mark. That escape comes from C and C#; Python does not recognize it (recent versions warn “invalid escape sequence”). A question mark needs no escape in Python. Also, ? is not allowed in Windows file names at all.

Tip for Windows paths: write "C:\\Music\\songs.txt", use forward slashes ("C:/Music/songs.txt" works on Windows too), or use a raw string r"C:\Music\songs.txt". C# programmers will recognize the raw string as the @"..." verbatim string.

Anatomy of a file

A file is a named collection of uninterrupted data, stored as a sequence of bytes. In a plain text file each character takes 1 byte (the book’s songs.txt has 449 characters, so 449 bytes), stored in binary (A is 01000001). Text files are simple to create and read and use little space.

When a file is created, the OS reserves a named location for it; data is added by writing to the file, and the file stays on disk until it is explicitly erased.

Components of a file:

File name: the complete name including the extension (songs.txt); the extension identifies the type (.txt plain text, .mp3 compressed music).
Contents: the text or other data in the file.
Delimiter: a tab or punctuation mark, such as a comma, that separates pieces of information (artist from song title).
End of file (EOF): a special marker for the end of the file, which the book shows as -1.
Newline: the special character \n marking the end of a line.
Access point: keeps track of your current position in the file, like a cursor.

The tab escape \t moves following text to the next tab stop.

King,Cherish,3
Magenta 7,Saturday Afternoon,5
Chartreuse,Drive You to Work,4
...

File usage

File input: code reads information from a file.
File output: code writes information to a file.
File I/O: both together. Common operations: open, read, write, close.

	Text file	Binary file
Stores	Readable 8-bit ASCII characters	Raw binary patterns that may not print
Opens in a text editor as	Readable text	Symbols and gibberish
Examples	`.txt`, `.py`, `.java`, `.cpp`, `.csv`, `.html`	`.exe`, `.mp3`, `.pdf`, `.doc`, `.bin`, images
Strength	Any OS can read it; simple	Smaller and faster (911.519 is 32 bits as an IEEE 754 float, 56 bits as seven ASCII characters)

Programs must know which kind they are reading: binary files require decoding according to their format, while text files only need each character’s byte translated.

A file handler is a built-in feature that lets you interact with a file: request information such as its size, type, location, creation time, and last-modified time, and open, read, write, and close it.

2. Processing a file

Linear vs. random access

A file access point acts like a cursor marking the current position.
Linear (sequential) access reads each byte in order from the start, like reading a book.
Random access jumps directly to a position by moving the access point forward (positive offset) or backward (negative offset) without reading what is in between. You must keep the access point within the file’s limits.

Beyond the book: in Python, file.seek(0) moves the access point back to the beginning, and file.tell() reports its current position.

Streams and buffering

A stream is a channel your program uses to exchange data with a file, like streaming a show.
A streamwriter creates a stream for writing text data to a file; a streamreader creates a stream for reading text from a file (these are the class names in C#).
Buffering moves data bit by bit into and out of a reserved part of memory called a buffer. Without buffering, the program waits for the entire file; with a buffered stream, it can start using data right away. The OS also reads files in blocks and parks the data in a buffer while it switches between tasks.

3. Reading from a file

Opening a file

file_variable = open("songs.txt", access_mode)

open() is a built-in function that returns a file object stored in file_variable.
The file name goes in quotes. If the file is not in the same folder as the program, include a relative or absolute path.
access_mode is optional; the default is "r".

Mode	Meaning	If the file does not exist	If it exists
`"r"` read	Retrieve contents	Error	Opens with the access point at the beginning
`"w"` write	Add data	Creates it	Access point at the beginning; existing contents are erased
`"a"` append	Add data at the end	Creates it	Access point at the end; new data goes after existing data

Opening can fail (the file was renamed, deleted, or locked), so wrap it in try:

try:
    file_variable = open("songs.txt", "r")
except Exception as e:
    print(e)

Clarifying note: the book says the message prints “and the program exits.” Catching the exception means the program does not crash; it continues after the except block, so any code that needs the file should be inside the try (Lesson 10). A missing file raises FileNotFoundError: [Errno 2] No such file or directory.

Three ways to read

Python’s read methods detect the end of the file for you and stop.

Read the entire file with read():

file_variable = open("songs.txt", "r")
contents = file_variable.read()
print(contents)

The whole file, newlines included, lands in one string; printing it shows real line breaks rather than \n.

Read one line at a time with readline(), which returns the line including its trailing \n, and returns the empty string "" at the end of the file. A while-loop suits an unknown number of lines:

file_variable = open("songs.txt", "r")
contents = []
line = file_variable.readline()
while line != "":
    contents.append(line)
    line = file_variable.readline()
print(contents)

Output (the list shows the escape characters): ['King,Cherish,3\n', 'Magenta 7,Saturday Afternoon,5\n', ...]

Read a number of characters with read(n):

file_variable = open("songs.txt", "r")
contents = []
two_characters = file_variable.read(2)
while two_characters != "":
    contents.append(two_characters)
    two_characters = file_variable.read(2)
print(contents)

Output begins ['Ki', 'ng', ',C', 'he', ...]. \n counts as one character (you will see pieces like '\nM'), and the last read returns whatever characters remain.

Clarifying note: the book says reading past the EOF “will crash the program.” In Python it does not: at the end of a file, read() and readline() simply return "". Also, Python files do not contain a stored EOF character; the end is detected from the file’s size.

Beyond the book: the most Pythonic way to read line by line is to loop over the file object directly: for line in file_variable:.

Common reading problems

The input stream is not open: an error occurred creating the file handler.
The file is empty: it opens, but there is nothing to read.
The file does not exist at the specified location.
The file is actually a folder.
The file is open in another program: many programs can read a file at once, but a program writing to it places a lock until writing finishes.

Catch the exceptions and use the file handler to check for unintended states such as an empty or locked file.

Comma-separated values

songs.txt is a comma-delimited text file, also called comma-separated values (.csv):

Each chunk of data up to a comma is a field (artist, song, rating).
Each line is a record.
Fields and records together form a table.

Reading field by field is more efficient than character by character. Split the contents into lines, then split each line on commas:

file_variable = open("songs.txt", "r")

artist_index = 0
song_index = 1
rating_index = 2

contents = file_variable.read()

split_by_line = contents.split("\n")
split_by_comma = [x.split(",") for x in split_by_line if x != ""]

print("First Artist:", split_by_comma[0][artist_index])
print("First Song:", split_by_comma[0][song_index])
print("First Rating:", split_by_comma[0][rating_index])

split("\n") splits the string into a list of lines.
The list comprehension splits each line on commas; if x != "" removes empty lines (such as the one after the final newline).
Named index variables make the field positions readable.

Ratings come back as strings. Convert them with a nested list comprehension that maps int() onto the rating:

final_table = [[line[artist_index], line[song_index],
                int(line[rating_index])] for line in split_by_comma]
song_count = len(final_table)
print("There are", song_count, "songs in the file.")

Output with the book’s file: There are 14 songs in the file.

Beyond the book: Python’s built-in csv module (import csv) handles quoted fields and commas inside values; it is the professional choice for real .csv files.

4. Closing a file

`close()`

When a program is finished with a file, it should close it:

file_variable.close()

If the program stops unexpectedly with a file open, the file can be corrupted or data lost.
Too many open files slows performance; closing returns resources such as buffers to the system.
A closed file becomes available to other code.
Reading from a file after closing it raises ValueError: I/O operation on closed file.

`with` and `as`

with and as open a file for a block of code and close it automatically when the block ends:

with open("songs.txt", "r") as file_variable:
    contents = file_variable.read()

    artist_index = 0
    song_index = 1
    rating_index = 2

    split_by_line = contents.split("\n")
    split_by_comma = [x.split(",") for x in split_by_line if x != ""]
    final_table = [[line[artist_index], line[song_index],
                    int(line[rating_index])] for line in split_by_comma]
    song_count = len(final_table)

print("There are", song_count, "songs in the file.")

Correction: the book says with/as “acts like a try-except block” so the program fails gracefully. It does not catch exceptions. What with guarantees is that the file is closed even if an exception occurs; the exception still propagates, and a missing file still raises FileNotFoundError. To handle the error, put the with statement inside a try block. C# students: this is exactly what using does.

5. Creating and writing files

Creating a file

output_file = open("favorite_songs.txt", "w")

Opening with "w" creates the file if it does not exist (in the program’s folder unless a path is given).
If the file exists, "w" erases its contents. To keep existing contents, use "a", which also creates the file if needed.

Choose "w" to build a new file (such as a favorites list) and "a" to add new songs to an existing list.

Writing a new file

write() writes a string. It does not add a newline, so include "\n" yourself:

with open("their_songs.txt", "w") as output_file:
    num_songs = int(input("Hello friend. How many songs would you like to enter? "))

    for i in range(num_songs):
        artist = input("Please enter the artist name. ")
        song = input("Please enter the song name. ")
        rating = input("Please enter the rating (1-5). ")

        output_file.write(artist + "," + song + "," + rating + "\n")

    print("Thanks! Your file has been written.")

The program opens the file with with, asks how many songs, loops that many times collecting the artist, song, and rating, writes each record with commas between fields and a newline at the end, and prints a message. The file closes when the block ends.

Reading one file while writing another

A single with statement can open two files. This copies every song rated 4 or higher from songs.txt into favorite_songs.txt:

with open("songs.txt", "r") as input_file, open("favorite_songs.txt", "w") as output_file:
    artist_index = 0
    song_index = 1
    rating_index = 2

    contents = input_file.read()
    split_by_line = contents.split("\n")
    split_by_comma = [x.split(",") for x in split_by_line if x != ""]
    final_table = [[line[artist_index], line[song_index],
                    int(line[rating_index])] for line in split_by_comma]

    for record in final_table:
        if record[rating_index] >= 4:
            output_file.write(record[artist_index] + "," + record[song_index]
                              + "," + str(record[rating_index]) + "\n")

The rating was converted to int for the comparison, so it must be converted back with str() before it can be concatenated and written.

Appending

To add new favorites later without erasing the list, open with "a":

open("favorite_songs.txt", "a")

6. The `os` module

import os gives access to operating-system functions for files and folders:

Function	What it does	Example
`os.path.join()`	Joins strings into a path using the correct separator for the current OS	`os.path.join("folder", "file.txt")`
`os.path.isfile()`	`True` if the path points to a file (not a folder)	`os.path.isfile("songs.txt")`
`os.path.isdir()`	`True` if the path points to a directory	`os.path.isdir("top_50")`
`os.path.exists()`	`True` if the path points to a file or directory	`os.path.exists("folder/song.txt")`
`os.listdir()`	List of names of all files and folders in a directory	`os.listdir("poems")`
`os.remove()`	Deletes a file (not a folder)	`os.remove("songs.txt")`
`os.rmdir()`	Deletes an empty directory (not files)	`os.rmdir("poems")`

Correction: the book’s example for os.rmdir() shows os.remove("poems"). The correct call is os.rmdir("poems"); os.remove() fails on a directory.

Building portable paths

Hard-coding / or \ can break a program on another OS. os.path.join() accepts any number of strings and uses the right separator:

import os

folder = input("Enter a folder name: ")
filename = input("Enter a filename: ")
filepath = os.path.join(folder, filename)
print(filepath)

On Windows, entering top_50 and bye.mp3 prints top_50\bye.mp3 (macOS and Linux print top_50/bye.mp3).

Counting `.txt` files

import os

folder = input("Enter a folder name: ")
all_files = os.listdir(folder)
count = 0

for file in all_files:
    if file.endswith(".txt"):
        count += 1

print("There are", count, "files ending in .txt.")

Counting subfolders

os.listdir() returns only names, so join each name with the folder before testing it:

import os

folder = input("Enter a folder name: ")
all_files = os.listdir(folder)
count = 0

for file in all_files:
    file_path = os.path.join(folder, file)
    if os.path.isdir(file_path):
        count += 1

print("There are", count, "subfolders in", folder + ".")

os.path.isdir() checks the actual item, so a folder named folder.txt still counts as a folder.

Checking a path

import os

folder = input("Enter a folder name: ")
filename = input("Enter a file name: ")
path = os.path.join(folder, filename)

if os.path.exists(path):
    print("The path", path, "exists.")
else:
    print("The path", path, "does not exist.")

Removing files

This deletes every file in a folder that does not end in .txt, skipping subfolders:

import os

folder = input("Enter a folder name: ")
all_files = os.listdir(folder)
removed_files = 0

for file in all_files:
    file_path = os.path.join(folder, file)
    if os.path.isfile(file_path) and not file.endswith(".txt"):
        os.remove(file_path)
        removed_files += 1

print(removed_files, "files were removed.")

Caution: os.remove() deletes permanently; files do not go to the Recycle Bin or Trash. Test file-deleting programs on a copy of a folder.

Classroom application

Paths on paper. Draw a small directory tree and have students write the absolute and relative paths to several files from a given current folder.
Three ways to read. Give students a short songs.txt and have them read it with read(), readline() in a loop, and read(2), printing each result to compare.
CSV to table. Build the two-dimensional final_table, then compute the average rating and count the songs by one artist.
Favorites filter. Write songs rated 4 or higher to a new file; then append two new favorites with mode "a" and confirm nothing was erased.
Safe open. Wrap a with open(...) in try/except FileNotFoundError and print a friendly message when the file is missing.
Folder report. Use os.listdir(), os.path.join(), and os.path.isdir() to report how many files and how many subfolders a folder contains.

Common misconceptions

“"w" adds to the end of a file.” It erases the file first; use "a" to add.
“write() starts a new line automatically.” It does not; add "\n" yourself.
“write() accepts numbers.” It only accepts strings; convert with str().
“Numbers read from a file are numbers.” Everything read from a text file is a string; convert with int() or float().
“readline() returns the line without the newline.” It keeps the \n; use strip() to remove it.
“Reading past the end of a file crashes the program.” Python returns "" at the end of the file.
“with catches file errors.” It only guarantees the file closes; use try/except to handle errors.
“os.listdir() returns full paths.” It returns names only; join them with the folder.
“os.remove() can delete a folder.” Use os.rmdir(), and only on an empty folder.
“A relative path is relative to the .py file.” It is relative to the current working directory, which is usually (but not always) the folder the program was run from.

Lesson summary

A file is a named container of data such as a document, picture, or song. Programs use files to read large amounts of data and to write data permanently (file input/output, or I/O).
Folders organize files into groups, and the OS arranges folders into a directory structure.
An absolute path gives a file’s exact location from the top of the drive; a relative path starts from the current folder and is more flexible.
Programs read text files and binary files. Files are usually read sequentially (linear access) until the end of the file; random access jumps to a position.
Streams and buffering move large amounts of data into and out of a program efficiently.
A program opens a file with open() in read, write, or append mode, then reads the whole file, one line at a time, or a set number of characters at a time.
A comma-delimited file stores records as lines and fields separated by commas; split() and list comprehension turn it into a two-dimensional list, and numeric fields must be converted from strings.
Close files with close(), or use with and as to close them automatically.
Writing creates or overwrites a file with "w"; appending adds to the end with "a"; write() needs explicit newlines.
The os module builds portable paths and lists, checks, and removes files and folders.