Common Use Cases and Examples – Real Python

0 0 12 minutes read

In Python, a closure is typically a function defined inside another function. This inner function grabs the objects defined in its enclosing scope and associates them with the inner function object itself. The resulting combination is called a closure.

Closures are a common feature in functional programming languages. In Python, closures can be pretty useful because they allow you to create function-based decorators, which are powerful tools.

In this tutorial, you’ll:

Learn what closures are and how they work in Python
Get to know common use cases of closures
Explore alternatives to closures

To get the most out of this tutorial, you should be familiar with several Python topics, including functions, inner functions, decorators, classes, and callable instances.

Take the Quiz: Test your knowledge with our interactive “Python Closures: Common Use Cases and Examples” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python Closures: Common Use Cases and Examples

In this quiz, you’ll test your understanding of Python closures. Closures are a common feature in functional programming languages and are particularly popular in Python because they allow you to create function-based decorators.

Getting to Know Closures in Python

A closure is a function that retains access to its lexical scope, even when the function is executed outside that scope. When the enclosing function returns the inner function, then you get a function object with an extended scope.

In other words, closures are functions that capture the objects defined in their enclosing scope, allowing you to use them in their body. This feature allows you to use closures when you need to retain state information between consecutive calls.

Closures are common in programming languages that are focused on functional programming, and Python supports closures as part of its wide variety of features.

In Python, a closure is a function that you define in and return from another function. This inner function can retain the objects defined in the non-local scope right before the inner function’s definition.

To better understand closures in Python, you’ll first look at inner functions because closures are also inner functions.

Inner Functions

In Python, an inner function is a function that you define inside another function. This type of function can access and update names in their enclosing function, which is the non-local scope.

Here’s a quick example:

In this example, you define outer_func() at the module level or global scope. Inside this function, you define the name local variable. Then, you define another function called inner_func(). Because this second function lives in the body of outer_func(), it’s an inner or nested function. Finally, you call the inner function, which uses the name variable defined in the enclosing function.

When you call outer_func(), inner_func() interpolates name into the greeting string and prints the result to your screen.

In the above example, you defined an inner function that can use the names in the enclosing scope. However, when you call the outer function, you don’t get a reference to the inner function. The inner function and the local names won’t be available outside the outer function.

In the following section, you’ll learn how to turn an inner function into a closure, which makes the inner function and the retained variables available to you.

Function Closures

All closures are inner functions, but not all inner functions are closures. To turn an inner function into a closure, you must return the inner function object from the outer function. This may sound like a tongue twister, but here’s how you can make outer_func() return a closure object:

In this new version of outer_func(), you return the inner_func function object instead of calling it. When you call outer_func(), you get a function object that’s a closure instead of a greeting message. This closure object remembers and can access the value of name even after outer_func() has returned. That’s why you get the greeting message when you call greeter().

To create a Python closure, you need the following components:

An outer or enclosing function: This is a function that contains another function, often referred to as the inner function. The outer function can take arguments and define variables that the inner function can access and update.
Variables that are local to the outer function: These are variables from its enclosing scope. Python retains these variables, allowing you to use them in the closure, even after the outer function has returned.
An inner or nested function: This is a function defined inside the outer function. It can access and update the variables from the outer function even after the outer function has returned.

In this section’s example, you have an outer function, a local variable (name), and an inner function. The final step to getting a closure object from this combination is to return the inner function object from the outer function.

It’s important to note that you can also use lambda functions to create closures:

In this modified version of outer_func(), you use a lambda function to build the closure, which works like the original one.

Captured Variables

As you’ve learned, a closure retains the variables from its enclosing scope. Consider the following toy example:

In this example, outer_arg, local_var, and another_local_var are all attached to the closure when you call outer_func(), even if its containing scope is no longer available. However, closure() can access these variables because they’re now part of the closure itself. That’s why you can say that a closure is a function with an extended scope.

Closures can also update the value of these variables, and this can result in two scenarios: the variables can point to either an immutable or mutable object.

To update the value of a variable that points to an immutable object, you need to use the nonlocal statement. Consider the following example:

In this example, count holds a reference to an integer value, which is immutable. To update the value of count, you use a nonlocal statement that tells Python you want to reuse the variable from the non-local scope.

When your variable points to a mutable object, you can modify the variable’s value in place:

In this example, the items variable points to a list object, which is mutable. In this case, you don’t have to use the nonlocal keyword. You can modify the list in place.

Creating Closures to Retain State

In practice, you can use a Python closure in several different situations. In this section, you’ll explore how to use closures to create factory functions, maintain state across function calls, and implement callbacks, enabling more dynamic, flexible, and efficient code.

Creating Factory Functions

You can write functions to build closures with some initial configuration or parameters. This is particularly handy when you need to create multiple similar functions with different settings.

For example, say that you want to compute numeric roots with different degrees and result precisions. In this situation, you can code a factory function that returns closures with predefined degrees and precisions like the example below:

The make_root_calculator() is a factory function that you can use to create functions that compute different numeric roots. In this function, you take the root degree and the desired precision as configuration parameters.

Then, you define an inner function that takes a number as an argument and computes the specified root with the desired precision. Finally, you return the inner function, creating a closure.

You can use this function to create closures that allow you to compute numeric roots of different degrees, like square and cubic roots. Note that you can also tweak the result’s precision.

Building Stateful Functions

You can use closures to retain state between function calls. These functions are known as stateful functions, and closures are a way to build them.

For example, say that you want to write a function that takes consecutive numeric values from a data stream and computes their cumulative average. Between calls, the function must keep track of previously passed values. In this situation, you can use the following function:

In cumulative_average(), the data local variable lets you retain the state between consecutive calls of the closure object that this function returns.

Next, you create a closure called stream_average() and call it with different numeric values. Note how this closure remembers the previously passed values and computes the average by adding the newly provided value.

Providing Callback Functions

Closures are commonly used in event-driven programming when you need to create callback functions that carry additional context or state information. Graphical user interface (GUI) programming is a good example of where these callback functions are used.

To illustrate, suppose you want to create a "Hello, World!" app with Tkinter, Python’s default GUI programming library. The app needs a label to show the greeting and a button to trigger it. Here’s the code for that little app:

This code defines a toy Tkinter app that consists of a window with a label and a button. When you click the Greet button, the label displays the "Hello, World!" message.

The callback() function returns a closure object that you can use to provide the button’s command argument. This argument accepts callable objects that take no arguments. If you need to pass arguments as you did in the example, then you can use a closure.

Writing Decorators With Closures

Decorators are a powerful feature in Python. You can use decorators to modify a function’s behavior dynamically. In Python, you have two types of decorators:

Function-based decorators
Class-based decorators

A function-based decorator is a function that takes a function object as an argument and returns another function object with extended functionality. This latter function object is also a closure. So, to create function-based decorators, you use closures.

As you already learned, decorators allow you to modify the behavior of functions without altering their internal code. In practice, function-based decorators are closures. The distinguishing characteristic is that their main goal is to modify the behavior of the function that you pass as an argument to the closure-containing function.

Here’s an example of a minimal decorator that adds messages on top of the input function’s functionality:

In this example, the outer function is the decorator. This function returns a closure object that modifies the original behavior of the input function object by adding extra features. The closure can act on the input function even after the decorator() function has returned.

Here’s how you can use the decorator syntax to dynamically modify the behavior of a regular Python function:

In this example, you use @decorator to modify the behavior of your greet() function. Note that now, when you call greet(), you get its original functionality plus the functionality added by the decorator.

Implementing Memoization With Closures

Caching can improve an algorithm’s performance by avoiding unnecessary recomputation. Memoization is a common caching technique that prevents a function from running more than once for the same input.

Memoization works by storing the result for a given set of input arguments in memory and then referencing it later when necessary. You can use closures to implement memoization.

In the following toy example, you take advantage of a decorator, which is also a closure, to cache values that result from a costly hypothetical computation:

Here, memoize() takes a function object as an argument and returns another closure object. The inner function runs the input function only for unprocessed numbers. The processed numbers are cached in the cache dictionary along with the input function’s result.

Now, say that you have the following toy function that mimics a costly computation:

This function holds the code’s execution for just half a second to mimic a costly operation. To do this, you use the sleep() function from the time module.

You can measure the function’s execution time using the following code:

In this code snippet, you use the timeit() function from the timeit module to find out the execution time of slow_operation() when you run this function with a list of values. To process the six input values, the code takes a bit more than three seconds. You can use memoization to make this computation more efficient by skipping the repeated input values.

Go ahead and decorate slow_operation() using @memoize as shown below. Then, run the timing code:

Now the same code takes half the time because of the memoization technique. That’s because the slow_operation() function isn’t running for repeated input values.

Achieving Encapsulation With Closures

In object-oriented programming (OOP), classes provide a way to combine data and behavior in a single entity. A common requirement in OOP is data encapsulation, a principle that recommends protecting an object’s data from the outside world and preventing direct access.

In Python, achieving strong data encapsulation can be a difficult task because there’s no distinction between private and public attributes. Instead, Python uses a naming convention to communicate whether a given class member is public or non-public.

You can use Python closures to achieve stricter data encapsulation. Closures allow you to create a private scope for data, preventing users from accessing that data. This helps maintain data integrity and prevent unintended modifications.

To illustrate, say that you have the following Stack class:

This Stack class stores its data in a list object called ._items and implements common stack operations, such as push and pop.

Here’s how you can use this class:

Your class’s basic functionality works. However, even though the ._items attribute is non-public, you can access its values using dot notation as you’d do with a normal attribute. This behavior makes it difficult to encapsulate data to protect it from direct access.

Again, a closure provides a trick for achieving a stricter data encapsulation. Consider the following code:

In this example, you write a function to create a closure object instead of defining a class. Inside the function, you define a local variable called _items, which will be part of your closure object. You’ll use this variable to store the stack’s data. Then, you define two inner functions that implement the stack operations.

The closure() inner function is a placeholder for your closure. On top of this function, you add the push() and pop() functions. Finally, you return the resulting closure.

You can use the Stack() function mostly in the same way you used the Stack class. One significant difference is that now you don’t have access to ._items:

The Stack() function allows you to create closures that work as if they were instances of the Stack class. However, you don’t have direct access to ._items, improving your data encapsulation.

If you get really picky, you can use an advanced trick to access the content of ._items:

The .__closure__ attribute returns a tuple of cells that contain bindings for the closure’s variables. A cell object has an attribute called cell_contents, which you can use to get the value of the cell.

Even though this trick allows you to access grabbed variables, it’s not typically used in Python code. In the end, if you’re trying to achieve encapsulation, why would you break it?

Exploring Alternatives to Closures

So far, you’ve learned that Python closures can help solve several problems. However, it can be challenging to grasp how they work under the hood, so using an alternative tool can make your code easier to reason about.

You can replace a closure with a class that produces callable instances by implementing the .__call__() special method. Callable instances are objects that you can call as you’d call a function.

To illustrate, get back to the make_root_calculator() factory function:

The function returns closures that retain the root_degree and precision arguments in its extended scope. You can replace this factory function with the following class:

This class takes the same two arguments as make_root_calculator() and turns them into instance attributes.

By providing the .__call__() method, you transform your class instances into callable objects that you can call as regular functions. Here’s how you can use this class to build root calculator function-like objects:

As you can conclude, the RootCalculator works pretty much the same as the make_root_calculator() function. As a plus, you now have access to configuration arguments like root_degree.

Conclusion

Now you know that a closure is a function object typically defined inside another function in Python. Closures grab the objects defined in their enclosing scope and combine them with the inner function object to create a callable object with an extended scope.

You can use closures in multiple scenarios, especially when you need to retain the state between consecutive function calls or write a decorator. So, knowing how to use closures can be an excellent skill for a Python developer.

In this tutorial, you’ve learned: