Mutable default arguments in Python

Recently, I came across an interesting feature in Python and thought I should share it with everyone.

Suppose we have a code snippet that looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13

def func(default_immutable_arg="Hello",
         default_mutable_arg={}):
    key = 'some_key'
    default_immutable_arg += "!"
    print(default_immutable_arg),
    if default_mutable_arg.get(key, None):
        print("{key} exists".format(key=key))
    else:
        print("{key} doesn't exist".format(key=key))
        default_mutable_arg[key] = "some_value"
for i in range(3):
    func()

Before reading further, please stop and go through the code snippet carefully.

Now, What output do you expect from the above snippet? There is a high probability (unless you are a Python wizard!) that you might expect the output to be

1
2
3

Hello! some_key doesn't exist
Hello! some_key doesn't exist
Hello! some_key doesn't exist

Well, if that’s what you expected, then you are wrong!

The correct output is

1
2
3

Hello! some_key doesn't exist
Hello! some_key exists
Hello! some_key exists

Interesting isn’t it? Let’s dig a little deeper and find out why this happens.

Perhaps, it’s quite known now.

Expressions in default arguments are calculated when the function is defined, not when it’s called.

Before I explain further, let’s verify the above statement quickly:

1
2
3
4
5

import time
def time_func(arg=time.time()):
    return arg
print [time_func() for _ in range(3)]

For me the output looks like

1
2

[1471723451.85, 1471723451.85, 1471723451.85]
# Notice the exact same timestamps here.

Clearly, arg was calculated when time_func was defined and not when it’s called otherwise you would expect arg to be different each time it’s executed.

Coming to our func example. When def statement is executed a new function object is created, bound to the name func and stored in the namespace of the module. Within the function object, for each argument object with a default value, an object is created that holds the default value.

In the above example, a string object (“Hello!”) and an empty dictionary object is created as a default for default_immutable_arg and default_mutable_arg respectively.

Now, whenever func is called without arguments, the arguments will use the values from their default bounded object i.e default_immutable_arg will always be “Hello!” but default_mutable_arg may change. It’s because of the fact that string objects are immutable whereas a dictionary objects are mutable. So whenever in line 4, we append “!” to default_immutable_arg, a new string object is created and returned which is then printed in next line, keeping default string object’s value still intact.

This isn’t the case with mutable dictionary objects. The first time we execute func without any arguments, default_mutable_arg takes its value from default dictionary object which is {} now. Hence, the else block will be executed. Since the dictionary objects are mutable, the else block changes the default dictionary object. So in the next execution of the function, when default_mutable_arg reads from default dictionary object, it receives {'some_key':'some_value'} and not {}. Interesting huh? Well that’s the explanation! :)

Solution

Don’t use the mutable argument objects as default arguments! Simple! So how do we improve our func ? Well, just use None as default argument value and check for None inside function body to determine if the arguments were passed or not.

1
2
3
4
5

def func(default_immutable_arg="Hello",
         default_mutable_arg=None):
    default_mutable_arg = ({} if default_mutable_arg is None
                              else default_mutable_arg)
    # rest is same..

Now, you can imagine what would happen if we overlook this feature in defining class methods. Clearly, all the class instances will share the same references to the same object which isn’t something you’d want to have in the first place! :)

I hope that was fun,

Cheers!