This weekend, I gave a talk in BangPypers based on the content of the first two articles in this blog related to the memory footprint of Python objects and the optmization of the same.
After the talk, during the interactive sessions, an engineer asked me a very interestng question:
"Doesn't the use of slots violate the Zen of Python ?"
My first reaction was defensive as I felt such a question didn't make sense. However when I thought about it a bit more, I was able to see where he was coming from, and that it was an excellent question. I believe I gave him a response about __slots__ balancing the practicality of Python vs its purity.
However it feels like a good topic for further elaboration and clarity.
The Zen of Python
The "Zen of Python" is this set of well-known aphorisms which encompasses the Python design philosophy, courtesy Tim Peters, one of the earliest contributors to core Python. It is embedded in Python and can be seen at any Python prompt by importing this.
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
Inspecting the "Zen"
If you spend some time reading the "Zen" and its principles - you can find it talks about both the pure and practical sides. The first few principles seem to be more about the purity of the language and the code that is written using it. For example these lines,
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
However as the Zen proceeds, you find some self contradictory messaging.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
The first one says that one should not have special cases which break the "rules" but the immediate next one says somewhat tongue-in-cheek that one should favor practical approaches over pure ones. These are two messages that exactly contradict each other!
The Zen appeared in the mailing list of Python around mid 1999 whose original thread is an interesting read for anyone who likes to be aware of the history of Python. It was a somewhat tongue-in-cheek, playful response to what contributes some necessary idiomatic recommendations of Python program design.
Hence, the Zen need not be taken as a hard-and-fast rule, in fact one can take it as a set of approximate guidelines and adapt them to your specific problem, trying to balance any conflicting principles wherever required.
How __slots__ may be violating the "Zen" ?
From a purity perspective one can argue that use of __slots__ violate the Zen and feels unpythonic. Here are some arguments.
Beautiful is better than ugly
Beauty is a somewhat subjective matter of opinion. However I think most Python programmers will agree that a class defition with a long __slots__ tuple sticking under the class is definitely more ugly than the same normal class which doesn't require one.
Simple is better than complex
Slots complicate the mental model of a class by adding a special __slots__ attribute to classes which one has to keep track in inheritance as well. The simplicity of having a Python object to which any attribute can be added breaks down with slots. One has to keep a mentral track record of slotted vs regular classes when writing code.
Special cases aren't special enough to break the rules
Slots is very special case for solving a specific problem. It changes the memory layout of a Python class instance. But is it special enough to break the rules ?
Let us look at the argument for slots.
How __slots__ embodies the Zen
Practicality beats purity
When you are solving the problem of a number of similar objects taking much more memory than required, having a simpler memory model is a practical approach that works by reducing the memory footprint. So much so it is used internally in Python core objects and standard library itself.
There should be one-- and preferably only one --obvious way to do it
To create memory efficient objects that doesn't need a dynamic __dict__ use __slots__. It is the one way specified by the Python programming language.
If the implementation is easy to explain, it may be a good idea
One can explain slots to a Python programmer in under 10 minutes, so it is definitely a good idea.
Special cases aren't special enough to break the rules
The language specification provides __slots__ explicitly. It is not a hack that is undocumented or breaks the rules.
Thus __slots__ can be thought of as illustrating the Zen of Python itself as it brings forth and merges its conflicting principles of purity vs practicality into one balancing beauty with pragmatism.
Let me conclude by saying that slots may be unpythonic when used just with instinct but very Pythonic when used with insight. Use your judgement and do it right.