Mixing cdef and regular python attributes in cdef class

i am learning Cython and now experimenting with it. I tried the basic cdef class sample program and it works perfectly.

Now what i want to do is have a mix of cdef and non cdef mix of attributes in the cdef class type, something like this

cdef class Context:
    cdef public int byteIndex, bitIndex

    def __init__(self, msg = "", msg_len = 0):
        self.msg = msg 
        self.msg_len = msg_len 
        self.byteIndex = 0
        self.bitIndex = 7

but as soon as i instantiate the object i get error

!! AttributeError: 'c_asnbase.Context' object has no attribute 'msg'

Does this mean once you define a python class with cdef all self.* attributes have to be cdef defined?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

Does this mean once you define a python class with cdef all self.* attributes have to be cdef defined?

Yes. This is stated pretty explicitly in the documentation:

Attributes in cdef classes behave differently from attributes in regular classes:

• All attributes must be pre-declared at compile-time
• …

You can quite happily store a string by defining the attribute to be of type object:

cdef public object msg

Internally, the reason for this is that the cdef class does not have a dictionary, which saves space and makes attribute access faster, but it does mean that it cannot have arbitrary attributes added at runtime. This is reasonably similar to using __slots__ in a normal Python class.

Method 2

As @DavidW has pointed out, the problem of cdef-classes is that they have no __dict__. You can add __dict__ to the class-definition, if really desired:

%%cython
cdef class A:
    cdef dict __dict__        # now the usual attribute look-up is possible
    cdef readonly int answer 
    def __init__(self):
        self.answer = 42             #cdef attribute
        self.question = "unknown"    #pure python attribute, possible

And now:

a=A()
print(a.answer)
# 42
print(a.question)
# 'unknown' 
a.question = 'Why?'
print(a.question)
# 'Why?' 
setattr(a, 'new_attr', None)
print(a.new_attr)
# None

Note: setattr(a,'new_attr', None) would be not possible if cdef class A were defined without __dict__, but with cdef public object question instead.

Obviously there are additional cost using __dict__, so probably one would use the predefined attributes whenever the performance matters.
One of advantages of cdef-classes is smaller memory-footprint (for example because there is no __dict__-slot). So adding __dict__-slot would negate at least some of advantages – one should ask, whether another design would be a better option – but there are obviously scenarios, where adding __dict__-slot makes sense.

Another way would be to create a subclass of the cdef class and use it rather than the base-class.

Once the __dict__ slot is defined, instances of class A have the__dict__-attribute (which is not the case for usual cdef-classes). However, __dict__ doesn’t contain cdef-attributes, e.g. answer from the example above (no matter whether they are public or not) – only the normal pure python attributes (e.g. question and new_attr in the example above).

Here for the example above:

# no answer-attribute in __dict__:
a.__dict__
# {'new_attr': None, 'question': 'Why?'}

NB: here is the part in the Cython-documentation about dynamic attributes.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0