Python generating payload and parsing payload

Question

(Using python) I am looking to generate a bytes (or can be string that I convert to bytes) that is a message to send over TCP. The format is [Header][Length][Payload]. Within [Header] is a [PayloadType] which determines the content of [Payload].

Equally, I am looking to take a response message of a similar format that I can parse into a class/object for later use. Again, the content of the response would determine what 'properties' the class object would have.

My first thought was something like:

class message():
    def __init__(self, payloadType, data, propA=None, propB=None):
        self._payloadType = payloadType
        self._data = data
        self._header = '123456'
        self._propA = propA
        self._propB = propB


    def createPacket(self):
        if self._payloadType == 1:
            return self._header + self._payloadType + self._propA + self._data
        elif self._payloadType == 2:
            return self._header + self._payloadType + self._propB + self._data
        # etc

    def parsePacket(self, packet)
        payloadType = packet[4:6]
        if payloadType == 1:
            self._propA = packet[10:12]
            # etc

The issues I see with this is there a number of payloadType and each has some commonality in the header but could have a number of different properties.

This would lead to lots of optional arguments being passed, lots of properties in the class that are potentially unused...all making the code hard to read I feel.

I feel that there is a better pattern for this in Python but I can't figure out what it is - I've read about inheritance, mixins, dataclasses, @class_method, etc but not sure if any of those fit the bill.

Is there a standard pattern way of doing this?

EDIT - more details. This relates to DoIP protocol (some info here). In there, there are different messages that can be sent that are similar (see pg 25/26 of link). And the response can have different payloads. I have a Send() function that just needs to be given a message to send (should not care about type). But when a response is received, I'd like to parse it into a message type format such that I can do things like message.nack_code to read the properties. It also means that it is all defined in one place and if anything changes in the spec, it's easier to handle.

But the properties are different depending on the type of response so I could have one large message class that can hold all properties or lots of (related) classes/objects that are specific to the type.

Answer 1

Serialization

Let's leave the parsePacket for now, and focus on the remaining part of the class. Let's also take as an example three types of messages from the link in your answer:

• 0x0001 Vehicle Identification request message
• 0x0002 Vehicle identification request message with EID
• 0x0005 Routing activation request

One could imagine three corresponding classes: VIR, VIR_EID, and RAR (names are ugly, but that's all I can get given the business domain).

Vehicle identification request message with EID is likely the same thing as the message of type 0x0001, but with an EID. Inheritance can help reducing code duplication:

class VIR:
    self.vehicle_id # Some arbitrary fields, just as an example.

class VIR_EID(VIR): # Inheritance here: VIR_EID will also have `vehicle_id`.
    self.eid

class RAR:
    self.request_uid

Now, if you have an instance of a message, you know what type it is not through some magical values of a property, but rather through the actual type of the object.

It belongs to every class to know how to convert its fields to a binary representation. One can imagine a method called to_bytes, which creates such binary representation:

class VIR:
    [...]
    def to_bytes(self):
        return self.vehicle_id.to_bytes(2, byteorder="big")

class VIR_EID(VIR):
    [...]
    def to_bytes(self):
        # Notice the usage of `super()` here: `VIR_EID` doesn't need to handle
        # the conversion of the fields which belong to `VIR`.
        return super().to_bytes() + self.eid.to_bytes(4, byteorder="big")

class RAR:
    [...]
    def to_bytes(self):
        return converters.uid_to_bytes(self.request_uid)

Deserialization

Now, the parsePacket. There is a error in your design: in order to create a message, you need to create a message. In other words, the message is in charge of its own creation, which is not ideal.

Instead, you can use a separate class (or a function) which would parse the packets. You can start with a small function:

def parse(packet):
    payload_type = int.from_bytes(packet[:2], byteorder="big")
    if payload_type == 0x0001:
        ...
    if payload_type == 0x0002:
        ...
    if payload_type == 0x0005:
        ...

While the function knows how to find what type is a given packet, it doesn't have to know all the details about how to deserialize a given type of message—this would make the function unreadable very quickly. Therefore, it can rely on the static methods of the classes to do the job:

class VIR:
    [...]
    def to_bytes(self):
        return self.vehicle_id.to_bytes(2, byteorder="big")

    @classmethod
    def from_bytes(self):
        vehicle_id = int.from_bytes(packet[2:4], byteorder="big")
        return VIR(vehicle_id)

class VIR_EID(VIR):
    [...]
    def to_bytes(self):
        return super().to_bytes() + self.eid.to_bytes(4, byteorder="big")

    @classmethod
    def from_bytes(self):
        vehicle_id = super().from_bytes().vehicle_id
        eid = int.from_bytes(packet[4:8], byteorder="big")
        return VIR_EID(vehicle_id, eid)

class RAR:
    [...]
    def to_bytes(self):
        return converters.uid_to_bytes(self.request_uid)

    @classmethod
    def from_bytes(self):
        uid = Uid(packet[2:18])
        return RAR(uid)

Let's come back to our parse function and make it rely on the newly created from_bytes methods:

def parse(packet):
    payload_type = int.from_bytes(packet[:2], byteorder="big")
    if payload_type == 0x0001:
        return VIR.from_bytes(packet)
    if payload_type == 0x0002:
        return VIR_EID.from_bytes(packet)
    if payload_type == 0x0005:
        return RAR.from_bytes(packet)

Since all our classes share the same interface when it comes to from_bytes, one can remove some code duplication by using a map:

def parse(packet):
    payload_type = int.from_bytes(packet[:2], byteorder="big")

    {
        0x0001: VIR,
        0x0002: VIR_EID,
        0x0005: RAR,
    }[payload_type].from_bytes(packet)