I have some binary data which is in Python in the form of an array of byte strings.
Is there a portable way to serialize this data that other languages could read?
JSON loses because I just found out that it has no real way to store binary data; its strings are expected to be Unicode.
I don’t want to use pickle because I don’t want the security risk, and that limits its use to other Python programs.
Any advice? I would really like to use a builtin library (or at least one that’s part of the standard Anaconda distribution).
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
If you just need the binary data in the strings and can recover the boundaries between the individual strings easily, you could just write them to a file directly, as raw strings.
If you can’t recover the string boundaries easily, JSON seems like a good option:
a = [b"abcxf3x9cxc6", b"xyz"]
serialised = json.dumps([s.decode("latin1") for s in a])
print [s.encode("latin1") for s in json.loads(serialised)]
will print
['abcxf3x9cxc6', 'xyz']
The trick here is that arbitrary binary strings are valid latin1, so they can always be decoded to Unicode and encoded back to the original string again.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0