I'm developing my own game from scratch, and up until now I've been using json (nlohmann) for de-/serialization. My game generates a lot of objects procedurally (think of it as Minecraft in size) and objects load/unload pretty slow and occupies a lot of disc space. I've seen lots of people recommend creating your custom serialization instead of using something like protocol buffers, but I cannot find much on the subject in terms of general guidlines and principles.
What I'm looking for:
Highly performant (probably a format that translates directly from and to the objects themselves?)
Simple to extend existing classes with serialization/deserialization instructions
Serialization of objects with nested objects
Handling of arrays/vectors and primitive types
I might be thinking wrongly on these wishes, please tell me if so =)
I've been pondering and searching for guidance but not finding anything concrete. I thought that there's probably some of you smart people that have experience with this!
I've used Message Pack in the past with good results. I think it does everything you're looking for. Difference being it has no formal schema declaration or code generation step so there are tradeoffs vs something like flatbuffers
Flatbuffers! It’s like protocol buffers, but much leaner. It was actually specifically designed for games, and meets all your requirements.
The only challenge is that, like protocol buffers, it requires an extra compilation step and involves code generation. Depending on your build system, that could be a challenge.