Revised mjlib serialization design (diagnostics part 2)
As discussed previously, I recently significantly revised the serialization format used by the mjbots quad A1 based on experience in previous professional domains, and from studying newer external projects like Apache AVRO. Here I’ll describe the design of the serialized representation, which is more completely defined at: mjlib/telemetry/README.md
Refresher and definitions
As a brief refresher, this serialization format is intended to be used primarily to record telemetry from embedded systems, where that telemetry data may be persisted on disk for a long time. Secondarily, it can be used to inspect the results of a live system. The primitive it operates on is a “record”, which is logically a structure of elements which is emitted at some intervals over time. For any given record, it logically breaks it up into a “schema” and a “data” portion. The schema describes what types of elements are present in the structure, their names and relationships. The “data” portion contains the minimum amount of information necessary to communicate one instance of the structure, assuming that the receiver already has a copy of the schema.