This is the testing Godot forums! All forum posts unique to this forum will be deleted! Please use the main forums here for any posts you want to keep. All forum rules still apply.

godot binary serialization format

vrjohnnyvrjohnny Posts: 2Member
edited June 2020 in Programming

I am trying to understand the godot binary serialization format.

It is documented here.
https://docs.godotengine.org/en/stable/tutorials/misc/binary_serialization_api.html

it is the format of a binary file created with file.store_var() and read with file.get_var()

It is documented but I get different results from the documentation.
For example this code:

file.store_var(1)

creates this binary file (in hexadecimal)

08 00 00 00 02 00 00 00 01 00 00 00

The first 4 bytes (08 00 00 00) are not documented [bytes 0-3].
the bytes 4-7 contain the header with an int that specify the type of data (2 in this case, it specify an integer in little endian encoding)
the bytes 8-11 cointain the actual stored integer (1, in little endian encoding)

So what are these first 4 bytes?
I think that they are not some sort of error correction because they remain the same if I change the stored int
example:

file.store_var(1)
file.store_var(2)
file.store_var(3)

binary file:

[unknown bytes] [int header] [actual integer]
08 00 00 00     02 00 00 00  01 00 00 00
08 00 00 00     02 00 00 00  02 00 00 00
08 00 00 00     02 00 00 00  03 00 00 00

Another difference is that the integer is actually 4 bytes (32 bits) long, not 64 bits as the documentation says.

Comments

  • bitshift-rbitshift-r Posts: 54Member

    Those first 4 bytes appear to be the number of bytes that follow.

    See encode_variant in marshalls.cpp - which is called by File::store_var

  • vrjohnnyvrjohnny Posts: 2Member
    edited June 2020

    Thank you very much for your reply, it helped a lot.

    I have another question

    32 bits integer

    This code

    file.store_var(  int(pow(2, 31)-1)  )

    note: 2^31 -1 is the last value for unsigned integer of 32 bits

    creates this file

    [length   ] [header   ] [int      ]
    08 00 00 00 02 00 00 00 FF FF FF 7F

    as expected

    64 bits integer

    this code:

    file.store_var(  int(pow(2, 31))  )

    note: the last value for unsigned integer of 32 bits + 1

    creates this file:

    [length   ] [header   ] [64 bits int          ]
    0C 00 00 00 02 00 01 00 00 00 00 80 00 00 00 00

    The length info is doesn't contain 8 anymore but 12 (0x0C) (as expected), the int is now a 64 bit integer (as expected) but the header now is different, it is

    02 00 01 00    -> 64 bit integer header
    02 00 00 00    -> 32 bit integer header

    32 bits float

    The same happens for floating point numbers
    this code:

    file.store_var(  (2 - pow(2,-23)) * pow(2, 127)  )

    note: this should be the biggest floating point number in 32 bits

    creates this file:

    [length   ] [header   ] [float    ]
    08 00 00 00 03 00 00 00 FF FF 7F 7F

    64 bits float

    this code

    file.store_var(  (2 - pow(2,-24)) * pow(2, 127)  )

    note: a slightly bigger number than the maximum floating point in 32 bits

    creates this file:

    [length   ] [header   ] [64 bits float           ]
    0C 00 00 00 03 00 01 00 00 00 00 F0 FF FF EF 47 66

    There is this byte with 01 in the header (that seem to appears when the number of bytes increase) that I don't understand

  • bitshift-rbitshift-r Posts: 54Member

    I believe that 01 00 (1 << 16) is ENCODE_FLAG_64 defined here.

Leave a Comment

Rich Text Editor. To edit a paragraph's style, hit tab to get to the paragraph menu. From there you will be able to pick one style. Nothing defaults to paragraph. An inline formatting menu will show up when you select text. Hit tab to get into that menu. Some elements, such as rich link embeds, images, loading indicators, and error messages may get inserted into the editor. You may navigate to these using the arrow keys inside of the editor and delete them with the delete or backspace key.