Serialization ¶
The serialization subpackage groups all serialization components and serializer implementations.
Base classes are defined which can allow for other serialization implementations beside the built-in specification serializers.
- class MsgPackSerializer ( bool timestamps_as_str=False , bool timestamps_as_iso8601=False ) ¶
-
Bases:
Serializer
Provides a serializer for the MessagePack specification.
- Parameters :
-
-
timestamps_as_str ( bool , default False ) – If the serializer converts uint64_t timestamps to integer strings on serialization, and back to uint64_t on deserialization.
-
timestamps_as_iso8601 ( bool , default False ) – If the serializer converts uint64_t timestamps to ISO 8601 strings on serialization, and back to uint64_t on deserialization.
-
- deserialize ( self , bytes obj_bytes ) ¶
-
Deserialize the given MessagePack specification bytes to an object.
- Parameters :
-
obj_bytes ( bytes ) – The object bytes to deserialize.
- Returns :
-
Instrument
- Raises :
-
RuntimeError – If obj_bytes cannot be deserialized.
- serialize ( self , obj ) bytes ¶
-
Serialize the given object to MessagePack specification bytes.
- Parameters :
-
obj ( object ) – The object to serialize.
- Returns :
-
bytes
- Raises :
-
RuntimeError – If obj cannot be serialized.
- timestamps_as_iso8601 ¶
-
If the serializer converts timestamp int64_t to ISO 8601 strings.
- Returns :
-
bool
- timestamps_as_str ¶
-
If the serializer converts timestamp int64_t to integer strings.
- Returns :
-
bool
- class ParquetSerializer ¶
-
Bases:
object
Provides an object serializer for the Parquet specification.
- static deserialize ( type cls , chunk ) ¶
-
Deserialize the given Parquet specification bytes to an object.
- Parameters :
-
-
cls ( type ) – The type to deserialize to.
-
chunk ( bytes ) – The chunk to deserialize.
-
- Returns :
-
object
- Raises :
-
TypeError – If chunk cannot be deserialized.
- static serialize ( obj ) ¶
-
Serialize the given instrument to Parquet specification bytes.
- Parameters :
-
obj ( object ) – The object to serialize.
- Returns :
-
bytes
- Raises :
-
TypeError – If obj cannot be serialized.
- get_cls_table ( type cls: type ) ¶
- get_partition_keys ( type cls: type ) ¶
- get_schema ( type cls: type ) ¶
- list_schemas ( ) ¶
- register_parquet ( type cls , serializer: Optional[Callable] = None , deserializer: Optional[Callable] = None , schema: Optional[pa.Schema] = None , bool chunk=False , type table=None , **kwargs ) ¶
-
Register a new class for serialization to parquet.
- Parameters :
-
-
cls ( type ) – The type to register serialization for.
-
serializer ( Callable , optional ) – The callable to serialize instances of type cls_type to something parquet can write.
-
deserializer ( Callable , optional ) – The callable to deserialize rows from parquet into cls_type .
-
schema ( pa.Schema , optional ) – If the schema cannot be correctly inferred from a subset of the data (i.e. if certain values may be missing in the first chunk).
-
chunk ( bool , optional ) – Whether to group objects by timestamp and operate together (Used for complex objects where we write each object as multiple rows in parquet, i.e. OrderBook or AccountState ).
-
table ( type , optional ) – An optional table override for cls . Used if cls is going to be transformed and stored in a table other than its own.
-
- class Serializer ¶
-
Bases:
object
The base class for all serializers.
Warning
This class should not be used directly, but through a concrete subclass.
- deserialize ( self , bytes obj_bytes ) ¶
-
Abstract method (implement in subclass).
- serialize ( self , obj ) bytes ¶
-
Abstract method (implement in subclass).
- register_serializable_object ( obj , to_dict : Callable [ [ Any ] , dict [ str , Any ] ] , from_dict : Callable [ [ dict [ str , Any ] ] , Any ] ) void ¶
-
Register the given object with the global serialization object maps.
- Parameters :
-
-
obj ( object ) – The object to register.
-
to_dict ( Callable [ [ Any ] , dict [ str , Any ] ] ) – The delegate to instantiate a dict of primitive types from the object.
-
from_dict ( Callable [ [ dict [ str , Any ] ] , Any ] ) – The delegate to instantiate the object from a dict of primitive types.
-
- Raises :
-
-
TypeError – If to_dict or from_dict are not of type Callable .
-
KeyError – If obj already registered with the global object maps.
-