multiformats.varint

Implementation of the unsigned-varint spec.

Suggested usage:

>>> from multiformats import varint

BytesLike

BytesLike

Type alias for bytes-like objects.

alias of Union[bytes, bytearray, memoryview]

byteslike

byteslike = (<class 'bytes'>, <class 'bytearray'>, <class 'memoryview'>): Tuple of bytes-like objects types (for use with isinstance checks).

decode

decode(b)[source]

Decodes an unsigned varint from a bytes-like object or a buffered binary stream.

if a stream is passed, only the bytes encoding the varint are read from it
if a bytes-like object is passed, the varint encoding must use all bytes

Example usage with bytes:

>>> from multiformats import varint
>>> varint.decode(b'\x80\x01')
128

Example usage with streams, for the (typical) situation where the varint is only part of the data:

>>> from io import BytesIO
>>> stream = BytesIO(b"\x80\x01\x12\xff\x01")
>>> varint.decode(stream)
128
>>> stream.read() # what's left in the stream
b'\x12\xff\x01'

Parameters:

b (BytesLike, BufferedIOBase or BinaryIO) – the bytes-like object or stream from which to decode a varint

Raises:

ValueError – if the input contains no bytes (from specs, the number 0 is encoded as 0b00000000)
ValueError – if the 9th byte of the input is a continuation byte (from specs, no number >= 2**63 is allowed)
ValueError – if the last byte of the input is a continuation byte (invalid format)
ValueError – if the decoded integer could be encoded in fewer bytes than were read (from specs, encoding must be minimal)
ValueError – if the input is a bytes-like object and the number of bytes used by the encoding is fewer than its length

The last point is a designed choice aimed to reduce errors when decoding fixed-length bytestrings (rather than streams). If this behaviour is undesirable, consider using decode_head instead.

Return type:: int

decode_raw

decode_raw(b)[source]

Specialised version of decode for partial decoding, returning a pair (x, n) of the decoded varint x and the number n of bytes read from the start and/or consumed from the stream. Unlike decode, this function doesn’t raise ValueError in case not all bytes are read in the process.

Example usage with bytes:

>>> bs = b"\x80\x01\x12\xff\x01"
>>> x, n, m = varint.decode_raw(bs)
>>> x
128
>>> n
2
# read first 2 bytes: b"\x80\x01"
>>> m
<memory at 0x000001A6E55DDA00>
>>> bytes(m)
b'\x12\xff\x01'
# memoryview on remaining bytes
# note: bytes(m) did not consume the bytes

Example usage with streams, for the (typical) situation where the varint is only part of the data:

>>> from io import BytesIO
>>> stream = BytesIO(b"\x80\x01\x12\xff\x01")
>>> x, n = varint.decode_head(stream)
>>> x
128
>>> n
2
# read first 2 bytes: b"\x80\x01"
>>> m
<_io.BytesIO object at 0x000001A6E554BBD0>
>>> m == stream
True
# original stream returned, containing remaining bytes
>>> stream.read()
b'\x12\xff\x01'
# 2 bytes were consumed decoding the varint, so 3 bytes were left in the stream
# note: stream.read() consumed the bytes

Parameters:: b (BytesLike, BufferedIOBase or BinaryIO) – the bytes-like object or stream from which to decode a varint
Raises:: ValueError – same reasons as decode, except for the last (where no error is raised)
Return type:: Tuple[int, int, Union[memoryview, BufferedIOBase, BinaryIO]]

encode

encode(x)[source]

Encodes a non-negative integer as an unsigned varint, returning the encoded bytes.

Example usage:

>>> from multiformats import varint
>>> varint.encode(128)
b'\x80\x01'

Parameters:

x (int) – the non-negative integer to encode

Raises:

ValueError – if x < 0 (varints encode unsigned integers)
ValueError – if x >= 2**63 (from specs, varints are limited to 9 bytes)

Return type:

bytes