models

Pydantic models used throughout mio.

These models should be kept as generic as possible, and any refinements needed for a specific acquisition class should be defined within that module, inheriting from the relevant parent class. Rule of thumb: keep what is common common, and what is unique unique.

Data models :)

class mio.models.BufferHeaderFormat(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.10.1.dev6+g8c206e3', linked_list: int, frame_num: int, buffer_count: int, frame_buffer_count: int, write_buffer_count: int, dropped_buffer_count: int, timestamp: int, write_timestamp: int)

Format model used to parse header at the beginning of every buffer.

Parameters:

  • linked_list (int) – Index of data buffers within the circulating structure. This increments with each buffer until it reaches [num_buffers](../api/stream.md), then resets to zero.

  • frame_num (int) – The index of the image frame, which increments with each image frame (comprising multiple data buffers).

  • buffer_count (int) – Index of data buffers, which increments with each buffer.

  • frame_buffer_count (int) – Index of the data buffer within the image frame. It is set to frame_buffer_count = 0 at the first buffer in each frame.

  • write_buffer_count (int) – Number of data buffers transmitted out of the MCU.

  • dropped_buffer_count (int) – Number of dropped data buffers.

  • timestamp (int) – Timestamp in milliseconds. This should increase approximately by 1 / framerate * 1000 every frame.

  • write_timestamp (int) – Timestamp in milliseconds when the buffer was transmitted out of the MCU.

buffer_count: int
dropped_buffer_count: int
frame_buffer_count: int
frame_num: int
linked_list: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

timestamp: int
write_buffer_count: int
write_timestamp: int
class mio.models.Container

Root model for models intended to be used as runtime data containers, eg. those that actually carry data from a buffer, rather than those that configure positions within a header.

See also: MiniscopeConfig

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class mio.models.DenoiseConfig(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.10.1.dev6+g8c206e3', interactive_display: InteractiveDisplayConfig | None = None, noise_patch: NoisePatchConfig | None = None, frequency_masking: FrequencyMaskingConfig | None = None, end_frame: int | None = None, minimum_projection: MinimumProjectionConfig | None = None, output_result: bool = True, output_dir: str | None = None)

Configuration for denoising a video.

end_frame: int | None
frequency_masking: FrequencyMaskingConfig | None
interactive_display: InteractiveDisplayConfig | None
minimum_projection: MinimumProjectionConfig | None
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

noise_patch: NoisePatchConfig | None
output_dir: str | None
output_result: bool
class mio.models.FrequencyMaskingConfig(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.10.1.dev6+g8c206e3', enable: bool = True, cast_float32: bool = False, spatial_LPF_cutoff_radius: int, vertical_BEF_cutoff: int = 5, horizontal_BEF_cutoff: int = 0, output_result: bool = False, output_mask: bool = False, output_freq_domain: bool = False)

Configuration for frequency filtering. This includes a spatial low-pass filter and vertical and horizontal band elimination filters.

cast_float32: bool
enable: bool
horizontal_BEF_cutoff: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output_freq_domain: bool
output_mask: bool
output_result: bool
spatial_LPF_cutoff_radius: int
vertical_BEF_cutoff: int
class mio.models.MiniscopeConfig

Root model for all configuration models, eg. those that are effectively static at runtime.

Note

Not named Config or BaseConfig because those are both in use already.

See also: Container

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class mio.models.MiniscopeIOModel

Root model for all mio models

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class mio.models.SDLayout(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.10.1.dev6+g8c206e3', sectors: SectorConfig, write_key0: int = 226277911, write_key1: int = 226277911, write_key2: int = 226277911, write_key3: int = 226277911, word_size: int = 4, header: SDHeaderPositions = SDHeaderPositions(gain=4, led=5, ewl=6, record_length=7, fs=8, delay_start=None, battery_cutoff=None), config: ConfigPositions = ConfigPositions(width=0, height=1, fs=2, buffer_size=3, n_buffers_recorded=4, n_buffers_dropped=5), buffer: SDBufferHeaderFormat = SDBufferHeaderFormat(id='sd-buffer-header', mio_model='mio.models.sdcard.SDBufferHeaderFormat', mio_version='0.10.1.dev6+g8c206e3', linked_list=1, frame_num=2, buffer_count=3, frame_buffer_count=4, write_buffer_count=5, dropped_buffer_count=6, timestamp=7, write_timestamp=None, length=0, data_length=8, battery_voltage=None))

Data layout of an SD Card.

Used by the io.SDCard class to tell it how data on the SD card is laid out.

buffer: SDBufferHeaderFormat
config: ConfigPositions
header: SDHeaderPositions
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

sectors: SectorConfig
word_size: int

I’m actually not sure what this is, but 4 is hardcoded a few times in the existing notebook and it appears to be used as a word size when reading from the SD card.

write_key0: int
write_key1: int
write_key2: int
write_key3: int

These don’t seem to actually be used in the existing reading/writing code, but we will leave them here for continuity’s sake :)

class mio.models.StreamDevConfig(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.10.1.dev6+g8c206e3', device: Literal['OK', 'UART'], bitstream: Path | None = None, port: str | None = None, baudrate: int | None = None, frame_width: int, frame_height: int, fs: int = 20, preamble: bytes, header_len: int, pix_depth: int = 8, buffer_block_length: int, block_size: int, num_buffers: int, reverse_header_bits: bool = False, reverse_header_bytes: bool = False, reverse_payload_bits: bool = False, reverse_payload_bytes: bool = False, dummy_words: int = 0, adc_scale: ADCScaling | None = ADCScaling(ref_voltage=1.1, bitdepth=8, battery_div_factor=5.0, vin_div_factor=11.3), runtime: StreamDevRuntime = StreamDevRuntime(serial_buffer_queue_size=10, frame_buffer_queue_size=5, image_buffer_queue_size=5, queue_put_timeout=5, plot=StreamPlotterConfig(keys=['timestamp', 'buffer_count', 'frame_buffer_count'], update_ms=1000, history=500), csvwriter=CSVWriterConfig(buffer=100), ntp_server=None, ntp_max_offset_seconds=0.01))

Format model used to parse DAQ configuration yaml file (examples are in ./config) The model attributes are key-value pairs needed for reconstructing frames from data streams.

Parameters:

  • device (str) – Interface hardware used for receiving data. Current options are “OK” (Opal Kelly XEM 7310) and “UART” (generic UART-USB converters). Only “OK” is supported at the moment.

  • bitstream (str, optional) – Required when device is “OK”. The configuration bitstream file to upload to the Opal Kelly board. This uploads a Manchester decoder HDL and different bitstream files are required to configure different data rates and bit polarity. This is a binary file synthesized using Vivado, and details for generating this file will be provided in later updates.

  • port (str, optional) – Required when device is “UART”. COM port connected to the UART-USB converter.

  • baudrate (Optional[int]) – Required when device is “UART”. Baudrate of the connection to the UART-USB converter.

  • frame_width (int) – Frame width of transferred image. This is used to reconstruct image.

  • frame_height (int) – Frame height of transferred image. This is used to reconstruct image.

  • fs (int) – Framerate of acquired stream

  • preamble (str) – 32-bit preamble used to locate the start of each buffer. The header and image data follows this preamble. This is used as a hex but imported as a string because yaml doesn’t support hex format.

  • header_len (int, optional) – Length of header in bits. (For 32-bit words, 32 * number of words) This is useful when not all the variable/words in the header are defined in MetadataHeaderFormat. The user is responsible to ensure that header_len is larger than the largest bit position defined in MetadataHeaderFormat otherwise unexpected behavior might occur.

  • pix_depth (int, optional) – Bit-depth of each pixel, by default 8.

  • buffer_block_length (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. Number of blocks per each data buffer. This is required to calculate the number of pixels contained in one data buffer.

  • block_size (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. Number of 32-bit words per data block. This is required to calculate the number of pixels contained in one data buffer.

  • num_buffers (int) – Defines the data buffer structure. This value needs to match the Miniscope firmware. This is the number of buffers that the source microcontroller cycles around. This isn’t strictly required for data reconstruction but useful for debugging.

  • reverse_header_bits (bool, optional) – If True, reverse the bits within each byte of the header. Default is False.

  • reverse_header_bytes (bool, optional) – If True, reverse the byte order within each 32-bit word of the header. This is used for handling endianness in systems where the byte order needs to be swapped. Default is False.

  • reverse_payload_bits (bool, optional) – If True, reverse the bits within each byte of the payload. Default is False.

  • reverse_payload_bytes (bool, optional) – If True, reverse the byte order within each 32-bit word of the payload. This is used for handling endianness in systems where the byte order needs to be swapped. Default is False.

  • dummy_words (int, optional) – Number of 32-bit dummy words in the header. This is used to stabilize clock recovery in FPGA Manchester decoder. This value does not have a meaning for image recovery.

  • ..todo:: – Move port (for USART) to a user config area. This should make this pure device config.

adc_scale: ADCScaling | None
baudrate: int | None
bitstream: Path | None
block_size: int
buffer_block_length: int
property buffer_npix: list[int]

List of pixel counts per buffer for a complete frame.

A frame is split across multiple buffers. This returns a list where each element is the number of pixels in that buffer. The last buffer may have fewer pixels (the remainder).

device: Literal['OK', 'UART']
dummy_words: int
classmethod ensure_exists(value: Path | None) → Path | None

If a bitstream file has been provided, ensure it exists

frame_height: int
frame_width: int
fs: int
header_len: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

  • self – The BaseModel instance.

  • context – The context.

num_buffers: int
pix_depth: int
port: str | None
preamble: bytes
classmethod preamble_to_bytes(value: str | bytes | int) → bytes

Cast preamble to bytes.

Parameters:

value (str, bytes, int) – Recast from str (in yaml like preamble: "0x12345" ) or int (in yaml like preamble: 0x12345

Returns:

bytes

property px_per_buffer: int

Number of pixels per buffer

classmethod resolve_relative(value: Path) → Path

If we are given a relative path to a bitstream, resolve it relative to the device path

reverse_header_bits: bool
reverse_header_bytes: bool
reverse_payload_bits: bool
reverse_payload_bytes: bool
runtime: StreamDevRuntime
class mio.models.UpdateBatch(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.10.1.dev6+g8c206e3', devices: list[DeviceUpdateEntry])

Batch update configuration. Only list form is supported:

devices: [DeviceUpdateEntry, …]

devices: list[DeviceUpdateEntry]
iter_updates(default_port: str | None) → list[tuple[str | None, int, Literal['LED', 'GAIN', 'ROI_X', 'ROI_Y', 'SUBSAMPLE'], int]]

Yield (port, device_id, key, value) tuples for all updates in this batch.

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].