Block

class imagepypelines.Block(name=None, batch_type='all', types=None, shapes=None, containers=None, void=False)

Bases: object

a contained algorithmic element used to construct pipelines. This class is designed to be inherited from, or used in the form of one of its child classes.

Note

you must overload Block.process() if you intend to inherit from this class

uuid

hex uuid for this pipeline

Type

str

name

user specified name for this pipeline, used to generate the unique id. defaults to the name of your subclass

Type

str

batch_type

the size of the batch fed into your process function. Will be an integer, “all”, or “each”

Type

str, int

void

Boolean value. By default all blocks return a value or values as output. However, if printing to screen, plotting, or saving data to a file, a block may not have a meaningful output that should be stored in a pipeline’s output dictionary. In this case, void should be set to True, so that the output of the block is ignored. The associated var key in the pipeline output will contain a value of None.

Type

bool

logger

Logger object for this block. When run in a pipeline this logger is temporaily replaced with a child of the Pipeline’s logger

Type

ImagepypelinesLogger

tags

tags to describe this block. unused as of March 2020

Type

set

_arg_spec

a named tuple describing the arguments for this block’s process function. Only defined if the property block.args is accessed.

Type

namedtuple,None

skip_enforcement

whether or not to enforce type and shape checking

Type

bool

types

Dictionary of input types. If arg doesn’t exist as a key, or if the value is None, then no checking is done

Type

dict

shapes

Dictionary of input shapes. If arg doesn’t exist as a key, or if the value is None, then no checking is done

Type

dict

containers

Dictionary of input containers. If arg doesn’t exist as a key, or if the value is None, then no checking is done if batch_type is “each”, then the container is irrelevant and can be safely ignored!

Type

dict

shape_fns

Dictionary of shape functions to retrieve. If type(arg_datum) doesn’t exist as a key, or if the value is None, then no checking is done.

Type

dict

Attributes Summary

args	arguments in the order they are expected
id	A unique id for this block
n_args	Number of arguments for the process function

Methods Summary

__call__(*args)	Allows any block to act as a callable, aliasing the process method
check_setup(task_args)	briefly checks setup with the provided task inputs.
copy()	fetches a shallow copy of this block with the UUID updated
deepcopy()	fetches a deep copy of this block with the UUID updated
enforce(arg[, types, shapes, containers])	sets the block up to make sure the given arg is the assigned type and shapes
get_default_node_attrs()	all values must be json serializable
postprocess()	runs after
preprocess()	runs before all batches are processed
process(*data_batches)
rename(name)	renames the block to the given name.

Attributes Documentation

args

arguments in the order they are expected

Type

list of str

id

A unique id for this block

This id is a combination of the block’s non-unique name and part of it’s uuid (last 6 characters by default). The entropy of this id can be increased by increasing ImagePypelines UUID_ORDER variable

Type

str

n_args

Number of arguments for the process function

Type

int

Methods Documentation

__call__(*args)

Allows any block to act as a callable, aliasing the process method

check_setup(task_args)

briefly checks setup with the provided task inputs.

This function can be overloaded to add additional functionality if desired. By default it simply checks if too many or too few arguments were provided.

Parameters

task_args (tuple of str) – Arguments for this task

Note

Be very careful making task-specific modifications to the block setup in this function. It’s called once for every task this block is in. Changes made for one task may not apply to another task.

copy()

fetches a shallow copy of this block with the UUID updated

deepcopy()

fetches a deep copy of this block with the UUID updated

enforce(arg, types=None, shapes=None, containers=None)

sets the block up to make sure the given arg is the assigned type and shapes

Parameters

• arg (str) – name the process function argument you want to enforce checking on

• types (tuple of type) – the types to restrict this argument to. If left as None, then no type checking will be done

• shapes (tuple of type) – the shapes to restrict this argument to. If left as None, then no shape checking will be done

• containers (tuple of type) – the containers to restrict this argument to. If left as None, then no container checking will be done. if batch_type is “each”, then the container is irrelevant and can be safely ignored!

Returns

self

Return type

Block

Note

This function must be called after the parent block is instantiated!

That is, in your __init__ function, you must call super().__init__ before calling self.enforce

get_default_node_attrs()

all values must be json serializable

postprocess()

runs after

all batches are processed

preprocess()

runs before all batches are processed

abstract process(*data_batches)

rename(name)

renames the block to the given name. The id is reset in this process