numpyop

Delete

Bases: NumpyOp

Delete key(s) and their associated values from the data dictionary.

The system has special logic to detect instances of this Op and delete its inputs from the data dictionary.

Parameters:

Name	Type	Description	Default
keys	Union[str, List[str]]	Existing key(s) to be deleted from the data dictionary.	required

Source code in fastestimator\fastestimator\op\numpyop\numpyop.py

@traceable()
class Delete(NumpyOp):
    """Delete key(s) and their associated values from the data dictionary.

    The system has special logic to detect instances of this Op and delete its `inputs` from the data dictionary.

    Args:
        keys: Existing key(s) to be deleted from the data dictionary.
    """
    def __init__(self, keys: Union[str, List[str]], mode: Union[None, str, Iterable[str]] = None) -> None:
        super().__init__(inputs=keys, mode=mode)

    def forward(self, data: Union[np.ndarray, List[np.ndarray]], state: Dict[str, Any]) -> None:
        pass

LambdaOp

Bases: NumpyOp

An Operator that performs any specified function as forward function.

Parameters:

Name	Type	Description	Default
fn	Callable	The function to be executed.	required
inputs	Union[None, str, Iterable[str]]	Key(s) from which to retrieve data from the data dictionary.	None
outputs	Union[None, str, Iterable[str]]	Key(s) under which to write the outputs of this Op back to the data dictionary.	None
mode	Union[None, str, Iterable[str]]	What mode(s) to execute this Op in. For example, "train", "eval", "test", or "infer". To execute regardless of mode, pass None. To execute in all modes except for a particular one, you can pass an argument like "!infer" or "!train".	None

Source code in fastestimator\fastestimator\op\numpyop\numpyop.py

@traceable()
class LambdaOp(NumpyOp):
    """An Operator that performs any specified function as forward function.

    Args:
        fn: The function to be executed.
        inputs: Key(s) from which to retrieve data from the data dictionary.
        outputs: Key(s) under which to write the outputs of this Op back to the data dictionary.
        mode: What mode(s) to execute this Op in. For example, "train", "eval", "test", or "infer". To execute
            regardless of mode, pass None. To execute in all modes except for a particular one, you can pass an argument
            like "!infer" or "!train".
    """
    def __init__(self,
                 fn: Callable,
                 inputs: Union[None, str, Iterable[str]] = None,
                 outputs: Union[None, str, Iterable[str]] = None,
                 mode: Union[None, str, Iterable[str]] = None):
        super().__init__(inputs=inputs, outputs=outputs, mode=mode)
        self.fn = fn
        self.in_list = True

    def forward(self, data: List[np.ndarray], state: Dict[str, Any]) -> Union[np.ndarray, List[np.ndarray]]:
        return self.fn(*data)

NumpyOp

Bases: Op

An Operator class which takes and returns numpy data.

These Operators are used in fe.Pipeline to perform data pre-processing / augmentation.

Source code in fastestimator\fastestimator\op\numpyop\numpyop.py

@traceable()
class NumpyOp(Op):
    """An Operator class which takes and returns numpy data.

    These Operators are used in fe.Pipeline to perform data pre-processing / augmentation.
    """
    def forward(self, data: Union[np.ndarray, List[np.ndarray]],
                state: Dict[str, Any]) -> Union[np.ndarray, List[np.ndarray]]:
        """A method which will be invoked in order to transform data.

        This method will be invoked on individual elements of data before any batching / axis expansion is performed.

        Args:
            data: The arrays from the data dictionary corresponding to whatever keys this Op declares as its `inputs`.
            state: Information about the current execution context, for example {"mode": "train"}.

        Returns:
            The `data` after applying whatever transform this Op is responsible for. It will be written into the data
            dictionary based on whatever keys this Op declares as its `outputs`.
        """
        return data

forward

A method which will be invoked in order to transform data.

This method will be invoked on individual elements of data before any batching / axis expansion is performed.

Parameters:

Name	Type	Description	Default
data	Union[np.ndarray, List[np.ndarray]]	The arrays from the data dictionary corresponding to whatever keys this Op declares as its inputs.	required
state	Dict[str, Any]	Information about the current execution context, for example {"mode": "train"}.	required

Returns:

Type	Description
Union[np.ndarray, List[np.ndarray]]	The data after applying whatever transform this Op is responsible for. It will be written into the data
Union[np.ndarray, List[np.ndarray]]	dictionary based on whatever keys this Op declares as its outputs.

Source code in fastestimator\fastestimator\op\numpyop\numpyop.py

def forward(self, data: Union[np.ndarray, List[np.ndarray]],
            state: Dict[str, Any]) -> Union[np.ndarray, List[np.ndarray]]:
    """A method which will be invoked in order to transform data.

    This method will be invoked on individual elements of data before any batching / axis expansion is performed.

    Args:
        data: The arrays from the data dictionary corresponding to whatever keys this Op declares as its `inputs`.
        state: Information about the current execution context, for example {"mode": "train"}.

    Returns:
        The `data` after applying whatever transform this Op is responsible for. It will be written into the data
        dictionary based on whatever keys this Op declares as its `outputs`.
    """
    return data

forward_numpyop

Call the forward function for list of NumpyOps, and modify the data dictionary in place.

Parameters:

Name	Type	Description	Default
ops	List[NumpyOp]	A list of NumpyOps to execute.	required
data	MutableMapping[str, Any]	The data dictionary.	required
mode	str	The current execution mode ("train", "eval", "test", or "infer").	required

Source code in fastestimator\fastestimator\op\numpyop\numpyop.py

def forward_numpyop(ops: List[NumpyOp], data: MutableMapping[str, Any], mode: str) -> None:
    """Call the forward function for list of NumpyOps, and modify the data dictionary in place.

    Args:
        ops: A list of NumpyOps to execute.
        data: The data dictionary.
        mode: The current execution mode ("train", "eval", "test", or "infer").
    """
    for op in ops:
        op_data = get_inputs_by_op(op, data)
        op_data = op.forward(op_data, {"mode": mode})
        if isinstance(op, Delete):
            for key in op.inputs:
                del data[key]
        if op.outputs:
            write_outputs_by_op(op, data, op_data)