Skip to content

numpyop

Delete

Bases: NumpyOp

Delete key(s) and their associated values from the data dictionary.

The system has special logic to detect instances of this Op and delete its inputs from the data dictionary.

Parameters:

Name Type Description Default
keys Union[str, List[str]]

Existing key(s) to be deleted from the data dictionary.

 required
Source code in fastestimator\fastestimator\op\numpyop\numpyop.py 
44
45
46
47
48
49
50
51
52
53
54
55
56
class Delete(NumpyOp):
    """Delete key(s) and their associated values from the data dictionary.

    The system has special logic to detect instances of this Op and delete its `inputs` from the data dictionary.

    Args:
        keys: Existing key(s) to be deleted from the data dictionary.
    """
    def __init__(self, keys: Union[str, List[str]], mode: Union[None, str, Iterable[str]] = None) -> None:
        super().__init__(inputs=keys, mode=mode)

    def forward(self, data: Union[np.ndarray, List[np.ndarray]], state: Dict[str, Any]) -> None:
        pass

NumpyOp

Bases: Op

An Operator class which takes and returns numpy data.

These Operators are used in fe.Pipeline to perform data pre-processing / augmentation.

Source code in fastestimator\fastestimator\op\numpyop\numpyop.py 
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
class NumpyOp(Op):
    """An Operator class which takes and returns numpy data.

    These Operators are used in fe.Pipeline to perform data pre-processing / augmentation.
    """
    def forward(self, data: Union[np.ndarray, List[np.ndarray]],
                state: Dict[str, Any]) -> Union[np.ndarray, List[np.ndarray]]:
        """A method which will be invoked in order to transform data.

        This method will be invoked on individual elements of data before any batching / axis expansion is performed.

        Args:
            data: The arrays from the data dictionary corresponding to whatever keys this Op declares as its `inputs`.
            state: Information about the current execution context, for example {"mode": "train"}.

        Returns:
            The `data` after applying whatever transform this Op is responsible for. It will be written into the data
            dictionary based on whatever keys this Op declares as its `outputs`.
        """
        return data

forward

A method which will be invoked in order to transform data.

This method will be invoked on individual elements of data before any batching / axis expansion is performed.

Parameters:

Name Type Description Default
data Union[np.ndarray, List[np.ndarray]]

The arrays from the data dictionary corresponding to whatever keys this Op declares as its inputs.

 required
state Dict[str, Any]

Information about the current execution context, for example {"mode": "train"}.

 required

Returns:

Type Description
Union[np.ndarray, List[np.ndarray]]

The data after applying whatever transform this Op is responsible for. It will be written into the data
Union[np.ndarray, List[np.ndarray]]

dictionary based on whatever keys this Op declares as its outputs.
Source code in fastestimator\fastestimator\op\numpyop\numpyop.py 
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
def forward(self, data: Union[np.ndarray, List[np.ndarray]],
            state: Dict[str, Any]) -> Union[np.ndarray, List[np.ndarray]]:
    """A method which will be invoked in order to transform data.

    This method will be invoked on individual elements of data before any batching / axis expansion is performed.

    Args:
        data: The arrays from the data dictionary corresponding to whatever keys this Op declares as its `inputs`.
        state: Information about the current execution context, for example {"mode": "train"}.

    Returns:
        The `data` after applying whatever transform this Op is responsible for. It will be written into the data
        dictionary based on whatever keys this Op declares as its `outputs`.
    """
    return data

forward_numpyop

Call the forward function for list of NumpyOps, and modify the data dictionary in place.

Parameters:

Name Type Description Default
ops List[NumpyOp]

A list of NumpyOps to execute.

 required
data MutableMapping[str, Any]

The data dictionary.

 required
mode str

The current execution mode ("train", "eval", "test", or "infer").

 required
Source code in fastestimator\fastestimator\op\numpyop\numpyop.py 
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
def forward_numpyop(ops: List[NumpyOp], data: MutableMapping[str, Any], mode: str) -> None:
    """Call the forward function for list of NumpyOps, and modify the data dictionary in place.

    Args:
        ops: A list of NumpyOps to execute.
        data: The data dictionary.
        mode: The current execution mode ("train", "eval", "test", or "infer").
    """
    for op in ops:
        op_data = get_inputs_by_op(op, data)
        op_data = op.forward(op_data, {"mode": mode})
        if isinstance(op, Delete):
            for key in op.inputs:
                del data[key]
        if op.outputs:
            write_outputs_by_op(op, data, op_data)