Validating Complex Types Using Pydantic
30 Minute Talk
Saturday at 2:15 pm in Orchid Ballroom East
For some Python Objects it is difficult to understand its contents, even if type annotations are used. For example, PyTorch's Tensor
or Pandas's DataFrame
have many different "shapes" to their data structures, but their type annotations are always the same. This can make it very difficult to understand functions that utilize these objects as parameters or return these objects. When reviewing code, it is very common to ask these sorts of questions:
- What is the shape of the
Tensor
I need to pass in? - What columns does this
DataFrame
need to have?
In this talk, we will describe how to utilize typing metadata and Pydantic to not only provide additional context for these data structures, but to validate the input/output of functions as well.
Links for the talk:
- Notebook used in talk; dev container compatible
- ds_validator, the package demonstrated in the talk