Validating Complex Types Using Pydantic

30 Minute Talk
Saturday at 2:15 pm in Orchid Ballroom East

For some Python Objects it is difficult to understand its contents, even if type annotations are used. For example, PyTorch's Tensor or Pandas's DataFrame have many different "shapes" to their data structures, but their type annotations are always the same. This can make it very difficult to understand functions that utilize these objects as parameters or return these objects. When reviewing code, it is very common to ask these sorts of questions:

  • What is the shape of the Tensor I need to pass in?
  • What columns does this DataFrame need to have?

In this talk, we will describe how to utilize typing metadata and Pydantic to not only provide additional context for these data structures, but to validate the input/output of functions as well.

Links for the talk:
- Notebook used in talk; dev container compatible
- ds_validator, the package demonstrated in the talk

Presented by

Daniel Schelkoph