Project Architecture

TypeRighter is a toolkit for structuring, validating, and reshaping data.

Using the toolkit means using one or more of the following things:

  • Type: a classification of some data which describes how to verify arbitrary data for coherence.

  • Record: a structure of data that has type instances, called _fields_, for attributes.

  • Schematic: the map of arguments used to instantiate either a Type or a Record.

  • View: a class that let’s you interact with a Record and some data as though it were an object instance.

Metaprogramming

The design of both Type and Record relies on metaprogramming to collect information about the way you choose to use them.

Generally speaking, metaprogramming is a way for programs to treat code like data. We can read, generate, analyze, or transform code, or modify it while running.

More specifically, TypeRighter can inspect the attributes and functions on any type at the moment a user creates one. This allows it to:

  • make lists of all member variables

  • make a list of all functions that start with someprefix_

And with that metadata users can:

  • map out the steps for complex data validation

  • generate a SQL statement automatically

  • easily define datatype conversion pipelines

Metadata

All Type and Record definitions have values for:

  • _validate_functions

  • _schematic

Records use two extra fields:

  • _fields

  • _field_functions

Types

A type’s validate() function will call each function listed in _validate_functions on its input.

The metaclass can be told about new validation functions by adding functions with validate_ as a prefix, ie. validate_uppercase.

class StrictStringType(StringType):
    def validate_strict(self, value):
        ...

Records

Records introduce the concept of a field by associating a name with a type. It adds two fields of metadata to the class definition.

Let’s define a record with a string stored as field s.

class Foo(Record):
    s = StringType(required=True)

Fields defined like this are stored as _fields.

It is also possible to use a function to generate field values.

class Foo(Record):
    def field_s(self):
        return 'an actual string'

Functions that behave like fields have a prefix field_, similar to the behavior for validation functions. This field is stored as _field_functions.

Views

We get views

Errors

Validation throws exceptions and we always throw exceptions