123table

a containerized data loader for db tables

View project on GitHub

Configuring the field mapper

Though 123table is not an ETL tool, it supports remapping fields to a different name or transforming the original value to a different one in the written rows.

This can be done defining a mapper, that is a list of field mappings, where each field mapping possibly specifies

  • the source field name
  • the target field name
  • the field type
  • how the target value is computed from the original one.

You can easily pass a mapper to 123table simply setting the env variable MAPPER_FILE to the path of a json file containing the mappings list.

Mapper format

The mapper is specified as a JSON list of objects. Each object supports the following properties:

from
Name of the source field (Mandatory unless using name)
to
Name of the target field. (Mandatory unless using name)
name
Common name for both source and target field. An alternative to from/to when the name is the same but the value should be computed.
expr
A simple groovy expression (aka formula) deriving a value from the original one (available as variable orig).
i.e.
orig.toLowerCase()
calc
A calculator (a groovy closure) deriving a value from the original one (available as orig along with the whole row map).
i.e.
{ def orig, def row ->
    "${orig.toLowerCase()} (${row.surname})"
}

How to specify a mapper

A mapper may be passed as

  • --mapper flag or MAPPER env var a json value (useful mainly for small mappers)
  • --mapper-file flag or MAPPER_FILE env var: the path to a json file (provided that it is available to the container via bind mount or a volume)

Example:

docker run ... \
  --mapper '[{ "name": "surname", "expr": "orig.capitalize()" }]