`environment_mapping`

Sometimes, Transformers use libraries or frameworks that assume environment that configuration will be a particular way. boto3 in Python and Spring Boot in Java are good examples. In order to reduce the number of code changes required to bring an external application into the SDL Pipeline Engine, such as re-mapping environment variables to match a naming scheme, an optional environment_mapping configuration can be specified in a Transformer’s contract.

Example

// ...
// Each mapping is <source name> -> <destination name>
"configuration": {
  "environment": [],
  "engine_provided_environment": [],
  "environment_mapping": {
    "MINIO_ACCESS_KEY": "AWS_ACCESS_KEY_ID",
    "MINIO_SECRET_KEY": "AWS_SECRET_ACCESS_KEY",
    "MINIO_ENDPOINT": "AWS_ENDPOINT_URL"
  }
}
// ...

The standard environment variables provided by a MinIO connection will be mapped to new names when the Transformer is instantiated. Note that these mappings can use any environment variables that are provided to the Transformer. The mapping is performed source → destination, meaning at most an environment variable is mapped a single time. The decision to go source → destination instead of the reverse was made to reduce environment variable bloat, i.e. multiple environment variables in a running Transformer with the exact same value. If doing so is desired (for example, if using multiple libraries who each assume different names for an environment variable containing a common configuration), then it is the Transformer author’s responsibility to perform any additional mapping upon startup.

When an environment variable is mapped into a new name, the previous variable will not be supplied to the Transformer. In the example above, the Transformer will not see MINIO_ACCESS_KEY, MINIO_SECRET_KEY, or MINIO_ENDPOINT in its environment.