Describe the feature you'd like
There is a discrepancy on passing environment variables in Processing and Estimator. The parameter is called env in Processing and environment in Estimator.
I would like these to be aligned. For backwards compatibility sake, this should probably be manifested through a environment_variables parameter, but any solution would work for me.
The problem
The problem is that we cannot have a unified interface to these entities using **kwargs to pass arguments without manually parsing a parameter ourselves.
Current situation van be something like this if we use env for both situations:
def data_processing(environment: Literal["dev", "preprod", "prod"], **kwargs):
initialize_environment(environment)
return Processing(**kwargs)
def model_training(environment: Literal["dev", "preprod", "prod"], **kwargs):
initialize_environment(environment)
env_vars = kwargs.pop("env")
if env_vars:
kwargs["environment"] = env_vars
return Estimator(**kwargs)
data_processing(environment="dev", env={"MY_VAR": 42})
model_training(environment="dev", env={"MY_VAR": 67})
Or with a more compatible interface
def data_processing(environment: Literal["dev", "preprod", "prod"], **kwargs):
initialize_environment(environment)
env_vars = kwargs.pop("environment_variables")
if env_vars:
kwargs["env"] = env_vars
return Processing(**kwargs)
def model_training(environment: Literal["dev", "preprod", "prod"], **kwargs):
initialize_environment(environment)
env_vars = kwargs.pop("environment_variables")
if env_vars:
kwargs["environment"] = env_vars
return Estimator(**kwargs)
data_processing(environment="dev", environment_variables={"MY_VAR": 42})
model_training(environment="dev", environment_variables={"MY_VAR": 67})
Ideally, we would want it to look like this because both classes accept a environment_variables parameter:
def data_processing(environment: Literal["dev", "preprod", "prod"], **kwargs):
initialize_environment(environment)
return Processing(**kwargs)
def model_training(environment: Literal["dev", "preprod", "prod"], **kwargs):
initialize_environment(environment)
return Estimator(**kwargs)
data_processing(environment="dev", environment_variables={"MY_VAR": 42})
model_training(environment="dev", environment_variables={"MY_VAR": 67})
Describe the feature you'd like
There is a discrepancy on passing environment variables in Processing and Estimator. The parameter is called
envin Processing andenvironmentin Estimator.I would like these to be aligned. For backwards compatibility sake, this should probably be manifested through a
environment_variablesparameter, but any solution would work for me.The problem
The problem is that we cannot have a unified interface to these entities using
**kwargsto pass arguments without manually parsing a parameter ourselves.Current situation van be something like this if we use
envfor both situations:Or with a more compatible interface
Ideally, we would want it to look like this because both classes accept a
environment_variablesparameter: