Support environment variables in config

[innerlee]

Motivation

Use environment variable to dynamically change parts of the config.
Kind of a template functionality.

Modification

Please briefly describe what modification is made in this PR.

BC-breaking (Optional)

no breaking change.

Use cases (Optional)

If you want to test many datasets on a checkpoint, and the only difference is the dataset path, then you can use environment variable as a placeholder. No need to create many configs anymore.

Before:

cfg1.py

...
data_root = 'data/dataset1'
...

cfg2.py

...
data_root = 'data/dataset2'
...

After:

cfg.py

...
data_root = '{{ _env_.DATA_ROOT }}'
...

Checklist

Tests and docs will be added later. Lets see the general reaction first.

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
• CLA has been signed and all committers have signed the CLA in this PR.

[zhouzaida]

Hi @innerlee , you can merge the upstream master to your branch to fix the error of building documentation

[zhouzaida]

https://mmdetection.readthedocs.io/en/latest/1_exist_data_model.html

Can we use the merge_from_dict to implement the same behavior like --cfg-options checkpoint_config.max_keep_ckpts=1

mmcv/mmcv/utils/config.py

Line 550 in 44e7eee

def merge_from_dict(self, options, allow_list_keys=True):

[innerlee]

• For simple replacement, yes. But the --cfg-options thing is verbose and hard to use
• For complex cfgs such as multiple references, the --cfg-options is not powerful enough. Try this example:

[zhouzaida]

• For simple replacement, yes. But the --cfg-options thing is verbose and hard to use
• For complex cfgs such as multiple references, the --cfg-options is not powerful enough. Try this example:

Got it. Great feature.

[zhouzaida]

We also need to update the usage of Config (https://mmcv.readthedocs.io/en/latest/understand_mmcv/config.html) and add some unit tests for the feature.

[MeowZheng]

Should we add argument use_environment_variables for fromfile function, as it call _file2dict

[zhouzaida]

Hi @innerlee , is there any progress?

[innerlee]

Sorry being busy. Will complete it when I got time

[zhouzaida]

Suggested change

	with pytest.raises(Exception):
	with pytest.raises(ValueError, match='...'):

[zhouzaida]

Adding the match argument here will help us know why it should raise an exception.

[zhouzaida]

Hi, is any progess?