Skip to content

Integration test fails to run evaluation #1790

@mtar

Description

@mtar

What happened?

Expected Result: Running the integration test the inference part should run through

Actual Result: The inference part errors with model_path not found in the configuration file.

Traceback (most recent call last):
  File "/.../weathergenerator/WeatherGenerator/src/weathergen/run_train.py", line 79, in inference_from_args
    trainer.inference(cf, devices, args.from_run_id, args.mini_epoch)
  File "/.../weathergenerator/WeatherGenerator/src/weathergen/train/trainer.py", line 200, in inference
    self.model, self.model_params = init_model_and_shard(
                                    ^^^^^^^^^^^^^^^^^^^^^
  File "/.../weathergenerator/WeatherGenerator/src/weathergen/model/model_interface.py", line 159, in init_model_and_shard
    model = load_model(cf, model, device, run_id_contd, mini_epoch_contd)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../weathergenerator/WeatherGenerator/src/weathergen/model/model_interface.py", line 176, in load_model
    path_run = Path(cf.model_path) / run_id
                    ^^^^^^^^^^^^^
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 355, in __getattr__
    self._format_and_raise(
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/base.py", line 231, in _format_and_raise
    format_and_raise(
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/_utils.py", line 899, in format_and_raise
    _raise(ex, cause)
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/_utils.py", line 797, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 351, in __getattr__
    return self._get_impl(
           ^^^^^^^^^^^^^^^
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 442, in _get_impl
    node = self._get_child(
           ^^^^^^^^^^^^^^^^
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/basecontainer.py", line 73, in _get_child
    child = self._get_node(
            ^^^^^^^^^^^^^^^
  File "/.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py", line 480, in _get_node
    raise ConfigKeyError(f"Missing key {key!s}")
omegaconf.errors.ConfigAttributeError: Missing key model_path
    full_key: model_path
    object_type=dict
[11] > /.../weathergenerator/WeatherGenerator/.venv/lib/python3.12/site-packages/omegaconf/dictconfig.py(480)_get_node()
-> raise ConfigKeyError(f"Missing key {key!s}")

What are the steps to reproduce the bug?

HPC: JUWELS (prop. others except alps-clariden / santis)

Hedgedoc link to logs and more information. This ticket is public, do not attach files directly.

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinginfraIssues related to infrastructure

Type

No type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions