Tracker

Custom trackers to log data and files during experiments.

Base class for trackers.

class ecgan.evaluation.tracker.base_tracker.BaseTracker(entity, project, run_name, save_pdf=False)[source]

Bases: abc.ABC

Base class for trackers.

advance_step()[source]

Advance by one step.

advance_fold()[source]

Advance by one fold.

abstract close()[source]

Close training run.

static collate_metrics(metrics)[source]

Transform a list with dictionaries to a single dictionary.

All values are collated in their respective keys and are then averaged.

Parameters

metrics (List[Dict]) -- A list of metrics.

Return type

Dict

Returns

Dictionary with the collated metrics.

abstract watch(model)[source]

Pass a module that should be watched during training.

Return type

None

abstract log_config(cfg)[source]

Log parameters for training setup.

Return type

None

abstract log_metrics(metrics)[source]

Take a dictionary with metrics and log content.

Parameters

metrics (Dict) -- Dict in shape {metric: val, ...}

Return type

None

abstract log_checkpoint(module_checkpoint, fold)[source]

Save a module checkpoint.

The checkpoint is a dictionary containing its state dict and optimizer parameters.

Parameters
  • module_checkpoint (Dict) -- Dictionary with model weights.

  • fold (int) -- Current fold. Should be extracted before beginning to upload to avoid concurrency.

Return type

None

abstract log_artifacts(artifacts)[source]

Log dictionary with artifacts.

Parameters

artifacts (Union[Artifact, List[Artifact]]) -- Dictionary containing artifacts to log.

Return type

None

abstract load_config(run_uri)[source]

Load config.

Parameters

run_uri (str) -- Path pointing to project root.

Return type

Dict

Tracker storing data locally.

class ecgan.evaluation.tracker.local_tracker.LocalTracker(entity, project, run_name, save_pdf=True, base_root='results')[source]

Bases: ecgan.evaluation.tracker.base_tracker.BaseTracker

Class to manage calculating metrics and logging them on the local file system.

close()[source]

Close training run.

watch(model)[source]

Watch models during training - not supported in LocalTracker.

Return type

None

log_config(cfg)[source]

Log parameters for training setup.

Return type

None

log_metrics(metrics)[source]

Take a dictionary with metrics and log content.

Parameters

metrics (Dict) -- Dict in shape {metric: val, ...}.

Return type

None

log_checkpoint(module_checkpoint, fold)[source]

Save a module checkpoint.

The checkpoint is a dictionary containing its state dict and optimizer parameters.

Parameters
  • module_checkpoint (Dict) -- Dictionary with model weights.

  • fold (int) -- Current fold. Should be extracted before beginning to upload to avoid concurrency.

Return type

None

log_artifacts(artifacts)[source]

Log dictionary with artifacts.

Parameters

artifacts (Union[Artifact, List[Artifact]]) -- Dictionary containing artifacts to log.

Return type

None

load_config(run_uri)[source]

Load config.

Parameters

run_uri (str) -- Path pointing to project root.

Return type

Dict

Returns

Loaded yml config as dict.

Tracker storing data using weights and biases.

class ecgan.evaluation.tracker.wb_tracker.WBTracker(entity, project, run_name, save_locally=False, save_pdf=False, save_checkpoint_s3=False)[source]

Bases: ecgan.evaluation.tracker.base_tracker.BaseTracker

Class to manage calculating metrics and logging them with Weights & Biases (W&B).

Parameters
  • entity (str) -- Weights and Biases entity name (given by W&B).

  • project (str) -- Weights and Biases project name (chosen by user).

  • run_name (str) -- Weights and Biases run name (chosen by user).

  • save_pdf (bool) -- Flag to save data as pdf ADDITIONALLY to normal plots.

  • save_locally (bool) -- Flag to retain local storage of data.

close()[source]

Close training run.

The uploading might require some time. The run is interrupted if the upload did not complete after 90 seconds. You can manually upload the data afterwards with a local sync.

watch(model)[source]

Pass a module that should be watched during training.

Return type

None

log_config(cfg)[source]

Log parameters for training setup.

Return type

None

log_metrics(metrics)[source]

Take a dictionary with metrics and log content.

Parameters

metrics (Dict) -- Dict in shape {metric: val, ...}.

Return type

None

log_checkpoint(module_checkpoint, fold)[source]

Save a module checkpoint.

The checkpoint is a dictionary containing its state dict and optimizer parameters.

Parameters
  • module_checkpoint (Dict) -- Dictionary with model weights.

  • fold (int) -- Current fold. Should be extracted before beginning to upload to avoid concurrency.

Return type

None

log_artifacts(artifacts)[source]

Log dictionary with artifacts to W&B.

Parameters

artifacts (Union[Artifact, List[Artifact]]) -- Dictionary containing artifacts to log.

Return type

None

load_config(run_uri)[source]

Load config as dict.

Return type

Dict

Tracker factory to retrieve the instance of a tracker defined in the configuration file.

class ecgan.evaluation.tracker.tracker_factory.TrackerFactory[source]

Bases: object

Create desired tracker.