plumpy.persistence module

class plumpy.persistence.Bundle(savable: Savable, save_context: LoadSaveContext | None = None, dereference: bool = False)

Bases: dict

unbundle(load_context: LoadSaveContext | None = None) → Savable

This method loads the class of the object and calls its recreate_from method passing the positional and keyword arguments.

Parameters:

load_context – The optional load context

Returns:

An instance of the Savable

class plumpy.persistence.InMemoryPersister(loader: ObjectLoader | None = None)

Bases: Persister

Mainly to be used in testing/debugging

_abc_impl = <_abc._abc_data object>

_checkpoints: Dict[Hashable, Dict[str | None, Bundle]]

delete_checkpoint(pid: Hashable, tag: str | None = None) → None

Delete a persisted process checkpoint. No error will be raised if the checkpoint does not exist

Parameters:

• pid – the process id of the plumpy.Process

• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process

delete_process_checkpoints(pid: Hashable) → None

Delete all persisted checkpoints related to the given process id

Parameters:

pid – the process id of the plumpy.Process

get_checkpoints() → List[PersistedCheckpoint]

Return a list of all the current persisted process checkpoints with each element containing the process id and optional checkpoint tag

Returns:

list of PersistedCheckpoint

get_process_checkpoints(pid: Hashable) → List[PersistedCheckpoint]

Return a list of all the current persisted process checkpoints for the specified process with each element containing the process id and optional checkpoint tag

Parameters:

pid – the process pid

Returns:

list of PersistedCheckpoint tuples

load_checkpoint(pid: Hashable, tag: str | None = None) → Bundle

Load a process from a persisted checkpoint by its process id

Parameters:

• pid – the process id of the plumpy.Process

• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process

Returns:

a bundle with the process state

Raises:

plumpy.PersistenceError Raised if there was a problem loading the checkpoint

save_checkpoint(process: Process, tag: str | None = None) → None

Persist a Process instance

Parameters:

• process – plumpy.Process

• tag – optional checkpoint identifier to allow distinguishing multiple checkpoints for the same process

Raises:

plumpy.PersistenceError Raised if there was a problem saving the checkpoint

class plumpy.persistence.LoadSaveContext(loader: ObjectLoader | None = None, **kwargs: Any)

Bases: object

copyextend(**kwargs: Any) → LoadSaveContext

Add additional information to the context by making a copy with the new values

class plumpy.persistence.PersistedCheckpoint(pid, tag)

Bases: tuple

_asdict()

Return a new dict which maps field names to their values.

_field_defaults = {}

_fields = ('pid', 'tag')

classmethod _make(iterable)

Make a new PersistedCheckpoint object from a sequence or iterable

_replace(**kwds)

Return a new PersistedCheckpoint object replacing specified fields with new values

pid

Alias for field number 0

tag

Alias for field number 1

class plumpy.persistence.Persister

Bases: object

_abc_impl = <_abc._abc_data object>

abstract delete_checkpoint(pid: Hashable, tag: str | None = None) → None

Delete a persisted process checkpoint. No error will be raised if the checkpoint does not exist

Parameters:

• pid – the process id of the plumpy.Process

• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process

abstract delete_process_checkpoints(pid: Hashable) → None

Delete all persisted checkpoints related to the given process id

Parameters:

pid – the process id of the plumpy.Process

abstract get_checkpoints() → List[PersistedCheckpoint]

Return a list of all the current persisted process checkpoints with each element containing the process id and optional checkpoint tag

Returns:

list of PersistedCheckpoint

abstract get_process_checkpoints(pid: Hashable) → List[PersistedCheckpoint]

Return a list of all the current persisted process checkpoints for the specified process with each element containing the process id and optional checkpoint tag

Parameters:

pid – the process pid

Returns:

list of PersistedCheckpoint tuples

abstract load_checkpoint(pid: Hashable, tag: str | None = None) → Bundle

Load a process from a persisted checkpoint by its process id

Parameters:

• pid – the process id of the plumpy.Process

• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process

Returns:

a bundle with the process state

Raises:

plumpy.PersistenceError Raised if there was a problem loading the checkpoint

abstract save_checkpoint(process: Process, tag: str | None = None) → None

Persist a Process instance

Parameters:

• process – plumpy.Process

• tag – optional checkpoint identifier to allow distinguishing multiple checkpoints for the same process

Raises:

plumpy.PersistenceError Raised if there was a problem saving the checkpoint

class plumpy.persistence.PicklePersister(pickle_directory: str)

Bases: Persister

Implementation of the abstract Persister class that stores Process states in pickles on a filesystem.

_abc_impl = <_abc._abc_data object>

_pickle_filepath(pid: Hashable, tag: str | None = None) → str

Returns the full filepath of the pickle for the given process id and optional checkpoint tag

delete_checkpoint(pid: Hashable, tag: str | None = None) → None

Delete a persisted process checkpoint. No error will be raised if the checkpoint does not exist

Parameters:

• pid – the process id of the plumpy.Process

• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process

delete_process_checkpoints(pid: Hashable) → None

Delete all persisted checkpoints related to the given process id

Parameters:

pid – the process id of the plumpy.Process

static ensure_pickle_directory(dirpath: str) → None

Will attempt to create the directory at dirpath and raise if it fails, except if the exception arose because the directory already existed

get_checkpoints() → List[PersistedCheckpoint]

Return a list of all the current persisted process checkpoints with each element containing the process id and optional checkpoint tag

Returns:

list of PersistedCheckpoint

get_process_checkpoints(pid: Hashable) → List[PersistedCheckpoint]

Return a list of all the current persisted process checkpoints for the specified process with each element containing the process id and optional checkpoint tag

Parameters:

pid – the process pid

Returns:

list of PersistedCheckpoint

load_checkpoint(pid: Hashable, tag: str | None = None) → Bundle

Load a process from a persisted checkpoint by its process id

Parameters:

• pid – the process id of the plumpy.Process

• tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process

Returns:

a bundle with the process state

static load_pickle(filepath: str) → PersistedPickle

Load a pickle from disk

Parameters:

filepath – absolute filepath to the pickle

Returns:

the loaded pickle

static pickle_filename(pid: Hashable, tag: str | None = None) → str

Returns the relative filepath of the pickle for the given process id and optional checkpoint tag

save_checkpoint(process: Process, tag: str | None = None) → None

Persist a process to a pickle on disk

Parameters:

• process – plumpy.Process

• tag – optional checkpoint identifier to allow distinguishing multiple checkpoints for the same process

class plumpy.persistence.Savable

Bases: object

CLASS_NAME: str = 'class_name'

_auto_persist: Set[str] | None = None

_ensure_persist_configured() → None

static _get_class_name(saved_state: MutableMapping[str, Any]) → str

static _get_create_meta(out_state: MutableMapping[str, Any]) → Dict[str, Any]

static _get_meta_type(saved_state: MutableMapping[str, Any], name: str) → Any

_get_value(saved_state: MutableMapping[str, Any], name: str, load_context: LoadSaveContext | None) → method | Savable

_persist_configured = False

static _set_class_name(out_state: MutableMapping[str, Any], name: str) → None

static _set_meta_type(out_state: MutableMapping[str, Any], name: str, type_spec: Any) → None

classmethod auto_persist(*members: str) → None

static get_custom_meta(saved_state: MutableMapping[str, Any], name: str) → Any

static load(saved_state: MutableMapping[str, Any], load_context: LoadSaveContext | None = None) → Savable

Load a Savable from a saved instance state. The load context is a way of passing runtime data to the object being loaded.

Parameters:

• saved_state – The saved state

• load_context – Additional runtime state that can be passed into when loading. The type and content (if any) is completely user defined

Returns:

The loaded Savable instance

load_instance_state(*args: Any, **kwargs: Any) → None

load_members(members: Iterable[str], saved_state: MutableMapping[str, Any], load_context: LoadSaveContext | None = None) → None

classmethod persist() → None

classmethod recreate_from(saved_state: MutableMapping[str, Any], load_context: LoadSaveContext | None = None) → Savable

Recreate a Savable from a saved state using an optional load context.

Parameters:

• saved_state – The saved state

• load_context – An optional load context

Returns:

The recreated instance

save(save_context: LoadSaveContext | None = None) → MutableMapping[str, Any]

save_instance_state(*args: Any, **kwargs: Any) → None

save_members(members: Iterable[str], out_state: MutableMapping[str, Any]) → None

static set_custom_meta(out_state: MutableMapping[str, Any], name: str, value: Any) → None

class plumpy.persistence.SavableFuture(*, loop=None)

Bases: Future, Savable

A savable future.

_auto_persist: Set[str] | None = {'_result', '_state'}

load_instance_state(saved_state: MutableMapping[str, Any], load_context: LoadSaveContext) → None

classmethod recreate_from(saved_state: MutableMapping[str, Any], load_context: LoadSaveContext | None = None) → Savable

Recreate a Savable from a saved state using an optional load context.

Parameters:

• saved_state – The saved state

• load_context – An optional load context

Returns:

The recreated instance

save_instance_state(out_state: MutableMapping[str, Any], save_context: LoadSaveContext) → None

plumpy.persistence.auto_persist(*members: str) → Callable[[SavableClsType], SavableClsType]