plumpy.persistence module¶
-
class
plumpy.persistence.
Bundle
(savable: plumpy.persistence.Savable, save_context: Optional[LoadSaveContext] = None, dereference: bool = False)[source]¶ Bases:
dict
-
unbundle
(load_context: Optional[LoadSaveContext] = None) → plumpy.persistence.Savable[source]¶ This method loads the class of the object and calls its recreate_from method passing the positional and keyword arguments.
- Parameters
load_context – The optional load context
- Returns
An instance of the Savable
-
-
class
plumpy.persistence.
InMemoryPersister
(loader: Optional[plumpy.loaders.ObjectLoader] = None)[source]¶ Bases:
plumpy.persistence.Persister
Mainly to be used in testing/debugging
-
_abc_impl
= <_abc_data object>¶
-
delete_checkpoint
(pid: Hashable, tag: Optional[str] = None) → None[source]¶ Delete a persisted process checkpoint. No error will be raised if the checkpoint does not exist
- Parameters
pid – the process id of the
plumpy.Process
tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process
-
delete_process_checkpoints
(pid: Hashable) → None[source]¶ Delete all persisted checkpoints related to the given process id
- Parameters
pid – the process id of the
plumpy.Process
-
get_checkpoints
() → List[plumpy.persistence.PersistedCheckpoint][source]¶ Return a list of all the current persisted process checkpoints with each element containing the process id and optional checkpoint tag
- Returns
list of PersistedCheckpoint
-
get_process_checkpoints
(pid: Hashable) → List[plumpy.persistence.PersistedCheckpoint][source]¶ Return a list of all the current persisted process checkpoints for the specified process with each element containing the process id and optional checkpoint tag
- Parameters
pid – the process pid
- Returns
list of PersistedCheckpoint tuples
-
load_checkpoint
(pid: Hashable, tag: Optional[str] = None) → plumpy.persistence.Bundle[source]¶ Load a process from a persisted checkpoint by its process id
- Parameters
pid – the process id of the
plumpy.Process
tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process
- Returns
a bundle with the process state
- Raises
plumpy.PersistenceError
Raised if there was a problem loading the checkpoint
-
save_checkpoint
(process: Process, tag: Optional[str] = None) → None[source]¶ Persist a Process instance
- Parameters
process –
plumpy.Process
tag – optional checkpoint identifier to allow distinguishing multiple checkpoints for the same process
- Raises
plumpy.PersistenceError
Raised if there was a problem saving the checkpoint
-
-
class
plumpy.persistence.
LoadSaveContext
(loader: Optional[plumpy.loaders.ObjectLoader] = None, **kwargs: Any)[source]¶ Bases:
object
-
copyextend
(**kwargs: Any) → plumpy.persistence.LoadSaveContext[source]¶ Add additional information to the context by making a copy with the new values
-
-
class
plumpy.persistence.
PersistedCheckpoint
(pid, tag)¶ Bases:
tuple
-
_asdict
()¶ Return a new dict which maps field names to their values.
-
_field_defaults
= {}¶
-
_fields
= ('pid', 'tag')¶
-
_fields_defaults
= {}¶
-
classmethod
_make
(iterable)¶ Make a new PersistedCheckpoint object from a sequence or iterable
-
_replace
(**kwds)¶ Return a new PersistedCheckpoint object replacing specified fields with new values
-
pid
¶ Alias for field number 0
-
tag
¶ Alias for field number 1
-
-
class
plumpy.persistence.
Persister
[source]¶ Bases:
object
-
_abc_impl
= <_abc_data object>¶
-
abstract
delete_checkpoint
(pid: Hashable, tag: Optional[str] = None) → None[source]¶ Delete a persisted process checkpoint. No error will be raised if the checkpoint does not exist
- Parameters
pid – the process id of the
plumpy.Process
tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process
-
abstract
delete_process_checkpoints
(pid: Hashable) → None[source]¶ Delete all persisted checkpoints related to the given process id
- Parameters
pid – the process id of the
plumpy.Process
-
abstract
get_checkpoints
() → List[plumpy.persistence.PersistedCheckpoint][source]¶ Return a list of all the current persisted process checkpoints with each element containing the process id and optional checkpoint tag
- Returns
list of PersistedCheckpoint
-
abstract
get_process_checkpoints
(pid: Hashable) → List[plumpy.persistence.PersistedCheckpoint][source]¶ Return a list of all the current persisted process checkpoints for the specified process with each element containing the process id and optional checkpoint tag
- Parameters
pid – the process pid
- Returns
list of PersistedCheckpoint tuples
-
abstract
load_checkpoint
(pid: Hashable, tag: Optional[str] = None) → plumpy.persistence.Bundle[source]¶ Load a process from a persisted checkpoint by its process id
- Parameters
pid – the process id of the
plumpy.Process
tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process
- Returns
a bundle with the process state
- Raises
plumpy.PersistenceError
Raised if there was a problem loading the checkpoint
-
abstract
save_checkpoint
(process: Process, tag: Optional[str] = None) → None[source]¶ Persist a Process instance
- Parameters
process –
plumpy.Process
tag – optional checkpoint identifier to allow distinguishing multiple checkpoints for the same process
- Raises
plumpy.PersistenceError
Raised if there was a problem saving the checkpoint
-
-
class
plumpy.persistence.
PicklePersister
(pickle_directory: str)[source]¶ Bases:
plumpy.persistence.Persister
Implementation of the abstract Persister class that stores Process states in pickles on a filesystem.
-
_abc_impl
= <_abc_data object>¶
-
_pickle_filepath
(pid: Hashable, tag: Optional[str] = None) → str[source]¶ Returns the full filepath of the pickle for the given process id and optional checkpoint tag
-
delete_checkpoint
(pid: Hashable, tag: Optional[str] = None) → None[source]¶ Delete a persisted process checkpoint. No error will be raised if the checkpoint does not exist
- Parameters
pid – the process id of the
plumpy.Process
tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process
-
delete_process_checkpoints
(pid: Hashable) → None[source]¶ Delete all persisted checkpoints related to the given process id
- Parameters
pid – the process id of the
plumpy.Process
-
static
ensure_pickle_directory
(dirpath: str) → None[source]¶ Will attempt to create the directory at dirpath and raise if it fails, except if the exception arose because the directory already existed
-
get_checkpoints
() → List[plumpy.persistence.PersistedCheckpoint][source]¶ Return a list of all the current persisted process checkpoints with each element containing the process id and optional checkpoint tag
- Returns
list of PersistedCheckpoint
-
get_process_checkpoints
(pid: Hashable) → List[plumpy.persistence.PersistedCheckpoint][source]¶ Return a list of all the current persisted process checkpoints for the specified process with each element containing the process id and optional checkpoint tag
- Parameters
pid – the process pid
- Returns
list of PersistedCheckpoint
-
load_checkpoint
(pid: Hashable, tag: Optional[str] = None) → plumpy.persistence.Bundle[source]¶ Load a process from a persisted checkpoint by its process id
- Parameters
pid – the process id of the
plumpy.Process
tag – optional checkpoint identifier to allow retrieving a specific sub checkpoint for the corresponding process
- Returns
a bundle with the process state
-
static
load_pickle
(filepath: str) → plumpy.persistence.PersistedPickle[source]¶ Load a pickle from disk
- Parameters
filepath – absolute filepath to the pickle
- Returns
the loaded pickle
-
static
pickle_filename
(pid: Hashable, tag: Optional[str] = None) → str[source]¶ Returns the relative filepath of the pickle for the given process id and optional checkpoint tag
-
save_checkpoint
(process: Process, tag: Optional[str] = None) → None[source]¶ Persist a process to a pickle on disk
- Parameters
process –
plumpy.Process
tag – optional checkpoint identifier to allow distinguishing multiple checkpoints for the same process
-
-
class
plumpy.persistence.
Savable
[source]¶ Bases:
object
-
_get_value
(saved_state: MutableMapping[str, Any], name: str, load_context: Optional[plumpy.persistence.LoadSaveContext]) → Union[method, plumpy.persistence.Savable][source]¶
-
_persist_configured
= False¶
-
static
_set_meta_type
(out_state: MutableMapping[str, Any], name: str, type_spec: Any) → None[source]¶
-
static
load
(saved_state: MutableMapping[str, Any], load_context: Optional[plumpy.persistence.LoadSaveContext] = None) → plumpy.persistence.Savable[source]¶ Load a Savable from a saved instance state. The load context is a way of passing runtime data to the object being loaded.
- Parameters
saved_state – The saved state
load_context – Additional runtime state that can be passed into when loading. The type and content (if any) is completely user defined
- Returns
The loaded Savable instance
-
load_members
(members: Iterable[str], saved_state: MutableMapping[str, Any], load_context: Optional[plumpy.persistence.LoadSaveContext] = None) → None[source]¶
-
classmethod
recreate_from
(saved_state: MutableMapping[str, Any], load_context: Optional[plumpy.persistence.LoadSaveContext] = None) → plumpy.persistence.Savable[source]¶ Recreate a
Savable
from a saved state using an optional load context.- Parameters
saved_state – The saved state
load_context – An optional load context
- Returns
The recreated instance
-
save
(save_context: Optional[plumpy.persistence.LoadSaveContext] = None) → MutableMapping[str, Any][source]¶
-
-
class
plumpy.persistence.
SavableFuture
(*, loop=None)[source]¶ Bases:
_asyncio.Future
,plumpy.persistence.Savable
A savable future.
-
load_instance_state
(saved_state: MutableMapping[str, Any], load_context: plumpy.persistence.LoadSaveContext) → None[source]¶
-
classmethod
recreate_from
(saved_state: MutableMapping[str, Any], load_context: Optional[plumpy.persistence.LoadSaveContext] = None) → plumpy.persistence.Savable[source]¶ Recreate a
Savable
from a saved state using an optional load context.- Parameters
saved_state – The saved state
load_context – An optional load context
- Returns
The recreated instance
-
save_instance_state
(out_state: MutableMapping[str, Any], save_context: plumpy.persistence.LoadSaveContext) → None[source]¶
-
-
plumpy.persistence.
auto_persist
(*members: str) → Callable[[Type[plumpy.persistence.Savable]], Type[plumpy.persistence.Savable]][source]¶