You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
💡 The Idea
Apprise needs to support persistent storage.
It requires a nice library like the Attachments, and Configuration that the Plugin library can readily/easily access.
The persistent storage should be a switch that defaults to being off, but can be toggled to on from the AppriseAsset() object (storage_mode="memory" by default)
The Persistent storage would be adapted by the NotifyBase object making it accessible in every plugins environment without the need of an extra import.
The CLI tool has Persistent Storage enabled by default.
The CLI tool provides an extra --storage-mode which can be set to auto (default), flush, and memory.
auto: This is the default option and pesistent storage is used when applicable (only the plugins that require it take advantage of local cache made available to them).
flush: Similar to auto except that any changes made are immediately flushed to disk. This mode creates a higher i/o but enforces the content on disk is the latest.
memory: Effectively turns off Persistent storage. No plugins are allowed to write to disk. This is exactly the way Apprise was prior ro the Persistent Storage feature.
To remove all accumulated persistent storage generated through the CLI tool, you can run the following:
apprise storage clean
You can compliment this call by providing URL IDs and/or --tag (or -g) values to focus on only cleaning specific persistently cached data. For example:
# Assuming we want to target the URL ID of abc123xy
apprise storage clean abc123xy
You can also clear cache based on tag references:
# Assuming we want to target the URL(s) associated with the tag 'family'
apprise storage clean --tag family
The current version of Apprise would be written to disk in the persisent storage space (if enabled by the AppriseAsset() object). Subsequent runs of Apprise would then check this version and compare it to the current one running. If they differ, all persistent data would be reset/cleared and the version file updated.
Note: This will allow newer versions of Apprise to introduce new schemas and saving mechanisms and not be broken due to past/earlier releases.
Today the use cases for Persistent Storage are:
Twitter IDs: Today a clean application entrance of Apprise to a Twitter endpoint requires the system to lookup the @user and get it's ID (An extra GET Request). From there it hangs onto it in memory for future notifications. With Persistent storage, this can be written to disk and save future hits even when the program is restarted from scratch. This would be a noticeable performance increase
Mastodon: Only for DM's to ourselves (visibility=direct) requires an additional hit to the Mastodon API to retrieve our information. Caching this information would be 1 less URL hit later on.
Opsgenie: Tracking the last message sent would allow subsequent messages to auto-close the previous;
classMyNotification(NotifyBase):
# ...defsend(self, body, *args, **kwargs):
# Value is returned from memory if stored there# if not found, and persistent storage is enabled, then the value is read from disk and stored into memory# The value is then returned# `None` is returned if content does not exist in either locationvalue=self.store.get('key')
# Store key/value pair in memory (and Disk if persistent flag is set)self.store.set('key', 'value')
## Some extra options you can use with the set()## Invalidate the data (forcing get() to return None after time is reached)self.store.set('key', 'value', expires=datetime(now) +timedelta(hours=1))
# By default persistent will always be set to True, but perhaps you don't want the content to# persist and only exist for the life of the application instead.self.store.set('key', 'value', persistent=False)
# Flushes elements to disk if they are configured to persist there; this # never needs to be called directly as the PersistentStore object looks# after this automatically. Those set to `auto` (default) mode can# leverage this to force an early write/sync to diskself.store.flush()
# The following would both flush the key from memory and persistent# storage if present.self.store.clear('key')
# Similar to flush, this works as well to provide a focused group clearself.store.clear('key1', 'key2', 'key3', ...)
# Remove all entries in memory and all entries in the persistent storeself.store.clear()
# Allow Apprise to persist files outside of key/value pairs as well. Such as maybe# an initial online registration that occurs and it needs to store a private/public# key.withself.store.open("key", "wb") asfp:
fp.write()
withself.store.open("key", "rb") asfp:
content=fp.read()
# To remove all persistent files you could use:self.store.clear()
# For consistency with the rest of the above logic:, this would clear a fileself.store.delete('key')
# multiple keys can be cleared if requiredself.store.delete('key1', 'key2', 'key3')
Persistent Store Details
I was thinking of the following; but am open for discussion:
Use an sqlite database, or perhaps just convert content to JSON and write straight to disk (+ zip) might be easier.
sha1() of Apprise URL would id the directory/namespace it all content would write to. e.g: 549f6c03f0fb0095e02a0e68deb1b49e149f29d6
key/value pairs using set(), get(), and clear() would write to a file called db stored in the namespace;
- e.g: %{namespace}/db would be something like: 549f6c03f0fb0095e02a0e68deb1b49e149f29d6/db
This would allow multiple variations of a URL to write to their own persistent store.
data written with self.persistent.open() would be written to the same %{namespace} but would reside in a sub directory called persist.
- e.g: %{namespace}/persist/ would be something like: 549f6c03f0fb0095e02a0e68deb1b49e149f29d6/persist/
Namespaces would all reside in the following path (additionally prefixed with a %{schema} such as discord or twitter, etc to make it easier to further troubleshoot or allow users to clear specific persisten directories..
the sha1() generated namespace can be overridden by the Notify() object itself through a defined namespace(self) function. Users can further look at the variables provided (ignoring extra kwargs that should not change the variance of the persist directory) and return a more specific/focused (and re-usable) location. Otherwise the parent NotifyBase() will return the general sha1(self.url()) one.
🔨 Breaking Feature
The functions/integration should be able to be implemented in such a way there would be no breaking change.
This may introduce plugins that depend on persistent storage to work and would have to be reported disabled via the Apprise.details() when the flag is off.
The text was updated successfully, but these errors were encountered:
💡 The Idea
Apprise needs to support persistent storage.
It requires a nice library like the Attachments, and Configuration that the Plugin library can readily/easily access.
The persistent storage should be a switch that defaults to being off, but can be toggled to
on
from theAppriseAsset()
object (storage_mode="memory"
by default)The Persistent storage would be adapted by the
NotifyBase
object making it accessible in every plugins environment without the need of an extra import.The CLI tool has Persistent Storage enabled by default.
The CLI tool provides an extra
--storage-mode
which can be set toauto
(default),flush
, andmemory
.auto
: This is the default option and pesistent storage is used when applicable (only the plugins that require it take advantage of local cache made available to them).flush
: Similar toauto
except that any changes made are immediately flushed to disk. This mode creates a higher i/o but enforces the content on disk is the latest.memory
: Effectively turns off Persistent storage. No plugins are allowed to write to disk. This is exactly the way Apprise was prior ro the Persistent Storage feature.To remove all accumulated persistent storage generated through the CLI tool, you can run the following:
You can compliment this call by providing URL IDs and/or
--tag
(or-g
) values to focus on only cleaning specific persistently cached data. For example:# Assuming we want to target the URL ID of abc123xy apprise storage clean abc123xy
You can also clear cache based on tag references:
# Assuming we want to target the URL(s) associated with the tag 'family' apprise storage clean --tag family
The current version of Apprise would be written to disk in the persisent storage space (if enabled by the AppriseAsset() object). Subsequent runs of Apprise would then check this version and compare it to the current one running. If they differ, all persistent data would be reset/cleared and the version file updated.
Today the use cases for Persistent Storage are:
@user
and get it's ID (An extra GET Request). From there it hangs onto it in memory for future notifications. With Persistent storage, this can be written to disk and save future hits even when the program is restarted from scratch. This would be a noticeable performance increasevisibility=direct
) requires an additional hit to the Mastodon API to retrieve our information. Caching this information would be 1 less URL hit later on.#640 would tie back to this enhancement as well.
Plugin Logic
Persistent Store Details
I was thinking of the following; but am open for discussion:
Use an sqlite database, or perhaps just convert content to JSON and write straight to disk (+ zip) might be easier.
sha1()
of Apprise URL would id the directory/namespace it all content would write to. e.g:549f6c03f0fb0095e02a0e68deb1b49e149f29d6
set()
,get()
, andclear()
would write to a file calleddb
stored in the namespace;- e.g:
%{namespace}/db
would be something like:549f6c03f0fb0095e02a0e68deb1b49e149f29d6/db
self.persistent.open()
would be written to the same%{namespace}
but would reside in a sub directory calledpersist
.- e.g:
%{namespace}/persist/
would be something like:549f6c03f0fb0095e02a0e68deb1b49e149f29d6/persist/
Namespaces would all reside in the following path (additionally prefixed with a
%{schema}
such asdiscord
ortwitter
, etc to make it easier to further troubleshoot or allow users to clear specific persisten directories..~/.apprise/persist/%{schema}/%{namespace}/
(Linux)~/.config/apprise/persist/%{schema}/%{namespace}/
(Linux)%APPDATA%/Apprise/persist/%{schema}/%{namespace}/
(Windows)%LOCALAPPDATA%/Apprise/persist/%{schema}/%{namespace}/
(Windows)the
sha1()
generated namespace can be overridden by the Notify() object itself through a definednamespace(self)
function. Users can further look at the variables provided (ignoring extra kwargs that should not change the variance of thepersist
directory) and return a more specific/focused (and re-usable) location. Otherwise the parentNotifyBase()
will return the generalsha1(self.url())
one.🔨 Breaking Feature
The functions/integration should be able to be implemented in such a way there would be no breaking change.
This may introduce plugins that depend on persistent storage to work and would have to be reported disabled via the
Apprise.details()
when the flag is off.The text was updated successfully, but these errors were encountered: