Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PermissionError when running with Persistance #81

Open
MarinSL opened this issue May 7, 2023 · 0 comments
Open

PermissionError when running with Persistance #81

MarinSL opened this issue May 7, 2023 · 0 comments

Comments

@MarinSL
Copy link

MarinSL commented May 7, 2023

I am running drain on multiple log files with persistence, as I want to later use the state to match newer logs.
However, after a random running time (varying from ~10 mins to ~80 mins), I get a Permission error:

...
Saving state of 1024 clusters with 85804 messages, 248088 bytes, reason: cluster_template_changed (7971)
Saving state of 1024 clusters with 86332 messages, 248052 bytes, reason: cluster_template_changed (7276)
Saving state of 1024 clusters with 86612 messages, 248088 bytes, reason: cluster_created (8046)
Saving state of 1024 clusters with 86668 messages, 248000 bytes, reason: cluster_template_changed (7996)
---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
Cell In[146], line 3
      1 train_csv= 'passed.csv'
      2 for f in filepaths:
----> 3     read_data(f,train_csv)

Cell In[140], line 44, in read_data(location, csv_location, drain, header)
     42             # Add to drain
     43         if drain:
---> 44             template_miner.add_log_message(message_cleaned)
     46 for row in rows:
     47         csv_writer.writerow(row[:-1])

File c:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\drain3\template_miner.py:141, in TemplateMiner.add_log_message(self, log_message)
    139 snapshot_reason = self.get_snapshot_reason(change_type, cluster.cluster_id)
    140 if snapshot_reason:
--> 141     self.save_state(snapshot_reason)
    142     self.last_save_time = time.time()
    143 self.profiler.end_section()

File c:\Users\user\AppData\Local\Programs\Python\Python311\Lib\site-packages\drain3\template_miner.py:107, in TemplateMiner.save_state(self, snapshot_reason)
    102     state = base64.b64encode(zlib.compress(state))
    104 logger.info(f"Saving state of {len(self.drain.clusters)} clusters "
    105             f"with {self.drain.get_total_cluster_size()} messages, {len(state)} bytes, "
...
   1042 if "b" not in mode:
   1043     encoding = io.text_encoding(encoding)
-> 1044 return io.open(self, mode, buffering, encoding, errors, newline)

PermissionError: [Errno 13] Permission denied: 'drain3_state.bin'

The drain state file is not empty, so it seems to be able to access it most of the time.

I set up drain this way:

config = TemplateMinerConfig()
config.load("drain3.ini")

persistence = FilePersistence("drain3_state.bin")

template_miner = TemplateMiner(persistence, config)

logger = logging.getLogger(__name__)
logging.basicConfig(stream=sys.stdout, level=logging.INFO, format='%(message)s')

I am running it using Jupyter extension for Visual Code, if it helps

UPD: Works fine if ran by opening Jupyter Notebook directly, so seems to be a Visual Code thing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant