Research does'nt work correctly

[NicolasDelage]

Hello

I have a problem with the search logs.
Filtering by 5 minutes works but as soon as I put more important values like 1 day, I lose a lot of information. And coupling the manual search with 1 day never works. Is there a way to fix this?

(I use the desktop version on ubuntu.)

Thanks

[mnapoli]

@NicolasDelage thanks for the report!

When loading logs over 1 day, do you expect a lot of results?

Why I am asking: the Dashboard calls the CloudWatch API to load the logs. But it doesn't log all the logs, as there could be hundreds of MB of logs during the period. So it stops loading after a limit. Does that make sense?

However, searching for a specific line should work. Let's imagine you have 100MB of logs in 1 day, if you don't search for anything specific, it will not load all 100MB of logs.

If you search for a word that is very common (e.g. error) and there are thousands of lines matching, then it's possible that not all results are displayed (again, because it's over the limit) -> is that what is happening here?

BUT if you search for a word and only 1 line matches (for example), then the 1 line should display in the search results (because it's much less than the limit). -> is that working for you or not?

[NicolasDelage]

It makes sense indeed! Thanks for the explanation.

However, when I try to find an error, for example, a 500 that arrived last night. (I have the logs from another app)
I look for "500" for 1 day or 2 days to test. I only get some results, which I think are far from reaching the limit. And I don't go back to find my error 500.

Example :

[mnapoli]

👍 I'll investigate that!

[mnapoli]

Hey @NicolasDelage, sorry for the delay. I am finally working on a fix that should address that. It involves completely rewriting the way logs are queried.

Here's an overview:

Assuming that the new search works perfectly for older logs, do you think that is OK if newer logs take 60 seconds to appear in the UI? (this is the compromise of the fix I am working on)

[mnapoli]

@NicolasDelage the new log viewer/search has been deployed! This should address the issues you have been experiencing.

More details here: https://github.com/brefphp/dashboard/blob/main/change-log.md#2023-06-19