[BUG] Queries take too long

[lakhoune]

Many queries take too long for the server to respond and just timeout. This is especially the case if the QV service needs to go through every single DB entry of a table (This also holds for WHERE clauses that check certain JSON paths).
In most cases, we can just improve the query by specifying the source agent with the $SERVICE$ wildcard.
However, this can not always be done, since we might want to include monitoring data from other services from which we don't know the source agent id. Examples for this are if you want to see the logs that the social bot manager service logs for your bot or you want to get the survey results for a survey done on your service.

[lakhoune]

What we could do here is add more wildcards such as $BOT_MANAGER$ or $SURVEYS$ and replace them on request with the agent ids for those specific services.

[Aran30]

This sounds like a good idea, maybe it would be nice to also have this for all services and not only the bot-manager and the survey-service.

[lakhoune]

I don't really see a use case for other services. Furthermore you would need to introduce wildcards for each service which would be a bit too much

[lakhoune]

we could also use indexing to further improve the performance

[klamma]

You can also fetch results in rows, e.g., the first 100 rows. While you display them, you can fetch more rows in the background.