Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync to Master #3824

Merged
merged 33 commits into from
Oct 13, 2024
Merged

Sync to Master #3824

merged 33 commits into from
Oct 13, 2024

Conversation

scudette
Copy link
Contributor

No description provided.

mgreen27 and others added 30 commits October 14, 2024 01:23
Add execution field for windows 10+ - displays integer
Add "NA" output for Win8/Win7 struct.
Updated extraction function to exclude any base64 blobs in a URL as
large scale testing I observed base64 characters extracted from URLs was
always a misfire and often above the minimum character threshold.
fix typo in RiskyExe definition
* Load fs accessors before exporting VQL information for `velociraptor
vql export` command

* Render event log messages as logs
* Remove compiled collector args for HuntList response so it can fit
within the gRPC size limit.
* Case insensitive comparison for expected hash to support artifacts
that specify hashes with caps.
* Remove error level logging from sqlite code - this causes artifact
collection to fail during normal operations

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This causes huge browser delays for certain large artifacts like
Windows.Hayabusa.Rules which contain large blobs in the artifact
definition.
Since Linux paths may already have quotes in them and a quote or
backslash are valid characters in a linux path.

This can cause issues when serializing Glob results to temp files.

Fixes: #3783
Update some logic and remove noisy Hostname mismatch
Allow the whole table, specific columns, or specific cells to be
compacted. This helps certain data which may be too large to fit in a
cell. Previously such data caused the cell to be expanded which made the
table more sparse and hard to see. This PR allows only certain cells to
be expanded as needed to keep the table compact taking less real estate.

Reworked the filter/sort buttons at the top of each column to appear
outside the header itself. Previously these buttons were part of the
header causing the header to be slightly wider than it needed to be-
making the table more sparse. This new CSS allows the buttons to extend
outside the column when they appear.

Changed the sense of drag and drop on column headers to insert the
dragged column before the dragged upon table - previously they would be
swapped but this is not intuitive.


![image](https://github.com/user-attachments/assets/812dab17-195f-4292-80a0-3ce34b9f2312)
The artifactset GUI was redesigned to use a multi-select which
serializes to a json array. However, artifactset expects to serialize to
a CSV type parameter.

This made it impossible to specify any artifacts in the GUI correctly.
* Reflect user preference timezone
* Shows times in RFC3339 format
* Enter time in RFC3339 format
* Dropdown calendar
* Quick settings - next week, prev week etc.
Mainly related to the new calendar control and dropdown menu
backgrounds.
This resulted in unsampled data overflowing the 2000 row maximum the GUI
is prepared to accept into the graphs. This means the graphs would
terminate once 2000 rows were reached. Therefore, for a lot of past data
the end of the graph was not very recent.
Add a notebook suggestion that creates a simple table that looks very
much like the default output from journalctl. I find myself studying the
identifier/unit and message exclusively most of the time. Having this
simplification as a VQL suggestion would be very handy for day-to-day
use.
Previously the timeline widget would display all times in UTC but other
times in the GUI are displayed in the user's selected timezone. This PR
changes this to make the timeline widget display times in the same
timezone selected by the user.

Also refactored CSV parameter editor to not use editable tables. These
were not intuitive as the user had to click on the table to realize they
were editable. Also it caused the layout to jump around. This PR fixes
it in favor of fixed simple text areas.

Additionally this PR patches the timeline widget to ensure ctrl-wheel
zooms in steps of 1 - the default upstream is to jump in steps of 10
which is way too much
This allows any row from any table to be annotated to a global timeline.
This is allowed even if there is no natural time column in the row (the
time is set to 1980 in that case).

This feature allows to selectively annotating noteworthy rows from any
artifact/client/flow/hunt into a central place for reporting and
analysis without first importing a time series.

The user simply right clicks on any row and selects "Annotate"
Also add timestamp in the artifact to record when the artifact was
built.
If the timeline message column was not a string, it would not be emitted
at all. This PR forces it to be a string.

Also handle floats in table output.

Update vfilter to fix LIMIT bug in GROUP BY clauses.
This preserves the column types more accurately to correspond with
internal data types.
adding IP analysis to VT enrichment artifact in order to query IP
addresses in addition to hashes/files.
Also added add_labels and del_labels to the hunt_update() VQL function
to allow hunt labels to be manipulated from VQL.

Modified Event Table UI to allow for resizable and draggable columns.
Extended the link_to() VQL function to support more objects, such as
uploads.

Also fixed timezone issues in graphs.
* Root org will create a backup for all other orgs. The data for each
org can be stored in the same zip file.
* It is now possible to restore only some providers selectively.
Add to timeline dialog broke due to the recent API changes. CSS Fix for
compressed JSON objects
Add column compression for event timeline viewer.
@scudette scudette merged commit 331c6c4 into v0.73 Oct 13, 2024
2 of 3 checks passed
@scudette scudette deleted the sync branch October 13, 2024 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants