Docs: Add nibble documentation #3983
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Great work @stephanie0x00, I made some suggestions to make the work even better :) |
ammar92
reviewed
Dec 24, 2024
There was a problem hiding this comment.
Nice work! This definitely helps in understanding Nibbles 👍
A few general things:
- We should capitalize Bits and Nibbles, as they refer to our modules and not the data units
- Use cross-references for linking to our own documentation, this is a safer method and results in less broken links
- The Octopoes page is getting quite big, we should consider splitting it up in smaller chunks
- Not for now per se, but we should prefer the reST syntax over Markdown as it's much more flexible and offer more tools
Maybe we can have some coffee together soon and discuss some docs related stuff?
Co-authored-by: ammar92 <[email protected]>
Co-authored-by: ammar92 <[email protected]>
Fix reference
Capitalise Nibbles
Capitalise bits and nibbles
stephanie0x00
commented
Jan 7, 2025
|
I changed the target branch to |
ammar92
approved these changes
Jan 9, 2025
originalsouth
requested changes
Jan 9, 2025
| **What are nibbles?** | ||
| Nibbles are the flexible replacement for bits. We learned a lot on what does and doesn't work around bits and ran into some limitations. The nibbles are solving these limitations. We hope that the nibbles are 4 times as efficient as bits, but this isn't officially bench marked (and is more of a bits-nibbles joke ;)). | ||
|
|
||
| Nibbles process the data in OpenKAT, they correlate data and try to extract as much information as possible based on the specified rules. Nibbles ensure for a mapping of one or more objects to another bunch of objects. Where bits would trigger on objects, nibbles can trigger on sets of objects. This makes it possible to map a single object to N objects (1 to N objects), but also map multiple objects to multiple objects (N to N objects). |
There was a problem hiding this comment.
Suggested change
| Nibbles process the data in OpenKAT, they correlate data and try to extract as much information as possible based on the specified rules. Nibbles ensure for a mapping of one or more objects to another bunch of objects. Where bits would trigger on objects, nibbles can trigger on sets of objects. This makes it possible to map a single object to N objects (1 to N objects), but also map multiple objects to multiple objects (N to N objects). | |
| Nibbles process normalized latent data in OpenKAT, they correlate this data and try to extract as much information as possible, based on the specified rules. Nibbles ensure for a mapping of one or more objects to another bunch of objects. Where bits would trigger on objects, nibbles can trigger on sets of objects. This makes it possible to map a single object to N objects (1 to N objects), but also map multiple objects to multiple objects (N to M objects). |
| Nibbles process the data in OpenKAT, they correlate data and try to extract as much information as possible based on the specified rules. Nibbles ensure for a mapping of one or more objects to another bunch of objects. Where bits would trigger on objects, nibbles can trigger on sets of objects. This makes it possible to map a single object to N objects (1 to N objects), but also map multiple objects to multiple objects (N to N objects). | ||
|
|
||
| **Limitation example of bits** | ||
| One of the limitations from bits was that if one of the bits (business rules) would identify an open port in network A, and for network B this port would identify as closed, how would you classify the port? With bits it was not possible to correctly identify data mismatches and decide upon this. Nibbles allow you to detect these differences in data observations. If there are any inconsistencies, this could other nibbles to be triggered to resolve the inconsistency. |
There was a problem hiding this comment.
Suggested change
| One of the limitations from bits was that if one of the bits (business rules) would identify an open port in network A, and for network B this port would identify as closed, how would you classify the port? With bits it was not possible to correctly identify data mismatches and decide upon this. Nibbles allow you to detect these differences in data observations. If there are any inconsistencies, this could other nibbles to be triggered to resolve the inconsistency. | |
| One example of the limitations from bits was that if one of the bits (business rules) would identify an open port in network A, and for network B this port would identify as closed, how would you classify the port? With bits it was not possible to correctly identify data mismatches and decide upon this. Nibbles allow you to detect these differences in data observations. If there are any inconsistencies, this could other nibbles to be triggered to resolve the inconsistency. |
|
|
||
| Nibbles are accessible from the web interface, to allow for enabling, disabling, editing and adding of nibbles. | ||
|
|
||
| Using the XTDB EDN query language you can perform queries for nibbles. You can query a set of objects and process this in your nibble, including all possible combinations on the queried set (of objects). One of the downsides is that this could result in a large number of queries that are structurally the same, but are slightly different (e.g. flags that are different). It is therefore important to properly document your queries to still comprehend them in the future (and make it easier to catch bugs). |
There was a problem hiding this comment.
Suggested change
| Using the XTDB EDN query language you can perform queries for nibbles. You can query a set of objects and process this in your nibble, including all possible combinations on the queried set (of objects). One of the downsides is that this could result in a large number of queries that are structurally the same, but are slightly different (e.g. flags that are different). It is therefore important to properly document your queries to still comprehend them in the future (and make it easier to catch bugs). | |
| Using the XTDB EDN query language you can perform queries for nibbles. You can query a set of objects and process this in your nibble, including all possible combinations on the queried set (of objects). One of the downsides is that this could result in a large number of sub-queries that are structurally the same, but are slightly different (e.g. flags that are different). It is therefore important to properly document your queries to still comprehend them in the future (and make it easier to catch bugs). |
| Nibbles are a successor for bits, all the mistakes and issues we learned from bits should be solved with nibbles. | ||
|
|
||
| - Dynamic loading from the user interface. This allows a user to enable, run or add nibbles from the web interface, instead of having to edit the code. | ||
| - Mapping of objects: bits would trigger if a specific object (ooi) was loaded. This required a lot of recalculation of bits, which is not cost effective. With nibbles, as soon as data is added to the XTDB database, the nibbles will pick this up and handle accordingly. This should speed things up and allows for more extensive querying. |
There was a problem hiding this comment.
Suggested change
| - Mapping of objects: bits would trigger if a specific object (ooi) was loaded. This required a lot of recalculation of bits, which is not cost effective. With nibbles, as soon as data is added to the XTDB database, the nibbles will pick this up and handle accordingly. This should speed things up and allows for more extensive querying. |
originalsouth
requested changes
Jan 9, 2025
Co-authored-by: originalsouth <[email protected]>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
This PR adds documentation to docs.openkat.nl on what nibbles are and why they replace bits.
Issue link
Closes #3976
Demo
n/a
QA notes
Check if what is written is correct on how nibbles work and if you actually understand what nibbles are now. This is currently still a work in progress while nibbles are further developed.
Code Checklist
.envchanges files if required and changed the.env-distaccordingly.Checklist for code reviewers:
Copy-paste the checklist from the docs/source/templates folder into your comment.
Checklist for QA:
Copy-paste the checklist from the docs/source/templates folder into your comment.