-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
graph view and central data store #1108
Commits on Nov 21, 2022
-
graph view experiment 1: get data
* We need to use the "tree data" whether we want a tree or not because this is the only data window which associates nodes with their parent workflows directly. * Allow edge deltas through.
Configuration menu - View commit details
-
Copy full SHA for ac271aa - Browse repository at this point
Copy the full SHA ac271aaView commit details -
graph view experiment 2: construct a global tree
* Construct a data tree in the workflow store to be used as the cenrtral data store. * This stores everything Cylc: * UID objects `~user/a/b/workflow//cycle/task/job`. * $namespaces * $edges * This fulfils common requirements e.g. sorting.
Configuration menu - View commit details
-
Copy full SHA for 176cf14 - Browse repository at this point
Copy the full SHA 176cf14View commit details -
graph: implement the graph view
* Add a GraphNode component. * Render nodes. * Load GraphViz (WASM). * Run GraphViz layout.. * Translate nodes to match layout coordinates. * Render edges. Make some adjustments to the central data store: * flat index (cylcTree.$index) * tree (cylcTree.children)
Configuration menu - View commit details
-
Copy full SHA for d3b2ca2 - Browse repository at this point
Copy the full SHA d3b2ca2View commit details -
store: refactor the store to eliminate cicrular references
* VueX doesn't like circular references (e.g. node.parent).
Configuration menu - View commit details
-
Copy full SHA for 18b1c18 - Browse repository at this point
Copy the full SHA 18b1c18View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9edf7fe - Browse repository at this point
Copy the full SHA 9edf7feView commit details -
* Auto refresh the graph at a hardcoded rate but only when the data has changed.
Configuration menu - View commit details
-
Copy full SHA for fbcd0e3 - Browse repository at this point
Copy the full SHA fbcd0e3View commit details -
graph node: make tasks clickable
* Jobs are not clickable yet, however, we don't have any job mutations yet and the graph view might want to provide a custom job overview here so omitting this for now.
Configuration menu - View commit details
-
Copy full SHA for 5d5e0d7 - Browse repository at this point
Copy the full SHA 5d5e0d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8377270 - Browse repository at this point
Copy the full SHA 8377270View commit details -
* Allow the graph to be transposed (i.e. switch from top-bottom to left-right edge orientation).
Configuration menu - View commit details
-
Copy full SHA for 5c4fafd - Browse repository at this point
Copy the full SHA 5c4fafdView commit details -
graph: add centre graph control
* Add a button to centre the graph. * Increase the permitted zoom range.
Configuration menu - View commit details
-
Copy full SHA for d8fdcfa - Browse repository at this point
Copy the full SHA d8fdcfaView commit details -
* The old task icon modifiers didn't render correctly when loaded into a larger document due to some relative transformation problem. * Simplified back to a regular template component. * Tweaked the icons a little. * Still needs work, the modifiers are too small!
Configuration menu - View commit details
-
Copy full SHA for c940d82 - Browse repository at this point
Copy the full SHA c940d82View commit details -
Configuration menu - View commit details
-
Copy full SHA for 817cf92 - Browse repository at this point
Copy the full SHA 817cf92View commit details -
workflow service: run global callback by default
* Enable the global callback by default irrespective of which views are open. * Disable local callbacks for the central data store.
Configuration menu - View commit details
-
Copy full SHA for b664721 - Browse repository at this point
Copy the full SHA b664721View commit details -
task: switch to new task component
* Switch views to use the re-implemented task component.
Configuration menu - View commit details
-
Copy full SHA for 92f648c - Browse repository at this point
Copy the full SHA 92f648cView commit details -
store: add family support to the tree
* Add a `familyTree` node under each cycle point when families are recieved by the store. * Populate this tree using `childTasks` and `ancestors` from the schema. * Use the flat ID for families as it is not possible to reconstruct the hierarchical ID for updates as the ancestors list will not have change so will not be present in the delta. * Test registration of children. * Fix cycle/family insertion order issue * Remove tasks from family tree when removed from childTasks
Configuration menu - View commit details
-
Copy full SHA for 6ee0155 - Browse repository at this point
Copy the full SHA 6ee0155View commit details -
workflow service: fix query merging
* Make queries mergable * In order to merge queries properly their definition names must match, otherwise the fragments can become isolated from the definitions which use them. * Fixed by standardising the naming of defintions so that they can merge cleanly. * Fix re-issue of queries after merging * Closes cylc#1110 * The query object has some derived properties which are not updated when we modify it in place. * The ApolloClient appears to by using these properties which causes it to see the query as it was *before* merging rather than after. * To fix we re-create the query object after merge in order to ensure its derived properties are set correctly.
Configuration menu - View commit details
-
Copy full SHA for 282dc1d - Browse repository at this point
Copy the full SHA 282dc1dView commit details -
tree: fix filtering & job-details nodes
* Remove root nodes from tree. * Fix job-details nodes. * Fix filtering.
Configuration menu - View commit details
-
Copy full SHA for 7d15851 - Browse repository at this point
Copy the full SHA 7d15851View commit details -
Configuration menu - View commit details
-
Copy full SHA for 769470d - Browse repository at this point
Copy the full SHA 769470dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e8bb6ab - Browse repository at this point
Copy the full SHA e8bb6abView commit details -
Configuration menu - View commit details
-
Copy full SHA for 712c862 - Browse repository at this point
Copy the full SHA 712c862View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4eb2e4f - Browse repository at this point
Copy the full SHA 4eb2e4fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 36f399b - Browse repository at this point
Copy the full SHA 36f399bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 372eea5 - Browse repository at this point
Copy the full SHA 372eea5View commit details -
gscan: convert to run off of the central data store
* Remove local callbacks. * Convert subscriptions to match the pattern of the others for merging purposes. * Search and filter are broken for the moment.
Configuration menu - View commit details
-
Copy full SHA for 801ac43 - Browse repository at this point
Copy the full SHA 801ac43View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3711326 - Browse repository at this point
Copy the full SHA 3711326View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8b0a955 - Browse repository at this point
Copy the full SHA 8b0a955View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6f85bda - Browse repository at this point
Copy the full SHA 6f85bdaView commit details -
gscan: fix sorting & filtering
* Adapt sort and filter mechanism to work with the central data store.
Configuration menu - View commit details
-
Copy full SHA for f3b1d29 - Browse repository at this point
Copy the full SHA f3b1d29View commit details -
Configuration menu - View commit details
-
Copy full SHA for efa99c7 - Browse repository at this point
Copy the full SHA efa99c7View commit details -
Configuration menu - View commit details
-
Copy full SHA for b7be28f - Browse repository at this point
Copy the full SHA b7be28fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 07a0acc - Browse repository at this point
Copy the full SHA 07a0accView commit details -
store: housekeep the store for workflow subscriptions
* When we used local callbacks we deleted the data obtained by the callback when the subscription was closed. * Now we use a central data store we cannot rely on this mechanism so need to manually wipe the data the subscription requested. * This is a crude initial implementation which assumes that no other subscriptions active at the time were requesting cycles/families/namespaces etc from the workflow(s) that were being requested. At the time of writing that is an accurate assumption.
Configuration menu - View commit details
-
Copy full SHA for 6645669 - Browse repository at this point
Copy the full SHA 6645669View commit details -
task: delete unused styles and re-calibrate progress
* Delete styles/cylc/_task (these styles now reside in the Vue component). * Fix the size of the task "hub" (the middle bit of the running icon). * Re-calibrate the progress indicator (which needs to be fiddled whenever we change its size).
Configuration menu - View commit details
-
Copy full SHA for 5a054de - Browse repository at this point
Copy the full SHA 5a054deView commit details -
* Nanageddon i.e. a bug where the transform matrix of the pan-zoom layer (the SVG group that SVGPanZoom inserts for us) gets NaN (not a number) values causing an error (and broken pan/zoom functionality). * Resolve this by registering the pan/zoom on the `$.nextTick` after mount rather than at mount time as the SVG was only initialised ~90% of the time.
Configuration menu - View commit details
-
Copy full SHA for 7c23fe2 - Browse repository at this point
Copy the full SHA 7c23fe2View commit details -
Configuration menu - View commit details
-
Copy full SHA for ad9a6dc - Browse repository at this point
Copy the full SHA ad9a6dcView commit details -
Configuration menu - View commit details
-
Copy full SHA for b730433 - Browse repository at this point
Copy the full SHA b730433View commit details -
Configuration menu - View commit details
-
Copy full SHA for e5ff894 - Browse repository at this point
Copy the full SHA e5ff894View commit details -
task: fix progress icon in webkit/safari
* The progress icon was rotated but the point around which it was rotated was not specified. Safari/Webkit picked the centre of the object, Opera/Chromium picked the top-left corner. Hardcoded this into the rotation to avoid issues. * Closes cylc#1078
Configuration menu - View commit details
-
Copy full SHA for c61be9c - Browse repository at this point
Copy the full SHA c61be9cView commit details -
Configuration menu - View commit details
-
Copy full SHA for cf8a682 - Browse repository at this point
Copy the full SHA cf8a682View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2d46207 - Browse repository at this point
Copy the full SHA 2d46207View commit details -
Configuration menu - View commit details
-
Copy full SHA for 51e41d4 - Browse repository at this point
Copy the full SHA 51e41d4View commit details -
graphql: only stripNull for updated Deltas
* Setting stripNull for Added deltas could cause null-like values to be absent in the node data resulting in undefined errors.
Configuration menu - View commit details
-
Copy full SHA for 3c88f3d - Browse repository at this point
Copy the full SHA 3c88f3dView commit details -
tree: fix missing tasks on root family
* Caused by subscription omitting updates for the root family
Configuration menu - View commit details
-
Copy full SHA for fb3bf1d - Browse repository at this point
Copy the full SHA fb3bf1dView commit details -
graphql: remove unnecessary sorting
* The central data store performs sorting so we don't need to bother the UIS to do this for us.
Configuration menu - View commit details
-
Copy full SHA for 69b8bb5 - Browse repository at this point
Copy the full SHA 69b8bb5View commit details -
table: convert to run on central data store
* Also add workflow icon and status columns to close cylc#1116
Configuration menu - View commit details
-
Copy full SHA for b5ba5da - Browse repository at this point
Copy the full SHA b5ba5daView commit details -
Configuration menu - View commit details
-
Copy full SHA for e1a9e06 - Browse repository at this point
Copy the full SHA e1a9e06View commit details -
graph: add toolbar and manual refresh
* Convert clunky buttons into slick toolbar. * Allow the auto-refresh timer to be disabled and add a manual refresh option. * Set edge colour to match node colour.
Configuration menu - View commit details
-
Copy full SHA for e114e95 - Browse repository at this point
Copy the full SHA e114e95View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4eafc93 - Browse repository at this point
Copy the full SHA 4eafc93View commit details -
store: remove old stores, callbacks and fix tests
* Remove the old workflow store code which powered the local callbacks. * Remove the local callbacks which have been replaced by the central data store. NOTE: Local callbacks are still possible, however, we don't need to provide a pattern for this as we currently have no uses. * Move a function for easier testing. * Fix tests. * tests/unit/task: remove in favour of component tests
Configuration menu - View commit details
-
Copy full SHA for 7391886 - Browse repository at this point
Copy the full SHA 7391886View commit details -
Configuration menu - View commit details
-
Copy full SHA for fddd54c - Browse repository at this point
Copy the full SHA fddd54cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6724223 - Browse repository at this point
Copy the full SHA 6724223View commit details -
tree: fix custom outputs and support messages
* Custom outputs were broken with the removal of the local callbacks. * Before that custom outputs weren't being correctly associated with the jobs which created them. * Tied outputs into messages in order to filter out the job which created the outputs. Closes cylc#1129 * Added messages in with outputs (after the filtering the default messages out). Closes cylc#595
Configuration menu - View commit details
-
Copy full SHA for 346beb6 - Browse repository at this point
Copy the full SHA 346beb6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 391ad39 - Browse repository at this point
Copy the full SHA 391ad39View commit details -
Configuration menu - View commit details
-
Copy full SHA for 88b05ef - Browse repository at this point
Copy the full SHA 88b05efView commit details -
Configuration menu - View commit details
-
Copy full SHA for 791efb1 - Browse repository at this point
Copy the full SHA 791efb1View commit details -
Configuration menu - View commit details
-
Copy full SHA for a7a655c - Browse repository at this point
Copy the full SHA a7a655cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 932fd77 - Browse repository at this point
Copy the full SHA 932fd77View commit details -
* Fix the description which appears at the top of the mutation menu. * This currently shows the node type and status.
Configuration menu - View commit details
-
Copy full SHA for 00a83d6 - Browse repository at this point
Copy the full SHA 00a83d6View commit details -
graphql: remove unused fields from queries
* Remove a coupld of fields which were never used. * Remove a few fields which are no longer required since the data store changes.
Configuration menu - View commit details
-
Copy full SHA for 16ec361 - Browse repository at this point
Copy the full SHA 16ec361View commit details -
tree: sort families before tasks
* Match the previous store implementation which sorted families before tasks in the tree. * This is similar to directory/file sort rules and makes the tree view easier to read.
Configuration menu - View commit details
-
Copy full SHA for 94e2a55 - Browse repository at this point
Copy the full SHA 94e2a55View commit details -
tree: fix erroneous filter issue
* Fix issue where nodes could sometimes be filtered "in" when they shouldn't have been as a result of the filter logic following the task tree rather than the family tree.
Configuration menu - View commit details
-
Copy full SHA for c784f41 - Browse repository at this point
Copy the full SHA c784f41View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8a73323 - Browse repository at this point
Copy the full SHA 8a73323View commit details -
Configuration menu - View commit details
-
Copy full SHA for ebe38c8 - Browse repository at this point
Copy the full SHA ebe38c8View commit details -
mutation: remove task icon from mutation form
* The task icon was a temporary development / debugging tool no longer required
Configuration menu - View commit details
-
Copy full SHA for b574a11 - Browse repository at this point
Copy the full SHA b574a11View commit details -
task: make the queued modifier more prominent & add expired state
* Increase the default modifier size. * Remove the outline for the queued modifier and increase the size of the horizontal bars. * Fix the modifier translation to better work for a wider range of modifier sizes. * Test the task icon with different modifier sizes. * Add icon for the expired task state.
Configuration menu - View commit details
-
Copy full SHA for 0e493af - Browse repository at this point
Copy the full SHA 0e493afView commit details -
* Remove unused getter. * Move getTree getter (test utility) into the unit tests. * Comments ++;
Configuration menu - View commit details
-
Copy full SHA for 1cf7973 - Browse repository at this point
Copy the full SHA 1cf7973View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2cc7e53 - Browse repository at this point
Copy the full SHA 2cc7e53View commit details -
store: centralise tree walking functions
* Provide an efficnent tree walking function as a vuex getter. * Use this to provide node lists for each of the Cylc "views".
Configuration menu - View commit details
-
Copy full SHA for 3ad9edc - Browse repository at this point
Copy the full SHA 3ad9edcView commit details -
graph: Apply suggestions from code review
Co-authored-by: Ronnie Dutta <[email protected]> * package: move wasm-loader from dependencies into devDependencies. * graph: change edge transformation for transposed graphs * We use static "fudge factors" to align the node and edge layers. * The graph now uses separate factors for transposed and non-transposed graphs for cleaner layout. * store: remove unnecessary familyPath variable.
Configuration menu - View commit details
-
Copy full SHA for 08cbec6 - Browse repository at this point
Copy the full SHA 08cbec6View commit details -
Configuration menu - View commit details
-
Copy full SHA for b43050e - Browse repository at this point
Copy the full SHA b43050eView commit details -
* Reduce task size. * Increase cycle size. * Increase size of first job.
Configuration menu - View commit details
-
Copy full SHA for c2a134f - Browse repository at this point
Copy the full SHA c2a134fView commit details -
store: fix subscription merging issue
* In some situations subscriptions were not being merged causing the new view to be missing data.
Configuration menu - View commit details
-
Copy full SHA for 3927ea0 - Browse repository at this point
Copy the full SHA 3927ea0View commit details -
* Fix issue where the $edges & $namespaces indicies were not being cleared when switching workflows. * This caused duplicate edges in the graph view.
Configuration menu - View commit details
-
Copy full SHA for 3e029be - Browse repository at this point
Copy the full SHA 3e029beView commit details -
* Fixes an issue where we tried to mount the svgPanZoom component before the dimensions of the SVG document could be measured. * This resulted in the console error: `Error: <g> attribute transform: Expected number, "matrix(NaN,0,0,NaN,NaN,…"` * The solution is to check the dimensions of the first item in the document, once it exists and has non-zero dimensions the SVG is ready.
Configuration menu - View commit details
-
Copy full SHA for 196a45e - Browse repository at this point
Copy the full SHA 196a45eView commit details -
* A vertial scrollbar could appear in some browsers as a result of the view toolbar pushing the SVG outside of the tab. * This makes the view toolbar a floating component to allow more space for the graph and prevent this issue.
Configuration menu - View commit details
-
Copy full SHA for 017873d - Browse repository at this point
Copy the full SHA 017873dView commit details -
* Updates are "merged" into our reactivate data blobs. * This merging used to use a nested algorithm which attempted to merge updates in arrays. * This isn't what we want as our updates are atomic, e.g. if an update contains `{childTasks: [{'id': 'a'}, {'id': 'b'}]}`, that means please update childTasks to match this data NOT please change the IDs of the first two childTasks on the node to match these values.
Configuration menu - View commit details
-
Copy full SHA for f95e3d5 - Browse repository at this point
Copy the full SHA f95e3d5View commit details -
graph: improve stability with large graphs
* Prevent parallel layout. This was unlikely but possible if layout time was greater than refresh time (~2s). * Wait for nodes to be rendered before attempting to layout. Sometimes one or more nodes hadn't yet rendered when layout was attempted causing an error reading the node dimensions. There's no "wait for everything to render" function so we must place a check in a loop. * Wait for all edges to be rendered before centering the graph. Auto-centering the graph on first load sometimes failed if the graph took a while to render. Put another condition in a loop to delay centering until the SVG is ready. * Catch layout errors. If anything does go wrong during layout, the graph will now re-try next time rather than crashing.
Configuration menu - View commit details
-
Copy full SHA for e26d672 - Browse repository at this point
Copy the full SHA e26d672View commit details -
* The `graphvizlib.wasm` file provided by `@hpcc-js/wasm` needs to be included in `dist/js`. * It is then loaded async by a JS script imported by the graph view. * At the time of writing we are using Webpack 4 which doesn't seem to play nicely with this approach so the best solution I can come up with is to symlink the wasm file into the `public/js` directory. * This symlink will be broken until you `yarn install`. * This hack might not be needed when we upgrade to Webpack 5, however, this might just be due to the way HPCC packages graphviz.
Configuration menu - View commit details
-
Copy full SHA for ccc69dc - Browse repository at this point
Copy the full SHA ccc69dcView commit details -
graph: add offline data and first e2e test
* Add edges into the offline data to allow the graph view to be tested offline. * Add a basic cypress test which checks the layout gets completed and the nodes and edges get rendered.
Configuration menu - View commit details
-
Copy full SHA for ac3bf77 - Browse repository at this point
Copy the full SHA ac3bf77View commit details -
graph: allow display of edgeless graphs
* Allow the graph view to display graphs which don't contain any edges.
Configuration menu - View commit details
-
Copy full SHA for ac2e835 - Browse repository at this point
Copy the full SHA ac2e835View commit details -
Configuration menu - View commit details
-
Copy full SHA for 62ab67a - Browse repository at this point
Copy the full SHA 62ab67aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 51d67cf - Browse repository at this point
Copy the full SHA 51d67cfView commit details
Commits on Nov 23, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 599e2cd - Browse repository at this point
Copy the full SHA 599e2cdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1a0f1b8 - Browse repository at this point
Copy the full SHA 1a0f1b8View commit details -
Update src/components/cylc/GraphNode.vue
Co-authored-by: Ronnie Dutta <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for fbe64e6 - Browse repository at this point
Copy the full SHA fbe64e6View commit details
Commits on Nov 24, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 3d28f92 - Browse repository at this point
Copy the full SHA 3d28f92View commit details