Implement a Jupyter Widget for ITables (#319)

* npm create anywidget@latest * npm install dt_for_itables --save * Implement the widget * Add init function * Move the widget to itables.widget * Document the ITable widget * Include itables_anywidget in build * Add traitlets as a dependency of itables[widget] * Add selected rows * Update the row selection in the table using model * Recreate the table when the dt_args change * fixup pyproject.toml * The value of the streamlit component is the list of selected rows * Add anywidget to the environment * fix typo * WIP get_selected_rows_after_downsampling * fix tests * Sync data and table def with destroy_and_recreate * Passing the data through dt_args seems to be faster * Selected rows conversion in JS * Add back [tool.hatch.build.targets.sdist] and [tool.hatch.build.targets.wheel] * Selected rows in Streamlit * Fix selected rows * New functions set/get_selected_rows * Pass filtered_row_count explicitly * table_id rather than tableId * selected rows and offline mode in Shiny apps * Set the initial row selection in shiny inputs * Document init_itables and selected_rows * New df property and setter, make some traits private * Version 2.2.0 * Print shiny version if test fails * Require shiny>=1.0 in tests * Install shiny>=1 when Python>3.7 * Skip test if recent shiny is not available * Update the documentation
mwouts · Sep 22, 2024 · ef5cd58 · ef5cd58
1 parent 20546b2
commit ef5cd58
Show file tree

Hide file tree

Showing 38 changed files with 1,275 additions and 111 deletions.
diff --git a/.github/workflows/continuous-integration.yml b/.github/workflows/continuous-integration.yml
@@ -87,6 +87,10 @@ jobs:
         if: matrix.polars
         run: pip install -e .[polars]
 
+      - name: Install shiny
+        if: matrix.python-version != '3.7'
+        run: pip install "shiny>=1.0"
+
       - name: Uninstall jinja2
         if: matrix.uninstall_jinja2
         run: pip uninstall jinja2 -y

diff --git a/.gitignore b/.gitignore
@@ -30,3 +30,6 @@ dt_bundle.css
 
 # Streamlit package
 src/itables/itables_for_streamlit
+
+# Jupyter Widget
+src/itables/widget/static
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -32,7 +32,7 @@ repos:
     rev: v1.16.2
     hooks:
     - id: jupytext
-      exclude: dt_for_itables/
+      exclude: packages/
       types: ["markdown"]
       args: ["--pipe", "isort {} --treat-comment-as-code '# %%' --profile black", "--pipe", "black", "--check", "ruff check {} --ignore E402"]
       additional_dependencies:

diff --git a/README.md b/README.md
@@ -6,6 +6,7 @@
 [![Conda Version](https://img.shields.io/conda/vn/conda-forge/itables.svg)](https://anaconda.org/conda-forge/itables)
 [![pyversions](https://img.shields.io/pypi/pyversions/itables.svg)](https://pypi.python.org/pypi/itables)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![Jupyter Widget](https://img.shields.io/badge/Jupyter-Widget-F37626.svg?style=flat&logo=Jupyter)](https://mwouts.github.io/itables/ipywidgets.html)
 [![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_red.svg)](https://itables.streamlit.app)
 
 This packages changes how Pandas and Polars DataFrames are rendered in Jupyter Notebooks.
@@ -48,7 +49,7 @@ and then render any DataFrame as an interactive table that you can sort, search
 If you prefer to render only selected DataFrames as interactive tables, use `itables.show` to show just one Series or DataFrame as an interactive table:
 ![show](docs/show_df.png)
 
-Since `itables==1.0.0`, the [jQuery](https://jquery.com/) and [DataTables](https://datatables.net/) libraries and CSS
+Since ITables v1.0, the [jQuery](https://jquery.com/) and [DataTables](https://datatables.net/) libraries and CSS
 are injected in the notebook when you execute `init_notebook_mode` with its default argument `connected=False`.
 Thanks to this the interactive tables will work even without a connection to the internet.
 
@@ -63,6 +64,7 @@ You can also use ITables in [Quarto](https://mwouts.github.io/itables/quarto.htm
 
 ITables works well in VS Code, both in Jupyter Notebooks and in interactive Python sessions.
 
-Last but not least, ITables is also available in
-[Streamlit](https://mwouts.github.io/itables/streamlit.html) or
-[Shiny](https://mwouts.github.io/itables/shiny.html) applications.
+Last but not least, ITables is also available as
+- a [Jupyter Widget](https://mwouts.github.io/itables/ipywidgets.html)
+- a [Streamlit](https://mwouts.github.io/itables/streamlit.html) component,
+- and it also works in [Shiny](https://mwouts.github.io/itables/shiny.html) applications.
diff --git a/docs/changelog.md b/docs/changelog.md
@@ -1,6 +1,21 @@
 ITables ChangeLog
 =================
 
+2.2.0 (2024-09-22)
+------------------
+
+**Added**
+- ITables has a Jupyter Widget ([#267](https://github.com/mwouts/itables/issues/267)). Our widget was developed and packaged using [AnyWidget](https://anywidget.dev/) which I highly recommend!
+- The selected rows are now available in the apps. Use either the `selected_rows` attribute of the `ITable` widget, the returned value of the Streamlit `interactive_table` component, or the `{table_id}_selected_rows` input in Shiny ([#208](https://github.com/mwouts/itables/issues/208), [#250](https://github.com/mwouts/itables/issues/250))
+- ITables works offline in Shiny applications too - just add `ui.HTML(init_itables())` to your application
+
+**Changed**
+- The `tableId` argument of `to_html_datatable` has been renamed to `table_id`
+
+**Fixed**
+- The dependencies of the Streamlit component have been updated ([#320](https://github.com/mwouts/itables/issues/320))
+
+
 2.1.5 (2024-09-08)
 ------------------
 
@@ -10,7 +25,7 @@ ITables ChangeLog
 - We have improved the function that determines whether a dark theme is being used ([#294](https://github.com/mwouts/itables/issues/294))
 - We have adjusted the generation of the Polars sample dataframes to fix the CI ([Polars-18130](https://github.com/pola-rs/polars/issues/18130))
 - The test on the Shiny app fallbacks to `ui.nav_panel` when `ui.nav` is not available
-- The dependencies of the streamlit component have been updated ([#313](https://github.com/mwouts/itables/issues/313), [#315](https://github.com/mwouts/itables/issues/315))
+- The dependencies of the Streamlit component have been updated ([#313](https://github.com/mwouts/itables/issues/313), [#315](https://github.com/mwouts/itables/issues/315))
 
 
 2.1.4 (2024-07-03)
@@ -35,7 +50,7 @@ ITables ChangeLog
 an automatic horizontal scrolling in Jupyter, Jupyter Book and also Streamlit if the table is too wide ([#282](https://github.com/mwouts/itables/pull/282)).
 
 **Fixed**
-- The dependencies of the streamlit components have been updated to fix a vulnerability in `ws` ([Alert 1](https://github.com/mwouts/itables/security/dependabot/1))
+- The dependencies of the Streamlit components have been updated to fix a vulnerability in `ws` ([Alert 1](https://github.com/mwouts/itables/security/dependabot/1))
 
 
 2.1.1 (2024-06-08)

diff --git a/docs/extensions.md b/docs/extensions.md
@@ -269,7 +269,12 @@ only the selected rows are exported
 ```{code-cell}
 :tags: [full-width]
 
-show(df, select=True, buttons=["copyHtml5", "csvHtml5", "excelHtml5"])
+show(
+    df,
+    select=True,
+    selected_rows=[2, 4, 5],
+    buttons=["copyHtml5", "csvHtml5", "excelHtml5"],
+)
 ```
 
 ```{tip}
@@ -283,8 +288,11 @@ however cell selection is not taken into account when exporting the data.
 ```
 
 ```{tip}
-At the moment it is not possible to get the selected rows back in Python. Please subscribe to
-[#250](https://github.com/mwouts/itables/issues/250) to get updates on this topic.
+It is possible to get the updated `selected_rows` back in Python but for this you will have to use,
+instead of `show`, either
+- the `ITable` [Jupyter Widget](ipywidgets.md)
+- the `interactive_table` [Streamlit component](streamlit.md)
+- or `DT` in a [Shiny app](shiny.md).
 ```
 
 ## RowGroup

diff --git a/docs/ipywidgets.md b/docs/ipywidgets.md
@@ -1,4 +1,118 @@
-# IPyWidgets
+---
+jupytext:
+  formats: md:myst
+  notebook_metadata_filter: -jupytext.text_representation.jupytext_version
+  text_representation:
+    extension: .md
+    format_name: myst
+    format_version: 0.13
+kernelspec:
+  display_name: itables
+  language: python
+  name: itables
+---
 
-ITables does not come as a [Jupyter Widget](https://ipywidgets.readthedocs.io) at the moment.
-You are welcome to subscribe or contribute to [#267](https://github.com/mwouts/itables/issues/267).
+# Jupyter Widget
+
+ITables is available as a [Jupyter Widget](https://ipywidgets.readthedocs.io) since v2.2.
+
+## The `ITable` widget
+
+The `ITable` widget has a few dependencies (essentially [AnyWidget](https://anywidget.dev),
+a great widget development framework!) that you can install with
+```bash
+pip install itables[widget]
+```
+
+The `ITable` class accepts the same arguments as the `show` method, but
+the `df` argument is optional.
+
+```{code-cell}
+from itables.sample_dfs import get_dict_of_test_dfs
+from itables.widget import ITable
+
+df = get_dict_of_test_dfs()["int_float_str"]
+
+table = ITable(df, selected_rows=[0, 2, 5], select=True)
+table
+```
+
+## The `selected_rows` traits
+
+The `selected_rows` attribute of the `ITable` object provides a view on the
+rows that have been selected in the table (remember to pass `select=True`
+to activate the row selection). You can use it to either retrieve
+or change the current row selection:
+
+```{code-cell}
+table.selected_rows
+```
+
+```{code-cell}
+table.selected_rows = [3, 4]
+```
+
+## The `df` property
+
+Use it to retrieve the table data:
+
+```{code-cell}
+table.df.iloc[table.selected_rows]
+```
+
+or to update it
+
+```{code-cell}
+table.df = df.head(6)
+```
+
+```{tip}
+`ITable` will raise an `IndexError` if the `selected_rows` are not consistent with the
+updated data. If you need to update the two simultaneously, use `table.update(df, selected_rows=...)`, see below.
+```
+
+## The `caption`, `style` and `classes` traits
+
+You can update these traits from Python, e.g.
+
+```{code-cell}
+table.caption = "numbers and strings"
+```
+
+## The `update` method
+
+Last but not least, you can update the `ITable` arguments simultaneously using the `update` method:
+
+```{code-cell}
+table.update(df.head(20), selected_rows=[7, 8])
+```
+
+## Limitations
+
+Compared to `show`, the `ITable` widget has the same limitations as the [Streamlit component](streamlit.md#limitations),
+e.g. structured headers are not available, you can't pass JavaScript callback, etc.
+
+The good news is that if you only want to _display_ the table, you do not need
+the `ITables` widget. Below is an example in which we use `show` to display a different
+table depending on the value of a drop-down component:
+
+```python
+import ipywidgets as widgets
+from itables import show
+from itables.sample_dfs import get_dict_of_test_dfs
+
+def use_show_in_interactive_output(table_name: str):
+    show(
+        sample_dfs[table_name],
+        caption=table_name,
+    )
+
+sample_dfs = get_dict_of_test_dfs()
+table_selector = widgets.Dropdown(options=sample_dfs.keys(), value="int_float_str")
+
+out = widgets.interactive_output(
+    use_show_in_interactive_output, {"table_name": table_selector}
+)
+
+widgets.VBox([table_selector, out])
+```
diff --git a/docs/quick_start.md b/docs/quick_start.md
@@ -20,6 +20,7 @@ kernelspec:
 [![Conda Version](https://img.shields.io/conda/vn/conda-forge/itables.svg)](https://anaconda.org/conda-forge/itables)
 [![pyversions](https://img.shields.io/pypi/pyversions/itables.svg)](https://pypi.python.org/pypi/itables)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![Jupyter Widget](https://img.shields.io/badge/Jupyter-Widget-F37626.svg?style=flat&logo=Jupyter)](ipywidgets.md)
 [![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_red.svg)](https://itables.streamlit.app)
 <a class="github-button" href="https://github.com/mwouts/itables" data-icon="octicon-star" data-show-count="true"></a>
 <script src="https://buttons.github.io/buttons.js"></script>

diff --git a/docs/shiny.md b/docs/shiny.md
@@ -5,10 +5,18 @@ You can use ITables in Web applications generated with [Shiny](https://shiny.rst
 from shiny import ui
 
 from itables.sample_dfs import get_countries
-from itables.shiny import DT
+from itables.shiny import DT, init_itables
 
-df = get_countries(html=False)
-ui.HTML(DT(df))
+# Load the datatables library and css from the ITables package
+# (use connected=True if you prefer to load it from the internet)
+ui.HTML(init_itables(connected=False))
+
+# Render the table with DT
+ui.HTML(DT(get_countries(html=False)))
 ```
 
+If you enable row selection and set an id on your table, e.g. `DT(df, table_id="my_table", select=True)` then
+ITables will provide the list of selected rows at `input.my_table_selected_rows()` (replace `my_table` with your
+own table id).
+
 See also our [tested examples](https://github.com/mwouts/itables/tree/main/tests/sample_python_apps).
diff --git a/docs/streamlit.md b/docs/streamlit.md
@@ -13,6 +13,13 @@ We have a sample application available at https://itables.streamlit.app (source
 style="height: 600px; width: 100%;"></iframe>
 ```
 
+## Selected rows
+
+This feature was added in ITables v2.2.0.
+
+Use the `selected_rows: list[int]` argument from `interactive_table` to
+select rows when the table is first displayed. Add `select=True` to let the user modify the selection. Then, the `interactive_table` component returns a dict, with a key `"selected_rows"` that points to the updated selection.
+
 ## Limitations
 
 In most cases, you will be able to use `interactive_table` in a
@@ -42,9 +49,3 @@ A sample application is available at https://to-html-datatable.streamlit.app (so
 <iframe src="https://to-html-datatable.streamlit.app?embed=true"
 style="height: 600px; width: 100%;"></iframe>
 ```
-
-## Future developments
-
-ITables' Streamlit component might see the following developments in the future
-- Return the selected cells
-- Make the table editable (will require a DataTable [editor license](https://editor.datatables.net/purchase/))
diff --git a/environment.yml b/environment.yml
@@ -25,5 +25,6 @@ dependencies:
   - ghp-import
   - shiny
   - streamlit
+  - anywidget
   - pip:
     - world_bank_data
diff --git a/packages/dt_for_itables/CHANGELOG.md b/packages/dt_for_itables/CHANGELOG.md
@@ -1,3 +1,7 @@
+# 2.0.13 (2024-09-22)
+
+- We have added two functions `set_selected_rows` and `get_selected_rows` to set and retrieve selected rows
+
 # 2.0.12 (2024-09-08)
 
 - We have added the datetime extension for DataTables ([#288](https://github.com/mwouts/itables/issues/288))

diff --git a/packages/dt_for_itables/package-lock.json b/packages/dt_for_itables/package-lock.json
diff --git a/packages/dt_for_itables/package.json b/packages/dt_for_itables/package.json
@@ -1,6 +1,6 @@
 {
   "name": "dt_for_itables",
-  "version": "2.0.12",
+  "version": "2.0.13",
   "description": "DataTables bundle for itables",
   "main": "src/index.js",
   "typings": "src/index.d.js",

diff --git a/packages/dt_for_itables/src/index.js b/packages/dt_for_itables/src/index.js
@@ -34,6 +34,26 @@ import 'datatables.net-select-dt/css/select.dataTables.min.css';
 
 import './index.css';
 
+DataTable.get_selected_rows = function (dt, filtered_row_count) {
+    // Here the selected rows are for the datatable.
+    // We convert them back to the full table
+    let data_row_count = dt.rows().count();
+    let bottom_half = data_row_count / 2;
+    return Array.from(dt.rows({ selected: true }).indexes().map(
+        i => (i < bottom_half ? i : i + filtered_row_count)));
+}
+
+DataTable.set_selected_rows = function (dt, filtered_row_count, selected_rows) {
+    let data_row_count = dt.rows().count();
+    let bottom_half = data_row_count / 2;
+    let top_half = bottom_half + filtered_row_count;
+    let full_row_count = data_row_count + filtered_row_count;
+    selected_rows = Array.from(selected_rows.filter(i => i >= 0 && i < full_row_count && (i < bottom_half || i >= top_half)).map(
+        i => (i < bottom_half) ? i : i - filtered_row_count));
+    dt.rows().deselect();
+    dt.rows(selected_rows).select();
+}
+
 export { DataTable, DateTime, jQuery };
 
 export default DataTable;
diff --git a/packages/itables_anywidget/README.md b/packages/itables_anywidget/README.md
@@ -0,0 +1,29 @@
+# itables_anywidget
+
+## Installation
+
+```sh
+pip install itables_anywidget
+```
+
+## Development installation
+
+Create a virtual environment and and install itables_anywidget in *editable* mode with the
+optional development dependencies:
+
+```sh
+python -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+You then need to install the JavaScript dependencies and run the development server.
+
+```sh
+npm install
+npm run dev
+```
+
+Open `example.ipynb` in JupyterLab, VS Code, or your favorite editor
+to start developing. Changes made in `js/` will be reflected
+in the notebook.
diff --git a/packages/itables_anywidget/js/widget.css b/packages/itables_anywidget/js/widget.css