Data Table specs and components

[HendrikThePendric]

In this issue I'd like to discuss the technical implementation of the the design specs for the DataTable. Also see:

• JIRA issue TECH-215
• Slack thread
• Design specs - GitHub
• Design specs - Netlify

General approach

This topic relates to the data-table specifically, but I think is also a relevant topic in a more general sense. We have a design system which describes organisms in various contexts, and we have a ui-library which should enable developers to build apps that conform to this design system, including these complex organisms. But what is the end-goal/ preferred approach for our ui-lib reg. these organisms?

1. If we want the ui-lib to include implementations of each complex organism we will end up with more complex components with a large API surface (lots of props) to facilitate a lot of different use-cases. This will result in a higher maintenance and development burden for the lib, but it could potentially prevent apps from "having to re-invent the wheel" across projects. There is a risk of getting to a point where we encounter use-cases that have mutually exclusive implementations and things could start getting messy.
2. On the other hand, by producing only smaller, composable components we remain flexible, have a smaller API surface, and keep the maintenance burden down. The obvious downside is that we will start seeing various apps implementing very similar code to glue these components together.

My preference would be the second approach for various reasons. I think when building the select and transfer, we sort of followed the first approach, and I don't think any of us are happy about where we ended up: despite of a lot of effort, we still don't have support for all use-cases, and we've ended up at a point where a lot requested features can only be implemented in the component, by adding additional branching and more props. I don't think this is going to stand the test of time.

In order to end up with a final set of components (incl. their APIs) that support the design specs, we need to settle on a consensus about the above.

I will proceed to list the components that are available now and relevant to the DataTable. After that I will address all aspects of the the design specs briefly.

What is available now

• Various building blocks for a regular table: Table, TableBody, TableCell, TableCellHead, TableFoot, TableHead, TableRow, TableRowHead
• Similar building blocks for a stacked table. I don't see any mention of a stacked table in the design specs, so perhaps we should keept the stacked table completely out of this discussion for now.
• A Pagination component
• Various form-inputs that could be used to build table filters, but have not been designed specifically for this purpose.

Requirements from the design-spec not currently catered for

• Toolbar: fits above the table and can hold a title, actions and filter elements. We don't have this yet.

• Search/Filtering: UI components for filtering the data. The design specs explicitly differentiate between searching and filtering. It is unclear to me why this is, to me they are all just ways to filter the results. @cooper-joe could you clarify this? The search-ui in the specs could already be supported by using a text input, but there are two new versions that we definitely don't cater for yet:

  • Standalone: This is a button with a popover to filter. In the popover we would find an input, an update and cancel button, and a reset link. To me it's not entirely clear why would need such complex interactions, at first glance it would make more sense combine various filter-inputs with only 1 set of buttons. Perhaps I am unaware of some specific use-cases, @cooper-joe could you clarify?
  • Inline: These filter elements are incorporated in the table-cell-head and activated by clicking an icon. Regardless of the approach we end up going with, we probably need to make some changes to the TableCellHead component for these.

• Table Actions: These are simply buttons, so if we take the composable-building-blocks approach, we don't need to do any work to support this.

• Row actions: Similar to Table Actions

• Sorting/Ordering: At the very least we will have to do work in the TableCellHead to make sure it can show a sort icon in 3 states (off | asc | desc) and an onClick handler.

• Fixed header and columns: I'd be in favour of just leveraging CSS sticky positioning like this example and using as little JS as possible. For freezing multiple columns, there is some interaction between element styles: the columns need to be aware of each-other's widths so they can have the correct offset. I'm not 100% where and how we'd add support for this if we were to use the composable-building-blocks approach due to this interaction between elements... If we were to only support freezing the first column, this would not be a problem at all.

• Inline editing: If we take the composable-building-blocks approach, there's not that much to do: we would need to make sure the TableCell can hold a ref (as a reference for the popover), and perhaps add an editable prop which would show the pencil icon.

• Expandable rows: I think this approach, using grid-layout and implementing the expandable content as a td is very clean. As with fixed columns, the styling we need requires some interaction between elements, in this case the tr needs to have display: grid; and grid-template-columns: repeat(N, 1fr);, where N is the number of (regular) tds in the tr. It would make sense to implement this feature as an additional prop (i.e. expandableContent of type string|node) on a TableRow.

• Selectable rows: If we follow the composable-building-blocks approach we don't need to do anything. We can simply render some checkboxes in TableCells and TableHeadCells. I think this is a good example of the fork in the road we are facing: implementing a feature like this at table-level is not really compatible with just letting components render children, so if we decided that we need to have a DataTable component with a selectedRows prop we are sort of forced to take control of what this component is rendering and things start to become more rigid...

• Reordering rows: as with selectable rows, if we wanted to solve this in the table component we would need to take charge of the how these rows are rendered in the first place and things are starting to become rigid. Alternatively we could just make sure the TableRow accepts a ref, or whatever else is a prerequisite for enabling a drag-and-drop UI and let the apps build the actual DnD implementation. BTW @cooper-joe where do we need row reordering? I can't think of a relevant use-case...

• Editable columns: I would say that regardless of which approach we take, this is a completely separate component, i.e. out of scope.

• Bordered / large cells and built-in-formatting: should probably be added to TableCell and TableCellHead

• Status: as it says in the specs, this is just a cell with a tag, so out-of-scope.

---

Please review the above, let me know what your preference would be reg. the general approach and if you agree with my observations reg. the design specs.

Once we've settled on a general approach, I'll prepare a proposal with list of new components+props / changes to current components+props we need to facilitate the design specs.

My thanks in advance 😋

[cooper-joe]

Here's some additional information about the points @HendrikThePendric tagged me in:

The design specs explicitly differentiate between searching and filtering. It is unclear to me why this is, to me they are all just ways to filter the results. @cooper-joe could you clarify this?

The difference is the user intention. Search is intended for users that have a single data item in mind that they want to seek out. This can be done before the data-table is even populated, for example. Filtering is for narrowing down returned/displayed results. Though technically the same, filtering is exploratory, search is more intentional. Most data-tables would use one or the other. (I can update the usage guidelines to better reflect this). There are also configuration options that can affect this: in some Tracker programs search is disabled (for security/privacy), but returned results based on a predefined list could be filtered by allowed attributes.

This is a button with a popover to filter. In the popover we would find an input, an update and cancel button, and a reset link. To me it's not entirely clear why would need such complex interactions, at first glance it would make more sense combine various filter-inputs with only 1 set of buttons. Perhaps I am unaware of some specific use-cases, @cooper-joe could you clarify?

Standalone filter controls (inside the popovers) was initially designed for a specific (tracker) use-case and has gotten bundled into the ui-compoent. Note that these standalone filters were designed and deployed before the inline-filtering was designed. Inline-filtering is probably the preferred method, so I'd rather that was easily enabled over than the standalone filters. I think I should make clear in the usage docs the differences here, thanks for the heads up.

Displaying all the controls with a single set of buttons (for example, in a left sidebar), is also a valid design. It might be worth including a suggested 'usage pattern' for this too. 🤔 (This wouldn't be built-in, see my thoughts below on that).

BTW @cooper-joe where do we need row reordering? I can't think of a relevant use-case...

The 'Editable columns' example on the design-system page is actually an example 😉. Also, in maintenance app there are situations where elements need to be put in a specific order to be reflected in a form, for example.

---

Generally speaking, approach number 2 seems like the sensible choice. I think this is an interesting topic that ties in the recent discussion about documentation, component usage guidelines and patterns. The data-table design documentation falls into the category of 'Pattern'. It was written to encourage developers to follow certain patterns when filtering, selecting multiple rows, sorting and so on. Establishing the difference between design specifications and patterns for use is helpful because I think it is fair to say that: "specifications should be set by the component, patterns for use should be enabled by the way the components are built". By that definition patterns do not need to be pre-built, but they should be as easy as possible to achieve with the ready-made components. This would avoid the rigidity mentioned above.

[HendrikThePendric]

Thanks for the info @cooper-joe 👍 I have some more questions:

The difference is the user intention. Search is intended for users that have a single data item in mind that they want to seek out. This can be done before the data-table is even populated, for example. Filtering is for narrowing down returned/displayed results. Though technically the same, filtering is exploratory, search is more intentional. Most data-tables would use one or the other. (I can update the usage guidelines to better reflect this). There are also configuration options that can affect this: in some Tracker programs search is disabled (for security/privacy), but returned results based on a predefined list could be filtered by allowed attributes.

Thanks for explaining this, the difference in usage is clear. Reg. UI components, would you agree the following is true?

• For searching we don't need anything new, we can just use the TextField component (or other available form input components).
• For filtering we definitely need to implement inline filters, and perhaps the standalone filters too (or perhaps cancel these, see next point)

Standalone filter controls (inside the popovers) was initially designed for a specific (tracker) use-case and has gotten bundled into the ui-compoent. Note that these standalone filters were designed and deployed before the inline-filtering was designed. Inline-filtering is probably the preferred method, so I'd rather that was easily enabled over than the standalone filters.

If the standalone filter and the inline filter provide the same functionality and the inline filtered is the preferred method, then would it perhaps make sense to cancel the standalone filter?

Displaying all the controls with a single set of buttons (for example, in a left sidebar), is also a valid design. It might be worth including a suggested 'usage pattern' for this too. 🤔 (This wouldn't be built-in, see my thoughts below on that).

Yeah, I think that would be very important to add information about. In my opinion there are two valid ways in which you may want to implement a filter:

1. An instant, field-based filter such as the inline-filter in the design specs. With instant I mean it will start fetching data whenever the input's value changes. This is suitable if it's likely that the user only want to filter on one field at-a-time and if the GET queries are not too resource-intensive, because these type of filters will produce quite a lot of requests.
2. A combined filter with multiple fields and buttons to apply/cancel/reset. A filter like this makes sense if it's likely that the users wants to combine multiple fields when filtering the data. If the GET query is resource intensive, implementing a filter like this can really help because it will result in fewer requests.

I'd say that the standalone-filter doesn't quite fit into either of the above categories and as such might not have that much use. I'd argue that having all these buttons to control only a single field doesn't make much sense.

The 'Editable columns' example on the design-system page is actually an example 😉. Also, in maintenance app there are situations where elements need to be put in a specific order to be reflected in a form, for example.

OK, noted. I think I have seen both examples you mention in action. One thing these examples have in common is that the table itself is very simple (i.e. just some simple rows without search/filter/paginating/inline-edit/etc). So I think that just making sure the table rows could be used in a DnD UI (probably just make sure they can hold a ref), and perhaps creating a dedicated SortableTable component would be preferred over including a DnD support into the Table itself IMO.

The data-table design documentation falls into the category of 'Pattern'.

Thanks, distinguishing between a "pattern" and a "specification" makes things very clear. And also settles the "support-everything-in-the-component" VS "composable-building-blocks" discussion firmly in favour of the latter.

I think that based on the info provided so far I can start describing the components that would need to be created/updated to facilitate the design-system's specification and patterns using a "composable-building-blocks" approach.

I guess the only thing I need confirmation on from @cooper-joe is whether or not we can drop the standalone-filter.

[cooper-joe]

Thanks for explaining this, the difference in usage is clear. Reg. UI components, would you agree the following is true?

* For searching we don't need anything new, we can just use the `TextField` component (or other available form input components).

* For filtering we definitely need to implement inline filters, and perhaps the standalone filters too (or perhaps cancel these, see next point)

Yup, I agree with those points. (Standalone filters addressed below).

If the standalone filter and the inline filter provide the same functionality and the inline filtered is the preferred method, then would it perhaps make sense to cancel the standalone filter?
...
I guess the only thing I need confirmation on from @cooper-joe is whether or not we can drop the standalone-filter.

Yes, I am on board with that conclusion. Standalone filter does not need to be included as a prebuilt option in this component. If needed, it can be quite easily implemented by an app using the building blocks we provide. I will revisit the usage guidelines and potentially remove as a recommended option, you make some good points above re: network requests and such.

So I think that just making sure the table rows could be used in a DnD UI (probably just make sure they can hold a ref), and perhaps creating a dedicated SortableTable component would be preferred over including a DnD support into the Table itself IMO.

Agreed. I am in favour of including a sortable prop or component variation to ensure consistent usage and placement of the 'draggable' icon.

Drag'n'drop is also needed for the Transfer, so building a generic solution that can be applied to different elements/components/sub-components makes sense to me.

I think that based on the info provided so far I can start describing the components that would need to be created/updated to facilitate the design-system's specification and patterns using a "composable-building-blocks" approach.

Sounds good! 👍 Agreed that the "composable-building-blocks" approach is the way to go.

[ismay]

I agree as well (so going with approach dhis2/ui#2 from the top comment). I think it's the more maintainable approach. From what I understood from @varl about the intention of this lib it fits with how he sees it as well.

This approach also fits with us supplying recipes for users, to show them how our components can be composed into more complicated, common components (like a datatable, etc.).

[Mohammer5]

I second what @ismay said: approach No. 2
Once we have a few apps that have a lot of duplicated code, we can think about extracting some and create specific components that solve specific common problems that are composed from the components we've introduced previously