Implement object sharing inside the exnode

[tacketar]

The old, pre-refactor, method had simple reference counts embedded in each object and the objects themselves did any cleanup. The new method using the tbx/{ref.h|objects.h} has the inc/dec along with object destruction combined. This lead to some logic changes to make sure things were properly destroyed. More importantly it highlighted the the difficulty to actually share and reuse objects.

Segment and data block reuse should be possible and encouraged to minimize data caching if nothing else. It naturally occurs when using the log segment drivers. Each log is essentially a snapshot of the file and logs can be layered on top of each other and can be exposed via the [view] section of the exnode. The current method would have the different snaps replicated.

A better solution would be to combine to the current approach with the lio/service_manager.c routines. The server_manager routines are already passed around and used as a grab bag to store drivers(segment drives, object service, data service, resource service, etc). Storing segments and data blocks in it as well would make it easy to share objects.

[PerilousApricot]

I'm not entirely sure of what you're proposing here. Mind fleshing it out?

[tacketar]

Exnodes are comprised of data blocks and segments. These are made available to applications through "views" defined in the [view] section of the exnode. The view section has at a minimum a default view, "default = ID" and an optional collection of "segment = ID" entries. The ID refers to the section, [segment-ID], defining the segment to load. Up until now we've only had the default view. As we start integrating with AuriStor and add support for snapshots these snapshots will show up as extra segment entries in the [view] section. There are other use cases you can envision as well. For example storing a matrix in both row- and col-major formats, using a log to keep frequently used data local, etc.

The duplication problem is easiest to describe using snapshots. Snapshots will be heavily based on using the log structured segment driver. Each log structured segment is comprised three other segments - extents, data changes, and base. The extents and data changes comprise the changes from the base. These individual segments will be comprised of other segments loaded recursively same as they are now. Using a typical exnode now this would mean each of these view segments would be comprised of a cache->jerasure->lun->data_blocks chain.

The current reference counting is pretty useless because they are only used within an individual chain. There's no way to reference objects from a different chain. As a result the same segment can be loaded multiple times creating cache inconsistencies and data duplication. Take for example an exnode that has 4 views:

• Base
• Snap-1
• Snap-2
• Snap-3

With the Segment sequences defined as:

• Base ->cache-base->jerasure-base->lun-base->data_blocks-base
• Snap-1->log-1->{extent-1, changes-1, Base}
• Snap-2->log-2->{extent-2, changes-2, log-1}
• Snap-3->log-3->{extent-3, changes-3, log-2}

Notice that snap-3 recursively loads snap-2 which loads snap-1 which loads the base. Currently no method exists to find currently loaded segments as a result each of these view chains duplicates all the internal segments.

The goal is to implement a method to share these segment objects and eliminate duplication. There currently exists a method to share objects across all layers defined in lio/service_manager.c. It provides a grab bag of objects sorted similar to an INI file searchable by sections and key/value pairs. This is where all the different OS, DS, segment, etc drivers are stored along with any other objects needed. This is already passed down through all the segments and used to find the proper driver for child segments.

The service_manager.c routines don't do any object reference counting though. It has a simple add/remove interface. The tbx_objects/ref.c routines do track references and implement object destruction. So combining these two routines can provide an easy way to make a grab bag of searchable objects that has proper reference counting to support object creation and eliminate segment duplication.

To complicate things the lc_object{get|put} routines defined in lio_config.c kind-of does this for stuff in the config and it doesn't use any of the tbx_objects/ref.c routines. So all of these should be rethought and reimplemented under a coherent framework.

[PerilousApricot]

So (roughly) a hash table where the keys are strings and the values are
pointers to reference counted objects?

It's dark in this basement.

[tacketar]

Yes.

[PerilousApricot]

Gotcha. I never messed with the service manager, but tying the refcounts to destruction sounds like it'll be really helpful for this usecase.

[tacketar]

Also need to fold in the data_block.c reference counting