You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#457 wants alternates storage mechanisms, and #471 is trying to fix performance of stacks.
Most performance problems are now in how the stack is rebuilt after map. The remaining fixes for that will involve simplifying the data, layerdims andf layermetadata fields.
all should be Tuple rather than NamedTuple to simplify the type.
keys are in type parameter K
layermetadata can often just be nothing
layerdims can be nothing when all layers have the same dims
combinedims should only happen when it absolutely has to, as it's slow once you have a lot of layers.
rebuild_from_arrays can be very simple and fast in the case where all layers have the same dims.
This could all be handled in a StackLayers object. Meaning it could also be used inside other wrappers like RasterStack with very little work, protecting packages that extend AbstractDimStack from dealing with these internals and performance considerations.
Other benefits are:
Rasters.FileStack could be another <: AbstractStackLayers
we can easily switch between mutable/immutable states by defining conversion between StackLayers and MStackLayers, with no changes required in to the AbstractDimStack wrapper.
We can define setproperty!(stack, ::Symbol) and push!(::AbstractDimStack, ::AbstractDimArray) and forward them to the StackLayers object, getting an error for immutable objects, and updates for MStackLayers.
A few things have led me to this possible refactor. An inner wrapper for stack layers:
#457 wants alternates storage mechanisms, and #471 is trying to fix performance of stacks.
Most performance problems are now in how the stack is rebuilt after
map
. The remaining fixes for that will involve simplifying thedata
,layerdims
andflayermetadata
fields.Tuple
rather thanNamedTuple
to simplify the type.K
layermetadata
can often just benothing
layerdims
can benothing
when all layers have the same dimscombinedims
should only happen when it absolutely has to, as it's slow once you have a lot of layers.rebuild_from_arrays
can be very simple and fast in the case where all layers have the same dims.This could all be handled in a
StackLayers
object. Meaning it could also be used inside other wrappers likeRasterStack
with very little work, protecting packages that extend AbstractDimStack from dealing with these internals and performance considerations.Other benefits are:
Rasters.FileStack
could be another<: AbstractStackLayers
MStackLayers
object that new layers can be pushed to.StackLayers
andMStackLayers
, with no changes required in to theAbstractDimStack
wrapper.setproperty!(stack, ::Symbol)
andpush!(::AbstractDimStack, ::AbstractDimArray)
and forward them to the StackLayers object, getting an error for immutable objects, and updates forMStackLayers
.@sethaxen any thoughts on this?
The text was updated successfully, but these errors were encountered: