dodgr_flows_aggregate() weight column unclear

[jonas260492]

I want to calculate the flows in a network for a base scenario and for a scenario where I reduce the speed for an edge to a very high number (I am interested in speed reduction so, I don't want to delete the edge).

The weight column for dodgr_flows_aggregate() is stated as "d_weighted" in the c++ function flows.cpp (line 719).

However, when I change the value of d_weighted to almost infinite the flows on this edge stay the same. There is an alternative route available!

However I found out that entering a very high number in "weight" column leads to almost no more flow. But since this column did not exist in the "graph" before, I cannot understand how it is used.

Also in the "graph" there is an "invisible" column called "d_weight" which contains same values as "d_weighted". You can print this column if you explicitly call it, but it does not show if you print or View the "graph"

Did I oversee something? Who can explain it?

Structure graph:

 head(graph)
  geom_num edge_id    from_id from_lon from_lat      to_id   to_lon   to_lat          d d_weighted highway   way_id component      time time_weighted
1        1       1  339318500 76.47491 15.34167  339318502 76.47612 15.34173 130.000241 130.000241    path 28565950         1 93.600174     93.600174
2        1       2  339318502 76.47612 15.34173  339318500 76.47491 15.34167 130.000241 130.000241    path 28565950         1 93.600174     93.600174
3        1       3  339318502 76.47612 15.34173 2398958028 76.47621 15.34174   8.890622   8.890622    path 28565950         1  6.401248      6.401248
4        1       4 2398958028 76.47621 15.34174  339318502 76.47612 15.34173   8.890622   8.890622    path 28565950         1  6.401248      6.401248
5        1       5 2398958028 76.47621 15.34174 1427116077 76.47628 15.34179   9.307736   9.307736    path 28565950         1  6.701570      6.701570
6        1       6 1427116077 76.47628 15.34179 2398958028 76.47621 15.34174   9.307736   9.307736    path 28565950         1  6.701570      6.701570

Here is a minimum reproducible example:

library(dodgr)

### Step 1: Generate the weighted street network 
graph <- weight_streetnet(hampi, wt_profile = "foot")
dodgr_flowmap(graph, linescale = 1)
#write_sf(dodgr_to_sf(graph),"debugging/Hampi_base.gpkg")

# Step 2: Generate random flows for 200 random 'from'-'to' relations ####
# random points
set.seed(123)  
from <- sample(graph$from_id, size = 200)
to <- sample(graph$to_id, size = 200)

# matrix creation
flows=matrix(data=runif(n=length(from) * length(to),min=1,max=13),
             nrow=length(from),ncol=length(to))

# Step 3: Aggregate flows take time_weighted as the base column ####
graph_with_flows <- dodgr_flows_aggregate(graph, from = from, to = to, flows = flows)
write_sf(dodgr_to_sf(graph_with_flows),"debugging/Hampi_flows.gpkg")


# Step 4: Change weight and aggregate flows ####
graph2=graph
graph2[graph2$edge_id%in%c(2146),c("d_weighted")]=99999999

graph_with_flows2 <- dodgr_flows_aggregate(graph2, from = from, to = to, flows = flows)
graph_with_flows2[graph_with_flows2$edge_id%in%c(2146),]
graph_with_flows[graph_with_flows$edge_id%in%c(2146),]
write_sf(dodgr_to_sf(graph_with_flows2),"debugging/Hampi_flows2.gpkg")


# Step 5: Change weight and aggregate flows ####
graph3=graph
graph3[graph3$edge_id%in%c(2146),c("weight")]=99999999

graph_with_flows3 <- dodgr_flows_aggregate(graph3, from = from, to = to, flows = flows)
graph_with_flows3[graph_with_flows3$edge_id%in%c(2146),]
graph_with_flows[graph_with_flows$edge_id%in%c(2146),]
write_sf(dodgr_to_sf(graph_with_flows3),"debugging/Hampi_flows3.gpkg")

[mpadge]

Thanks @jonas260492, you're correct that there's something odd going on there. The problem arises because of dodgr's internal caching system. A work-around in this case is to simply turn the cache off:

library(dodgr)
dodgr_cache_off ()

And after that point, everything works as expected:

graph <- weight_streetnet(hampi, wt_profile = "foot")

set.seed(123)
from <- sample(graph$from_id, size = 200)
to <- sample(graph$to_id, size = 200)

flows=matrix(
    data=runif(n=length(from) * length(to),min=1,max=13),
    nrow=length(from),ncol=length(to)
)

graph_with_flows <- dodgr_flows_aggregate(graph, from = from, to = to, flows = flows, contract = FALSE)
graph_with_flows [2146, ]
#>      geom_num edge_id    from_id from_lon from_lat      to_id   to_lon   to_lat
#> 2146       70    2146 1376768839 76.46362 15.32709 1376768475 76.46331 15.32718
#>             d d_weighted highway    way_id component     time time_weighted
#> 2146 35.37003   35.37003    path 123463598         1 25.46642      25.46642
#>           flow
#> 2146 0.9961289

graph2 <- graph
graph2[graph2$edge_id%in%c(2146),c("d_weighted")]=99999999
graph_with_flows2 <- dodgr_flows_aggregate(graph2, from = from, to = to, flows = flows, contract = FALSE)
graph_with_flows2[graph_with_flows2$edge_id%in%c(2146),]
#>      geom_num edge_id    from_id from_lon from_lat      to_id   to_lon   to_lat
#> 2146       70    2146 1376768839 76.46362 15.32709 1376768475 76.46331 15.32718
#>             d d_weighted highway    way_id component     time time_weighted
#> 2146 35.37003      1e+08    path 123463598         1 25.46642      25.46642
#>      flow
#> 2146    0

^{Created on 2024-12-06 with reprex v2.1.1}

But that doesn't really help if you don't know what causes the problem in the first place ... 😕

The caching system starts graph contraction in the background as soon as a graph is constructed, so that the whole package functions much quicker. Caching and re-caching require checking for changes in a graph, but graphs are often too big to check everything, so shortcuts are used instead. That means caching won't always catch every change to a graph, which is what's happening here.

I'm not sure at the moment what a solution might be ... I'll think about it and get back to you.

[mpadge]

@jonas260492 Updated the function docs to explain what you're seeing: https://urbananalyst.github.io/dodgr/reference/dodgr_flows_aggregate.html - Let me know if that's clear, or feel free to suggest any improvements to the explanation. Also via PR to the docs if you want. Thanks!

[jonas260492]

Thank you very much for this work-around.
I think the way how you explained it, is clear. However I would add to the documentation for all functions which column is used for least-cost-path calculation.

Still, there is the issue with the "invisible" column of "d_weight" which cannot get deleted and is actually used for the least-cost-path instead of d_weighted.

[mpadge]

Yeah, good point. I'll reopen the issue to address that.