Thanos High Availability Architecture

[grimz-ly]

I feel I'm overcomplicating my thoughts on this so giving a shot at getting an answer here.

I have the following today

• X Prometheus sidekick hosts in DC A all federated to a single hero host in same DC
• X Prometheus sidekick hosts in DC B all federated to a single hero host in same DC
• same metrics collected, mirrored setup, to have HA
• Grafana dashboards having both hero hosts listed so if one is down, they can choose the other

My thought is to move to the following model to get long term storage and simplify

• X Prometheus sidekick hosts in DC A all w/ 90 days tsdb retention
• X Prometheus sidekick hosts in DC B all w/ 90 days tsdb retention
• all sidekick hosts configured to write to Thanos receiver
• Thanos in DC A keeps metrics for 13 months with no downsampling
• Thanos in DC B keeps 30 days (or less) with no downsampling
• single datasource in Grafana for Thanos that can read metrics from both DCs and deduplicate it

My question is how to architect the Thanos components. A store, query, query frontend, and receiver in each DC sending data to s3 buckets? How then do I query from a single datasource from Grafana? Load balancer?