https://blog.andygol.co.ua/en/2023/05/07/osm-2-0-api-using-git/

[MTRNord]

Hi for lack of another way of communication :)

I just wanted to chime in that I am currently having fun implementing your blog post. I took some liberty in choice of formats.

https://github.com/MTRNord/osm-git is the repo. I must note that it currently doesn't work production grade.
A major step missing specifically is currently what I wrote down in https://gist.github.com/MTRNord/38ad93a6e4d2f6315517c683d6c11a44 otherwise the repository size skyrockets. (the proposed format is optimized for the size of the repo and partially for usage. But it may not be the best format for every use.) I also took the liberty to diverge from some of the things in your post.

The goal is essentially to first write code to migrate the existing data to the git format that I am defining, and second to provide a webserver API implementing the current API to allow it to be actually used in a useful way. Obviously, converting is a very slow process :) I am hoping to eventually be able to use gitoxide for this since it is a lot faster. However, currently it's not in a usable state for this.

Will it be useful in the end? Time will need to tell. But it is for sure a fun thing to figure out, define and try :)

Your blog post was fascinating to me :) Thanks for writing it!

[Andygol]

Hello, @MTRNord!

I wanted to express my gratitude for your interest in my writing. My intention was to outline the existing limitations of OSM data storage and processing in the draft I shared. I had my hopes up, but I didn't expect anyone to embrace the idea so quickly and take steps in that direction. Since my writing is still a work in progress, there are many aspects that require further development. Among these is the enhancement of the OSM data model.

I've already shared some ideas regarding the rework of the data model in a discussion thread on GitHub, which you can find here: osmlab/osm-data-model#36. However, it is important to note that there is still a significant amount of work that needs to be done in this area. The ideas I presented in the discussion are just the beginning, and they require further exploration, refinement to achieve a comprehensive and effective improvement of the OSM data model. I believe that by addressing the limitations and shortcomings of the current data model, we can unlock new possibilities and enhance the overall usability and functionality of OSM. It's encouraging to see interest and involvement from individuals like you, and I look forward to continuing the conversation and working together to bring about positive changes in the OSM ecosystem.

PS. If you require a private channel for communication, please feel free to send me a direct message on <osm.org/user/andygol>. I am open to discussing further details or addressing any specific concerns you may have through this private channel. Looking forward to connecting with you there!

[MTRNord]

I had my hopes up, but I didn't expect anyone to embrace the idea so quickly and take steps in that direction.

:D It certainly was a "hey I love OSM, I love git, and I can learn here" reaction :) Basically I saw an opportunity to learn about both OSM and git here while doing something I love to do :)

Since my writing is still a work in progress, there are many aspects that require further development. Among these is the enhancement of the OSM data model.

Yes. My initial way of doing this was actually to just throw the YAML files into one folder, just to realize that git really doesn't like this very much at this scale. The 5 GB data in YAML files blew up to 320 GB of git metadata with that. That's how that little piece of folder Spec originated :)

(Also reacting to your next paragraph:)
I intentionally have chosen to not change the format of the data itself too much, since it allows for easier conversion from and to the current XML based format. I just used YAML for git diff benefits and readability. I will still need to have a read on that discussion though :)
Another thing coming to mind is to use the native YAML references feature for relations and ways, for example. But that's nothing I tried yet.

PS. If you require a private channel for communication, please feel free to send me a direct message on <osm.org/user/andygol>. I am open to discussing further details or addressing any specific concerns you may have through this private channel. Looking forward to connecting with you there!

I am happy here as well, if you are happy with that. :) In the end, my project is public and open source. :) So no need to hide talking about this in my opinion. Someone else may stumble over this and might find value in reading this too. :)