refactor(*.repos): keep autoware repos to be build-able (1st Oct. 2024)

[sasakisasaki]

Description

Fixes the versions in the autoware.repos, simulator.repos, and tools.repos.

In this PR, following files are changed and provided.

• [Changed]: autoware.repos: All the repositories are specified by commit hash or its tag.
• [Added]: autoware-rolling.repos: Same as the current autoware.repos.
• [Changed]: simulator.repos: All the repositories are specified by commit hash or its tag.
• [Added]: simulator-rolling.repos: Same as the current simulator.repos.
• [Changed]: tools.repos: All the repositories are specified by commit hash or its tag.
• [Added]: tools-rolling: Same as the current tools.repos.

Related links

Start of the version control for autoware.universe

Tests performed

Following checks have passed. Please let me know the missing tests. I'll be happy for doing all the necessary tests. Thank you!

• Perform logging simulator
• Perform driving log replayer and checked the result is "passed"
• Checked if the built Autoware is compatible with the AWSIM

Notes for reviewers

If we consider to merge this PR, we might need to update the document related installation such as Source installation.
This PR fixes the version of vcs imported repositories in the .repos files. If you prefer to use the current *.repos file, you need to switch to use *-rolling.repos. Perhaps this causes a confusion for the users (any your ideas are highly appreciated!).

Interface changes

The PR author will do double check.

ROS Topic Changes

The PR author will do double check.

ROS Parameter Changes

The PR author will do double check.

Effects on system behavior

The user needs to switch the .repos files depending on their use cases.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.
• The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

• The PR follows the pull request guidelines.
• The PR has been properly tested.
• The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

• There are no open discussions or they are tracked via tickets.
• The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

[sasakisasaki]

Status:

• I'm doing double check if the provided .repos is build-able and run-able.

[mitsudome-r]

Are there any reasons for downgrading the version?

[sasakisasaki]

Thank you @mitsudome-r san! Actually I tried to fix the version with that of on the 1st Oct. 2024. There is no reason to use the downgraded version. Let me check with the latest version, I think the latest version would work too!

[SakodaShintaro]

It's a very basic question, but does the word “rolling” have a meaning that suggests the latest version?
There is also a version of ROS called “rolling”, and I thought that it might be confusing.

[youtalk]

This method will make maintaining .repos harder, so I'm sorry but I’d like to avoid it. The alternatives I have in mind are as follows:

• *.repos: Keep as is
• *-rolling.repos: Give main for all version values

What I want to achieve with the proposal https://github.com/orgs/autowarefoundation/discussions/5292 is the version management of autoware.universe. However, this PR results in hash management rather than version management, which is already being done by 202X.YZ tags (e.g. https://github.com/autowarefoundation/autoware/blob/2024.09/autoware.repos).

[sasakisasaki]

It's a very basic question, but does the word “rolling” have a meaning that suggests the latest version? There is also a version of ROS called “rolling”, and I thought that it might be confusing.

@SakodaShintaro Thank you for sharing your view! Yes, ... if the rolling is used as one of the fixed version in ROS, the word rolling might give a different meaning depending on the user.

Perhaps autoware-latest.repos is one of the choices.

(CC: @mojomex )

[mojomex]

It's a very basic question, but does the word “rolling” have a meaning that suggests the latest version? There is also a version of ROS called “rolling”, and I thought that it might be confusing.

We were discussing the naming here, and rolling was my suggestion. It should indeed mean "the latest version".

Other possibilities I thought about but did not like as much:

• latest -- is mostly used for the latest release, not the latest (untagged) version
• nightly -- is mostly used for nightly builds (i.e. built once every night) and not for the latest version

rolling as in rolling release is used most commonly for continuous delivery in many projects.

Please let me know if you still prefer other names!

@SakodaShintaro @sasakisasaki

[youtalk]

@mojomex It's a good suggestion. I think nightly is better than rolling since osrf/ros2 docker images use nightly.
https://hub.docker.com/r/osrf/ros2/tags

[SakodaShintaro]

I agree with using 'nightly' as well.

[xmfcx]

*-rolling.repos: Give main for all version values

I agree with everything @youtalk -san says except this one.

Having main or branch in autoware.repos file for everything is too restrictive.

I think we should separate between hardly versioned slowly updated dependencies and more frequently updated dependencies.

I think the quickly changing repos would be:

• autoware.core (not now but as we start populating it)
• autoware.universe
• autoware_launch
• sample_sensor_kit_launch
• awsim_sensor_kit_launch
• awsim_labs_sensor_kit_launch
• single_lidar_sensor_kit_launch
• sample_vehicle_launch
• awsim_labs_vehicle_launch

The reason behind is that we make a lot of changes to these repos all the time and developers and without a rolling/nightly version for these, it will be hard for the developers to deal with versions in these repos.

As you can see, there are a lot of launch file repositories here, I think we must get rid of almost all of them, see my recent discussion: Improve the Autoware Launch System #5313

Truth is, I would prefer there to be only 3 repositories:

• autoware.core
• autoware.universe
• autoware_launch

And the life would be much easier.

But until that is resolved, I would suggest to keep them in 2 versions:

• For stable monthly releases, hash/version them, I don't mind.
• For usual development, keep them on the main branch. For any other repositories, keep them tagged.

ROS 2 way

Please check the structure of ros2.repos file for rolling:
https://github.com/ros2/ros2/blob/rolling/ros2.repos

Some are versioned to tags, some are pointing to rolling branch.

Then how do they not have the problems we have?
It's because they have proper CI set up, when a change is made to any of those repositories, they are tested against to the entire system:

• Avoid redefinition of default color materials (backport #1281) ros2/rviz#1282 (comment)
• Initialize the NULL strucutre with static value. ros2/rmw#378 (comment)

We should adopt a similar system.

Conclusion

Therefore, I think having x.repos and x-nightly.repos doesn't make much sense.

Just release and version like @mitsudome-r does with monthly releases.
And increase the frequency if you need so.
Even up to daily if there is a demand.

And keep the slow dependencies tagged, fast dependencies pointing to branches. And add proper CI to these fast changing repositories to test against the entire system.

[sasakisasaki]

@youtalk @xmfcx Thank you very much for your informative feedback. I strongly believe I should understand how the whole versioning issue will be handled. First of all, please give time for my understanding before doing something.

[sasakisasaki]

@xmfcx @youtalk As we discussed in the software working group, now I think I understand the points. Doing versioning from autoware.repos makes it difficult to control the version on the vcs repositories such as autoware.universe. The versioning step should start from the vcs repository's side, not from autoware.repos.

Though this my PR provides the build-able autoware.repos, this would cause much complexity in the future's version update. Moreover, using commit hash prevents from detecting the version update on the vcs repos side by the workflow (create-prs-to-update-vcs-repositories).

Again, thank you very much @xmfcx and @youtalk for your kind and insightful views. That totally helped me to understand the right steps for the versioning. Let me close this PR. All the discussions in this PR were precious learning for me.