Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(workflow): Add esp-idf network examples build #124

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

roma-jam
Copy link
Collaborator

@roma-jam roma-jam commented Jan 21, 2025

Description

This is a 3/3 PR to verify fixing the problem.

PR plan:

  1. Add all esp-idf releases to the current CI build process (ci(workflow): Added esp-idf releases matrix for examples build #123)
  2. Fix the NCM Driver memory usage on ESP32S2 (fix(ncm): Changed NTB default value to fix DRAM overflow on esp32s2 #125)
  3. Add network examples to the CI build process (this PR)

Related

Testing

N/A


Checklist

Before submitting a Pull Request, please ensure the following:

  • 🚨 This PR does not introduce breaking changes.
  • All CI checks (GH Actions) pass.
  • Documentation is updated as needed.
  • Tests are updated or added as necessary.
  • Code is well-commented, especially in complex areas.
  • Git history is clean — commits are squashed to the minimum necessary.

@roma-jam roma-jam added this to the esp_tinyusb v1.7.1 milestone Jan 21, 2025
@roma-jam roma-jam self-assigned this Jan 21, 2025
Base automatically changed from ci/build_all_esp_idf to master January 21, 2025 14:11
@roma-jam roma-jam force-pushed the ci/add_esp_idf_network_examples branch from 084d360 to 2817f4a Compare January 21, 2025 14:19
@roma-jam roma-jam changed the base branch from master to fix/ncm_buffer_count_default_value January 21, 2025 14:20
@roma-jam roma-jam marked this pull request as ready for review January 21, 2025 14:54
@roma-jam roma-jam mentioned this pull request Jan 21, 2025
6 tasks
Base automatically changed from fix/ncm_buffer_count_default_value to master January 21, 2025 16:08
@roma-jam roma-jam force-pushed the ci/add_esp_idf_network_examples branch from 2817f4a to 56cfcbb Compare January 21, 2025 16:08
Copy link
Collaborator

@peter-marcisovsky peter-marcisovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thx for the update.

.github/workflows/build_idf_examples.yml Outdated Show resolved Hide resolved
.github/workflows/build_idf_examples.yml Show resolved Hide resolved
@roma-jam roma-jam force-pushed the ci/add_esp_idf_network_examples branch from 17462de to 3c4828c Compare January 22, 2025 09:29
Copy link
Collaborator

@tore-espressif tore-espressif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roma-jam I can see that the CI is doing what you need, but it seems that it can be optimized a little

@@ -47,6 +48,11 @@ def override_with_local_component_all(component, local_path, apps):

# Go through all collected apps
for app in apps_with_glob:
# Verify that the app is a valid directory
if not path.isdir(app):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use modern pathlib.Path instead of os.path in this file. Lets keep it that way for consistency

Suggested change
if not path.isdir(app):
if not Path(app).is_dir():

@@ -11,18 +11,38 @@ jobs:
strategy:
matrix:
idf_ver: ["release-v5.0", "release-v5.1", "release-v5.2", "release-v5.3", "release-v5.4", "latest"]
target: ["esp32s2", "esp32s3", "esp32p4"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, we build all targets in one job. Now we have one job for each target, which is slower.

Could you please check if idf-build-apps --target all can handle the idf version correctly? Eg. it will not build for esp32p4 on idf release which do not support P4?

Please test this locally, so we do not onverload the test runners...

Copy link
Collaborator

@tore-espressif tore-espressif Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just confirmed this works. idf-build-apps --target all on idf v5.1.2 will not list ESP32-P4 target

FYI like this

[(v5.1.2)]> idf-build-apps find -p examples/peripherals/usb/host --recursive --manifest-file examples/peripherals/.build-test-rules.yml

Comment on lines +15 to +17
example:
- { name: "USB Device", path: "examples/peripherals/usb/device", manifest_path: "examples/peripherals"}
- { name: "Network", path: "examples/network", manifest_path: "examples/network" }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This creates another diminsion of the matrix, now we have 23 jobs for simple example builds, this should be optimized

number of jobs: idf_ver * target * example

image

Comment on lines +47 to +48
idf-build-apps find --path ${{ matrix.example.path }} --recursive --target ${{ matrix.target }} --manifest-file ${{ matrix.example.manifest_path }}/.build-test-rules.yml --build-dir build_@t_@w --work-dir @f_@t_@w
idf-build-apps build --path ${{ matrix.example.path }} --recursive --target ${{ matrix.target }} --manifest-file ${{ matrix.example.manifest_path }}/.build-test-rules.yml --build-dir build_@t_@w --work-dir @f_@t_@w
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--manifest-files accepts multiple parameters, so you can pass both .build-test-rules.yml to one command

@roma-jam
Copy link
Collaborator Author

@tore-espressif Thanks for the review, will address all the notes asap.

@roma-jam
Copy link
Collaborator Author

@tore-espressif I think optimization could take some time, so I think that this feature (have a example/network in CI) is not that important for the current fix-release.

I think we might remove this PR from the release, to push it asap to get back on track.
And then add the CI verification of network (or implement any other way of DRAM usage, I will think about how we can do it) later.

For fix-release, PTAL: #126

@roma-jam roma-jam marked this pull request as draft January 22, 2025 21:36
@roma-jam roma-jam removed this from the esp_tinyusb v1.7.1 milestone Jan 22, 2025
@tore-espressif
Copy link
Collaborator

OK, agreed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants