Use ORFS from docker image #16

lpawelcz · 2024-03-18T12:37:44Z

This PR modifies the existing rules to run physical design flow in a docker container with installed OpenROAD-flow-scripts.

It uses rules_oci to import and load external docker image into local docker runtime.
The solution is based on generating config.mk files for each design (once per build_openroad()) with new macro called write_config(). The config consist of common and design-specific environment variables, as well as definitions of all rules for make targets used in bazel-orfs.
The config and other inputs to the flow are passed down to the docker container using bind mount mechanism.
Currently it is required to disable bazel's sandboxing feature in order to provide correct paths for mounting directories with input files for the flow.

The flow is started with an entrypoint script based on docker_shell.sh utility imported from OpenROAD-flow-scripts.

oharboe

break into one rules.mk per stage or we have to rebuild everything when we tinker with one rule. redoing a 7200s synthesis is annoying when tweaking e.g. floorplan rules...

Required for passing correct paths to mount in the docker container Signed-off-by: Pawel Czarnecki <[email protected]>

oharboe

Remove util.tcl from bazel-orfs, this is part of what the user of bazel-orfs needs to set up and provide. The user may or may not have a util.tcl.

oharboe · 2024-03-25T09:40:14Z

@lpawelcz I realize this is WIP, but thought I would try it quickly. Removed util.tcl references in bazel-orfs and got this far:

$ bazel build L1MetadataArray_test_generate_abstract
INFO: Invocation ID: 7fa43090-2474-4138-ac1f-4d75ce72668d
ERROR: error loading package '@@bazel-orfs//': Unable to find package for @@rules_oci//oci:defs.bzl: The repository '@@rules_oci' could not be resolved: Repository '@@rules_oci' is not defined.
ERROR: /home/oyvind/megaboom/BUILD.bazel:936:15: error loading package '@@bazel-orfs//': Unable to find package for @@rules_oci//oci:defs.bzl: The repository '@@rules_oci' could not be resolved: Repository '@@rules_oci' is not defined. and referenced by '//:L1MetadataArray_test_generate_abstract'
ERROR: Analysis of target '//:L1MetadataArray_test_generate_abstract' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.262s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

oharboe · 2024-03-25T09:41:21Z

README.md

@@ -100,6 +100,11 @@ The example comes from the `BUILD` file in this repository. For details about ta

 This script loads the ORFS environment and evaluates the rest of the command line that called the script.

+### util.tcl


remove. Not part of bazel-orfs.

oharboe · 2024-03-25T09:42:43Z

generate_abstract-bazel.mk

@@ -5,5 +5,5 @@ bazel-generate_abstract:

 .PHONY: bazel-generate_abstract_mock_area
 bazel-generate_abstract_mock_area: bazel-generate_abstract
-	cp $(RESULTS_DIR)/../mock_area/$(DESIGN_NAME).lef $(RESULTS_DIR)/


hmmm.... was the removal of mock_area intentional here?

If I understand this correctly, this target was meant to copy files from mock_area flow variant output directory to base (or other flow variant provided in one of the build_openroad attributes) output directory.

After my modifications RESULTS_DIR in the context of mock_area stages already points to the directory relevant to mock_area flow variant. We then have to build the path for the other flow variant from scratch.

oharboe · 2024-03-25T09:43:20Z

test/constraints-sram.sdc

@@ -1,4 +1,4 @@
-source util.tcl


move util.tcl into the test/ folder. It is not part of bazel-orfs, but part of the design being built.

oharboe

nits.

Also, we can squash these numerous commits into a single one or a few. There has been a fair deal of back and forth testing out things, the history of trying out things isn't important going forward.

lpawelcz · 2024-03-25T11:13:23Z

Removed util.tcl references in bazel-orfs and got this far

For the time being docker flow is supported and tested only in bazel-orfs repo. If you want to test this there you must first run bazel run orfs_env in order to load the bazel image into your docker runtime. Then you can run e.g. L1MetadataArray targets ~~(expect failures in cts and further stages)~~.

EDIT:
After bumping the docker ORFS image all stage targets from L1MetadataArray should work

we can squash these numerous commits into a single one or a few

Sure. For now this is WIP. At this stage I prefer to keep small commits in case something has to be reverted or slightly modified. I will surely clean the history before putting up this work for a review.

lpawelcz · 2024-03-28T13:38:35Z

Reworked TCL and constraints handling, cleaned up the code responsible for flow arguments, source and outputs specification.
Added generating configs per stage (those are included in config generated per design by new env var STAGE_CONFIG).
Fixed the local flow after adding docker flow.
Enabled CI tests for docker flow and reworked local flow.

The plan for now is to add modified version of L1MetadataArray with enabled grt and final stages. Then we can work on https://github.com/Pinata-Consulting/ascenium-antmicro/issues/15.
Please note that those changes were not yet tested in megaboom and we expect some failures there.
Once we have grt and final stages tested here we will start integrating and testing the changes in megaboom workspace

Signed-off-by: Pawel Czarnecki <[email protected]>

Import slightly modified docker_shell.sh utility from ORFS Signed-off-by: Pawel Czarnecki <[email protected]>

Signed-off-by: Pawel Czarnecki <[email protected]>

…unction Signed-off-by: Pawel Czarnecki <[email protected]>

Assume util.tcl is always a dependency for synth, floorplan and place stages. Introduce UTIL_TCL env var which should be used for including util.tcl in different scripts Signed-off-by: Pawel Czarnecki <[email protected]>

Signed-off-by: Pawel Czarnecki <[email protected]>

…parate functions Signed-off-by: Pawel Czarnecki <[email protected]>

Signed-off-by: Pawel Czarnecki <[email protected]>

Signed-off-by: Paweł Czarnecki <[email protected]>

* Run job in default GitHub Runner container * Load the ORFS image before running bazel targets Signed-off-by: Pawel Czarnecki <[email protected]>

Signed-off-by: Pawel Czarnecki <[email protected]>

Add modified configuration for L1MetadataArray design Perform additional 'route' and 'flow' stages Signed-off-by: Paweł Czarnecki <[email protected]>

Signed-off-by: Pawel Czarnecki <[email protected]>

…nd Make version Signed-off-by: Eryk Szpotanski <[email protected]>

Signed-off-by: Eryk Szpotanski <[email protected]>

Check if it is possible to locally continue physical design flow which was started with the docker flow. Signed-off-by: Eryk Szpotanski <[email protected]>

…lues in GDS_ALLOW_EMPTY Signed-off-by: Eryk Szpotanski <[email protected]>

Signed-off-by: Eryk Szpotanski <[email protected]>

Signed-off-by: Pawel Czarnecki <[email protected]>

Signed-off-by: Eryk Szpotanski <[email protected]>

oharboe · 2024-04-11T14:13:30Z

@lpawelcz ready?

oharboe · 2024-04-11T14:18:12Z

@lpawelcz I need an accompanying megaboom pull request...

lpawelcz · 2024-04-11T14:18:58Z

@oharboe ready, here is the megaboom PR: The-OpenROAD-Project/megaboom#29

oharboe · 2024-04-11T14:29:54Z

Continue testing and file github issues based upon master.

oharboe requested changes Mar 18, 2024

View reviewed changes

lpawelcz force-pushed the 56027-use-docker-orfs branch 2 times, most recently from 115fce0 to 03c11cb Compare March 22, 2024 12:52

Bazel: Disable sandboxing

3f37684

Required for passing correct paths to mount in the docker container Signed-off-by: Pawel Czarnecki <[email protected]>

lpawelcz force-pushed the 56027-use-docker-orfs branch from 03c11cb to e596cf7 Compare March 25, 2024 09:26

oharboe self-requested a review March 25, 2024 09:38

oharboe requested changes Mar 25, 2024

View reviewed changes

oharboe reviewed Mar 25, 2024

View reviewed changes

oharboe requested changes Mar 25, 2024

View reviewed changes

lpawelcz force-pushed the 56027-use-docker-orfs branch from 8794666 to e62cdac Compare March 27, 2024 11:31

lpawelcz force-pushed the 56027-use-docker-orfs branch from 6e0ec05 to eafbf96 Compare March 29, 2024 09:43

lpawelcz added 14 commits April 11, 2024 15:07

Introduce ORFS Docker image

bd0131e

Signed-off-by: Pawel Czarnecki <[email protected]>

Import docker_shell.sh

4dd5a91

Import slightly modified docker_shell.sh utility from ORFS Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: Add macro for generating config files

fc26801

Signed-off-by: Pawel Czarnecki <[email protected]>

Use ORFS from docker image

910f38a

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: set docker_shell commandline in a separate helper function

aaaa411

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: move _mock_area targets generation to a separate macro

28122c3

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: Specify make targets to execute in ORFS in a separate f…

5cbb9e5

…unction Signed-off-by: Pawel Czarnecki <[email protected]>

Rework util.tcl handling

49d7b61

Assume util.tcl is always a dependency for synth, floorplan and place stages. Introduce UTIL_TCL env var which should be used for including util.tcl in different scripts Signed-off-by: Pawel Czarnecki <[email protected]>

Fix generate_abstract stage

9826ec7

Signed-off-by: Pawel Czarnecki <[email protected]>

Fix synth_sdc for L1MetadataArray

aadb9ae

Signed-off-by: Pawel Czarnecki <[email protected]>

Rework TCL handling

489c4e4

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: Improve stage dictionaries handling

23def9d

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: move outputs and reports dictionary specification to se…

a96c71a

…parate functions Signed-off-by: Pawel Czarnecki <[email protected]>

Generate configs per stage

4cb6262

Signed-off-by: Pawel Czarnecki <[email protected]>

lpawelcz and others added 19 commits April 11, 2024 15:51

Fix local flow

f0a79b8

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: stage targets: synthesis depends on clock_period stage

519ac33

Signed-off-by: Paweł Czarnecki <[email protected]>

CI: Enable tests of _stage targets with docker flow

42ed8b5

* Run job in default GitHub Runner container * Load the ORFS image before running bazel targets Signed-off-by: Pawel Czarnecki <[email protected]>

CI: clean disk space on GitHub runner

fbfcd2a

Signed-off-by: Pawel Czarnecki <[email protected]>

CI: local flow: run all targets through local flow

0d7bbbb

Signed-off-by: Pawel Czarnecki <[email protected]>

Introduce full L1MetadataArray build

d1414e5

Add modified configuration for L1MetadataArray design Perform additional 'route' and 'flow' stages Signed-off-by: Paweł Czarnecki <[email protected]>

README: update after adding docker flow

2ec0a45

Signed-off-by: Pawel Czarnecki <[email protected]>

Dockerfile: install docker

905ac3d

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: memory target: Fix no rule to make target, add Docker a…

5510b9a

…nd Make version Signed-off-by: Eryk Szpotanski <[email protected]>

docker_shell: Add parameter for mounting cache

15a9223

Signed-off-by: Eryk Szpotanski <[email protected]>

CI: Add tests for _memory target

750fe08

Signed-off-by: Eryk Szpotanski <[email protected]>

CI: Implement test using stage target and make scripts

089efaa

Check if it is possible to locally continue physical design flow which was started with the docker flow. Signed-off-by: Eryk Szpotanski <[email protected]>

BUILD: Create rule for lb_32x128 and L1MetadataArray with multiple va…

d4cd578

…lues in GDS_ALLOW_EMPTY Signed-off-by: Eryk Szpotanski <[email protected]>

CI: Add tests for multiple values in GDS_ALLOW_EMPTY

abaccf8

Signed-off-by: Eryk Szpotanski <[email protected]>

openroad.bzl: fix constraint handling when attribute is empty

fb542fa

Signed-off-by: Pawel Czarnecki <[email protected]>

openroad.bzl: generalize path handling

758f74a

Signed-off-by: Pawel Czarnecki <[email protected]>

docker_shell: generalize path handling

78536a1

Signed-off-by: Pawel Czarnecki <[email protected]>

Rework mock_area targets

53389b3

Signed-off-by: Pawel Czarnecki <[email protected]>

Use Bzlmod project configuration

9c6a77e

Signed-off-by: Eryk Szpotanski <[email protected]>

lpawelcz force-pushed the 56027-use-docker-orfs branch from c653c41 to 9c6a77e Compare April 11, 2024 13:55

lpawelcz marked this pull request as ready for review April 11, 2024 14:18

lpawelcz mentioned this pull request Apr 11, 2024

Introduce bazel-orfs docker flow The-OpenROAD-Project/megaboom#29

Merged

lpawelcz changed the title ~~[WIP] Use ORFS from docker image~~ Use ORFS from docker image Apr 11, 2024

oharboe merged commit 5a76d99 into The-OpenROAD-Project:main Apr 11, 2024
8 checks passed

lpawelcz deleted the 56027-use-docker-orfs branch April 11, 2024 14:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use ORFS from docker image #16

Use ORFS from docker image #16

lpawelcz commented Mar 18, 2024 •

edited

Loading

oharboe left a comment

oharboe left a comment

oharboe commented Mar 25, 2024

oharboe Mar 25, 2024

lpawelcz Mar 28, 2024

oharboe Mar 25, 2024

lpawelcz Mar 25, 2024

oharboe Mar 25, 2024

lpawelcz Mar 28, 2024

oharboe left a comment

lpawelcz commented Mar 25, 2024 •

edited

Loading

lpawelcz commented Mar 28, 2024

oharboe commented Apr 11, 2024

oharboe commented Apr 11, 2024

lpawelcz commented Apr 11, 2024

oharboe commented Apr 11, 2024

		@@ -100,6 +100,11 @@ The example comes from the `BUILD` file in this repository. For details about ta

		This script loads the ORFS environment and evaluates the rest of the command line that called the script.

		### util.tcl

Use ORFS from docker image #16

Use ORFS from docker image #16

Conversation

lpawelcz commented Mar 18, 2024 • edited Loading

oharboe left a comment

Choose a reason for hiding this comment

oharboe left a comment

Choose a reason for hiding this comment

oharboe commented Mar 25, 2024

oharboe Mar 25, 2024

Choose a reason for hiding this comment

lpawelcz Mar 28, 2024

Choose a reason for hiding this comment

oharboe Mar 25, 2024

Choose a reason for hiding this comment

lpawelcz Mar 25, 2024

Choose a reason for hiding this comment

oharboe Mar 25, 2024

Choose a reason for hiding this comment

lpawelcz Mar 28, 2024

Choose a reason for hiding this comment

oharboe left a comment

Choose a reason for hiding this comment

lpawelcz commented Mar 25, 2024 • edited Loading

lpawelcz commented Mar 28, 2024

oharboe commented Apr 11, 2024

oharboe commented Apr 11, 2024

lpawelcz commented Apr 11, 2024

oharboe commented Apr 11, 2024

lpawelcz commented Mar 18, 2024 •

edited

Loading

lpawelcz commented Mar 25, 2024 •

edited

Loading