[eder-matheus]

Summary

OpenROAD stages read binary liberty (.blib) files from a cache shared by all designs, instead of parsing the text libraries in every stage. Supersedes #4682.

• The cache is objects/binary_libs/<platform>/<library>@<size>-<mtime>-<dirhash>.blib, so a changed library gets a new entry. Libraries in the design directory, or generated by the flow, are cached in the run's own objects directory.
• The libraries are converted before 1_synth.odb, the first OpenROAD step, and again when one changes. A no-op make converts nothing. do-<step>, run and the GUI targets never convert. Yosys reads the text libraries.
• Each entry is written to a temporary file and renamed into place, so concurrent flows never read a partial file.
• A library whose cached .blib is missing or unreadable is read as given, so flows that bring their own binary libraries, like bazel-orfs, need no setting.
• make binary-libs converts on demand, and make clean_binary_libs deletes the cache.
• genReport.py reports the most recently written log as the last one.

Type of Change

New feature.

Impact

Liberty reads are about 5x faster: 677 s to 135 s over 16 stages on a private PDK with 44 libraries.

Verification

• Earlier CI runs of this change on OpenROAD b9ec64c305 matched nightly 7662, which ran the same ORFS base and OpenROAD with the text libraries, on every metric of all 62 designs.
• Locally, gcd and i2c-gpio-expander (with BLOCKS and CORNERS) give the same results as with the text libraries. Also tested: concurrent conversions, changed libraries, entries from another format version, an empty cache and a no-op make.

[gemini-code-assist]

Code Review

This pull request introduces a binary liberty (.blib) caching mechanism to optimize OpenROAD stages by reading binary files instead of parsing text libraries. It adds configuration variables BINARY_LIBS_DIR and USE_BINARY_LIBS, updates documentation, and includes scripts to manage and load the cached files. The review feedback points out a potential issue in the Makefile where BINARY_LIBS_DIR should be quoted during rm -rf to prevent accidental deletions if the path contains spaces.

[gemini-code-assist]

If BINARY_LIBS_DIR contains spaces, the rm -rf command will undergo word splitting, potentially leading to accidental deletion of unintended files or directories. Quoting the expanded path prevents this issue.

	rm -rf "$(if $(strip $(BINARY_LIBS_DIR)),$(BINARY_LIBS_DIR),$(error BINARY_LIBS_DIR is empty))"

[openroad-ci]

🔍 QoR check

Metrics reflect the PR merge build — i.e. what will land on the target branch.

Commit be90e77 · Jenkins build #10 · Baseline: build · View build on dashboard

62 design(s) checked — 9 with regression(s), 0 without a comparable baseline.

❌ asap7/chi_async_bridge base — 4 failing metric(s)

Metric	target	base	delta	limit	band
cts__timing__setup__tns	-18795.100	-10700.300	-75.650%	-12840.360	PeriodPadding 20.000%
cts__timing__setup__ws	-35.572	-16.626	-113.957%	-31.126	PeriodPadding 5.000%
finish__timing__setup__tns	-4654.780	-3518.420	-32.297%	-4222.104	PeriodPadding 20.000%
globalroute__timing__setup__tns	-7287.390	-4351.980	-67.450%	-5222.376	PeriodPadding 20.000%

❌ asap7/ethmac_lvt base — 3 failing metric(s)

Metric	target	base	delta	limit	band
cts__timing__setup__tns	-319.418	-218.431	-46.233%	-278.431	PeriodPadding 20.000%
finish__timing__setup__tns	-289.589	-178.186	-62.521%	-238.186	PeriodPadding 20.000%
globalroute__timing__setup__tns	-418.170	-307.720	-35.893%	-369.264	PeriodPadding 20.000%

❌ asap7/gcd base — 1 failing metric(s)

Metric	target	base	delta	limit	band
cts__timing__setup__tns	-214.517	-110.067	-94.897%	-172.067	PeriodPadding 20.000%

❌ asap7/swerv_wrapper base — 4 failing metric(s)

Metric	target	base	delta	limit	band
finish__timing__hold__tns	-21627.100	0.000	—	-260.000	PeriodPadding 20.000%
finish__timing__hold__ws	-251.846	15.821	-1691.876%	-65.000	PeriodPadding 5.000%
finish__timing__setup__tns	-14928.200	-2532.010	-489.579%	-3038.412	PeriodPadding 20.000%
finish__timing__setup__ws	-340.516	-108.880	-212.744%	-173.880	PeriodPadding 5.000%

❌ gt2n/jpeg base — 1 failing metric(s)

Metric	target	base	delta	limit	band
cts__timing__setup__tns	-1084.510	-715.977	-51.473%	-859.172	PeriodPadding 20.000%

❌ nangate45/bp_be_top base — 1 failing metric(s)

Metric	target	base	delta	limit	band
finish__timing__setup__tns	-0.835	-0.0145	-5667.962%	-0.534	PeriodPadding 20.000%

❌ nangate45/tinyRocket base — 1 failing metric(s)

Metric	target	base	delta	limit	band
globalroute__timing__setup__tns	-34.520	-28.060	-23.022%	-33.672	PeriodPadding 20.000%

❌ sky130hd/jpeg base — 3 failing metric(s)

Metric	target	base	delta	limit	band
cts__timing__setup__tns	-97.688	-71.034	-37.523%	-85.241	PeriodPadding 20.000%
finish__timing__setup__tns	-88.855	-64.055	-38.715%	-76.866	PeriodPadding 20.000%
globalroute__timing__setup__tns	-106.657	-71.192	-49.816%	-85.431	PeriodPadding 20.000%

❌ sky130hs/gcd base — 2 failing metric(s)

Metric	target	base	delta	limit	band
detailedroute__route__wirelength	10851.000	9432.000	15.045%	10846.800	Padding 15.000%
globalroute__timing__setup__tns	-10.052	-8.233	-22.084%	-9.880	PeriodPadding 20.000%

[eder-matheus]

@codex review

[chatgpt-codex-connector]

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[eder-matheus]

@oharboe The new USE_BINARY_LIBS should help with the bazel-orfs. Could you take a look?

[oharboe]

A suggestion that would let flows which already provide binary liberty files work with the defaults, without opting out.

read_liberty recognises a binary library by its contents, not its name. lib_file_to_read and write_binary_libs.tcl decide by name: they strip .gz/.lib, then look for <name>@<size>-<mtime>-<dirhash>.blib in the cache. A LIB_FILES entry that is already binary therefore gets:

• an FLW-0026 warning for every library in every stage, because there is no cache entry for it. It's harmless, since the "fallback" reads a binary file anyway, but it's noise in every log;
• a reconversion attempt in do-binary-libs, which fails with FLW-0028.

This happens in bazel-orfs. It converts each platform library once, as a cached build action, and gives the OpenROAD stages a copy of the platform where every liberty file is binary under its original name, so foo.lib.gz holds a .blib. The platform's config.mk builds LIB_FILES unchanged, and that has relied on read_liberty sniffing the contents.

Proposed change: if a library is already binary, use it as given.

proc lib_file_to_read { lib_file } {
  if { ![env_var_equals USE_BINARY_LIBS 1] || ![file exists $lib_file] } {
    return $lib_file
  }
  # Already binary: read it as given, whatever its name
  if { ![catch { liberty_binary_info $lib_file }] } {
    return $lib_file
  }
  ...

and the same check at the top of the loop in write_binary_libs.tcl, so binary inputs are skipped, not converted.

With this, USE_BINARY_LIBS=0 is no longer needed for that case and could be documented as a plain opt-out. bazel-orfs will still set it, to make clear that it converts the libraries itself.

Nit: I would discourage the new FLW-0026/0027/0028 messages. Message IDs belong to OpenROAD, where each tool owns a range and uniqueness is checked. Nothing checks ORFS's numbers: FLW 25 already names two different messages. See the Tcl contract proposed in #4692 ("A script never adds error numbers or checks that belong in OpenROAD"). A plain puts or error that says what happened is enough here. FLW-0026 in particular fires for every library in every stage of any flow that runs only do-* targets, which never convert, so the warning is the normal case there and not a sign that anything is wrong.

[oharboe]

@eder-matheus Thanks for adding USE_BINARY_LIBS with bazel-orfs in mind. Two small changes would let you drop it altogether: one variable fewer to document and support, and the cache works with the defaults for every flow. bazel-orfs gets simpler too, since it has nothing to set.

1. Read an already-binary library as given (liberty_binary_info succeeds), as in the snippet above. bazel-orfs's OpenROAD stages read a binary copy of the platform, so they need no cache.
2. Fall back to the text library silently on a cache miss. That covers the text libraries bazel-orfs still passes (macro .lib files from other flows, and the synthesis stage's text platform), and every do-* run, which never converts.

With both in place, USE_BINARY_LIBS, its ifeq in the Makefile, and its entries in variables.yaml, variables.json and FlowVariables.md can go. Every flow, make or Bazel, gets the speedup with no setting.

[eder-matheus]

@oharboe Thanks, both comments are addressed in 95f03da:

• USE_BINARY_LIBS is gone, along with its ifeq and docs. bazel-orfs don't need to set anything.
• "A cache miss falls back silently. When a library's cached .blib is missing or unreadable, the library is read as given. The log still shows which file each stage read, because read_liberty is logged.
• The FLW IDs are gone, replaced by a plain message when a conversion fails.

The binary check turned out to be unnecessary:

• In lib_file_to_read, a miss already returns the file as given, binary or text, so your mirror is read unchanged.
• In write_binary_libs.tcl, a binary library can't reach the conversion. In the make flow, Yosys reads LIB_FILES first and would fail on it. bazel-orfs only runs do- targets, and none of them converts.

Following the Makefile contract, 1_synth.odb now depends on the binary-libs alias, which calls do-binary-libs the same way generate_abstract calls its do- target.

[oharboe]

@eder-matheus Thanks, this is much simpler now. bazel-orfs is fine as it is, both in how it works with this and for its users: it runs only do-* targets, and its binary platform is read as given.

The suggestions below are for ORFS users: fewer things to know about and less to maintain.

1. BINARY_LIBS_DIR is the only way the cache can race, so I'd drop it. The default under objects/ is already shared by every design in a clone, and one clone runs one OpenROAD. Pointing several clones at one directory adds a check-then-read race:

   • lib_file_to_read checks an entry and read_liberty opens it later.
   • In between, a flow running another OpenROAD version can find the entry unreadable and replace it with file rename -force.
   • The reader then fails in read_liberty, where nothing falls back to the text library.

   Locally nobody will see this. In CI, where builds with different OpenROAD versions run side by side, anything that can happen will. Sharing between clones is a local turnaround trick, not something the flow needs to support. Without the variable, the docs entries, the empty-value guard and clean_binary_libs go too, and make nuke already removes the default location.

2. Make the cache easy to turn off, in one line. If .blib reading ever misbehaves (a QoR difference, a library the binary format doesn't handle, a version mismatch), the fix should be a revert so small that reviewers merge it the same day. It also lets anyone chasing a QoR difference compare text and binary reads with one edit. Today, read_liberty.tcl calls lib_file_to_read in both the corner and no-corner branches. Call it once, and an early return $lib_file in that proc turns the cache off.

3. Every make starts OpenROAD and changes the reported last log. A no-op make, or a rerun of a finished design, should do nothing. Here it starts OpenROAD and checks every library. It also rewrites write_binary_libs.log, which then becomes the newest log, and the genReport.py change reports it as last_log. A finished design is then reported as having last run the conversion. The cause is that binary-libs is .PHONY, and make still runs a phony prerequisite even when it is order-only. A stamp file instead of a phony target, or keeping the conversion log out of the stage logs, fixes it and lets the genReport.py change be reverted.

4. CI needs an explanation before merge. Jenkins build Allow multiple whitespace characters after END of lef files #3 errored (pr-head and pr-merge), and its QoR check flags 10 designs with regressions on the merge of 95f03da. The description says the metrics match nightly 7662, so something has to explain the difference. Is it the OpenROAD bump? Bumping OpenROAD in a separate PR first would show which change moved the metrics.

Minor: binary_lib_usable calls unsuppress_message, which resets the message counter to 0. If the user had suppressed STA 1900/1902 themselves, those messages come back after the first library check.

[eder-matheus]

@oharboe Thanks, all four points are addressed:

1. BINARY_LIBS_DIR is gone. The cache is always $(WORK_HOME)/objects/binary_libs, so each clone has its own and OpenROAD versions can't race on it. I kept make clean_binary_libs, now a one-line rm -rf, because make nuke also removes every design's results.
2. The one-line switch already exists. Both branches of read_liberty.tcl call lib_file_to_read, so a return $lib_file at its top turns the cache off.
3. A no-op make does nothing now. $(LOG_DIR)/write_binary_libs.log is a file target that depends on LIB_FILES, and an order-only prerequisite of 1_synth.odb. A changed library is converted again. A change make can't see, such as a library added to the list without being newer than the log, means that library is read as text until make binary-libs.
   The genReport.py change stays. The original bug was name order: write_binary_libs.log sorts after the numbered logs, so a stamp alone wouldn't fix it. Also, last_log is only printed for designs that didn't finish, so a finished design is never reported as having last run the conversion.
4. The OpenROAD bump is gone; the branch keeps master's pin. The earlier builds' QoR check compared against master builds on the old OpenROAD. All three builds, head and merge, matched nightly 7662 on every metric of all 62 designs; that build ran the same ORFS base and OpenROAD b9ec64c305 with the text libraries. The error status was the QoR check failing.

On the minor point: unsuppress_message does reset the counter, and the logger has no call to read the previous state. No ORFS script or platform suppresses STA 1900 or 1902, so I'd leave it rather than add code for a case the flow doesn't have.

I also merged master and, following the Tcl contract, removed the file exists checks on the libraries, since the flow declares them as inputs.

[eder-matheus]

@maliberty @dsengupta0628 I believe this PR is ready for review

[oharboe]

1. Put the cache in $(OBJECTS_DIR)/binary_libs, not $(WORK_HOME)/objects/binary_libs. That one change removes a lot of code:

   • In binary_lib_path, the run_dirs loop goes, along with the DESIGN_DIR and results special cases and the <platform> subdirectory. Every library maps to the same directory.
   • clean_binary_libs goes, because clean_all already does rm -rf $(OBJECTS_DIR).
   • The temp file and rename in write_binary_libs.tcl (.tmp.<hostname>.<pid>) goes. Only one make writes to a given OBJECTS_DIR, so no other flow can read an entry while it is half written.

   Each design then converts its own libraries, once per design rather than once per stage, so the 5x per-stage speedup stays.

2. Drop <dirhash> from the entry name, and refuse duplicates instead. Keep <name>@<size>-<mtime>.blib. If two entries of LIB_FILES map to the same name, stop with an error that names both paths. A crc32 of the directory hides a collision instead of reporting it.

3. Move the stamp out of LOG_DIR and drop the genReport.py change. Make $(OBJECTS_DIR)/binary_libs/write_binary_libs.log the target, as the order-only prerequisite of 1_synth.odb. Then nothing changes how logs sort, and genReport.py keeps reporting what it reported before.

4. Skip libraries that are already binary. In both lib_file_to_read and write_binary_libs.tcl, if liberty_binary_info $lib_file succeeds, use the file as given and don't compute a cache path. Flows that bring their own .blib files, bazel-orfs among them, are then left alone because of that check, not because no cache entry happens to exist. Two things go with it:

   • make binary-libs no longer converts a .blib into another .blib.
   • No stage reads a file the build didn't declare as an input.

5. Make a failed write an error. Today write_binary_libs.tcl prints Could not write … and continues, and every later stage quietly reads the text library, losing the speedup with nothing to show for it. A missing entry, or one written by another OpenROAD version, is expected and is converted again. A write that fails is a bug and should stop the run.

[eder-matheus]

@maliberty The failures here are the same from master. Final results don't change with the new .blib files

[oharboe]

@eder-matheus One more round of nits, consistency and cleanup? #4720 (comment)

[eder-matheus]

@eder-matheus One more round of nits, consistency and cleanup? #4720 (comment)

I will not apply most of them:

1. It is basically useless. It suggests to fallback to my previous implementation, where I was caching per design. Declined
2. I will keep the dirhash as a extra safety, especially considering a shared cache for the PDK, instead of per design.
3. From AI: Decline. The bug is already on master: memories_extract.log, in LOG_DIR for AUTO_MEMORIES designs, also sorts after the numbered logs and gets reported as the last log. Moving our log into objects/ would also hide it from where users and CI look for logs. The genReport.py fix can be its own PR.
4. From AI: Decline again; it doesn't change behavior. A .blib can't be converted into another: write_liberty_binary refuses binary input, and no flow sends binary libraries to make binary-libs. The second argument doesn't hold either: stages would still look up the cache for the text libraries bazel-orfs passes. In the bazel-orfs sandbox, every lookup finds nothing.
5. Implemented, will be in the PR soon.

It would be nice to have a human triage on these comments. Clearly wrong suggestions such as the first one from your last comment could be avoided.

[oharboe]

@eder-matheus Mea culpa: I do curate the feedback Claude writes, but I was unsure about some of the last round and posted it anyway to get your input, and it came out garbled. Points 2, 3 and 4 are withdrawn.

Two things remain:

1. Cache in $(OBJECTS_DIR)/binary_libs, not $(WORK_HOME)/objects/binary_libs. A cache per design has no race between designs; one cache shared by every design in WORK_HOME does. Nobody will see it locally, but in CI, where many designs run side by side, Murphy's law applies: if it can happen, it will.
2. The stamp misses the corner libraries. It depends on $(LIB_FILES), but write_binary_libs.tcl converts every corner's <CORNER>_LIB_FILES, so a changed corner library is never converted again:

$(LOG_DIR)/write_binary_libs.log: $(LIB_FILES) \
    $(foreach corner,$(shell echo $(CORNERS) | tr a-z A-Z),$($(corner)_LIB_FILES))

[eder-matheus]

1. Cache in $(OBJECTS_DIR)/binary_libs, not $(WORK_HOME)/objects/binary_libs. A cache per design has no race between designs; one cache shared by every design in WORK_HOME does. Nobody will see it locally, but in CI, where many designs run side by side, Murphy's law applies: if it can happen, it will.

The race doesn't exist when every design in a WORK_HOME uses the same OpenROAD, which is always the case in CI:

• A conversion only writes an entry that is missing or unreadable by the running OpenROAD. A usable entry is never rewritten, so an entry a stage has checked can't change before its read_liberty.
• When two designs convert the same missing library at once, each writes its own temporary file and renames it into place. Both produce identical bytes, and a reader sees either no entry, falling back to text, or a complete one.

2. The stamp misses the corner libraries. It depends on $(LIB_FILES), but write_binary_libs.tcl converts every corner's <CORNER>_LIB_FILES, so a changed corner library is never converted again:

I'm implementing this one.