Discussion:
Memoization profiling
(too old to reply)
Ludovic Courtès
2017-12-12 17:14:20 UTC
Permalink
Hello Guix,

Memoization plays a critical role in the performance of Guix. I’ve
finally added support to profile memoization, which should allow us to
better understand how it performs and to use it more wisely:

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=252c4083779a488c86e74362b4f3bb4bf927cc67
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=6c80641d54a95f2da95e480a4a746761d25161e9

Here’s an example of what it gives (you need to “make clean-go &&
make”):

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=memoization ./pre-inst-env guix build libreoffice -d --no-grafts
/gnu/store/x2dk4zwkfvlypmk7havzhk71bgrlbfk3-libreoffice-5.3.7.2.drv
Memoization: 113 tables, 34 non-empty
guix/utils.scm:744:2: 6274 entries, 7645 lookups, 18% hits
guix/derivations.scm:582:2: 1942 entries, 5230 lookups, 63% hits
guix/derivations.scm:636:2: 1779 entries, 5066 lookups, 65% hits
guix/derivations.scm:629:2: 889 entries, 24116 lookups, 96% hits
guix/packages.scm:776:4: 710 entries, 770 lookups, 8% hits
gnu/packages/bootstrap.scm:136:2: 307 entries, 7634 lookups, 96% hits
guix/build-system/asdf.scm:164:4: 20 entries, 20 lookups, 0% hits
guix/build-system/asdf.scm:164:4: 11 entries, 38 lookups, 71% hits
guix/packages.scm:776:4: 10 entries, 10 lookups, 0% hits
guix/derivations.scm:1028:2: 8 entries, 5116 lookups, 100% hits
guix/packages.scm:776:4: 7 entries, 7 lookups, 0% hits
guix/build-system/gnu.scm:88:6: 7 entries, 26 lookups, 73% hits
guix/packages.scm:776:4: 4 entries, 4 lookups, 0% hits
guix/build-system/gnu.scm:88:6: 4 entries, 8 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 3 entries, 6 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 2 entries, 4 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 2 entries, 4 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 2 entries, 4 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 2 entries, 4 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
guix/packages.scm:776:4: 1 entries, 1 lookups, 0% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 1 entries, 1 lookups, 0% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 1 entries, 1 lookups, 0% hits
guix/build-system/gnu.scm:88:6: 1 entries, 4 lookups, 75% hits
guix/build-system/gnu.scm:88:6: 1 entries, 7 lookups, 86% hits
guix/build-system/gnu.scm:88:6: 1 entries, 1 lookups, 0% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
guix/build-system/gnu.scm:88:6: 1 entries, 1 lookups, 0% hits
guix/build-system/gnu.scm:88:6: 1 entries, 2 lookups, 50% hits
--8<---------------cut here---------------end--------------->8---

Here memoization stats are dumped when exiting, but you can also force
them to be displayed after each GC:

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=memoization GUIX_PROFILING_EVENTS=after-gc \
./pre-inst-env guix build libreoffice -d --no-grafts
--8<---------------cut here---------------end--------------->8---

The new (guix profiling) module provides basic support for this.

Ludo’.

Loading...