a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

Hi,

I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.

I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
There's an issue that I can't seem to get rid of, I'll try my best to describe it now:

I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).

Repository 1: /home/user/src/infotropique/ports
Repository 2: /home/user/src/infotropique/pkgs

The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.

I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
the reository. Both repos worked this way.

After the transition to the new layout, ports works.
pkgs however is throwing errors at me for days. I've tried moving all but one module
out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt

I can not share the actual repo at this moment on this mailinglist, but maybe the
repl message is something that someone could give me a hint, even if it's just a
tiny bit of a "read this manpage" or something like that would help. I've tried
debugging this for days and any help's welcome.

Thanks,
N.

[Chris Marusich]

Post by Nils Gillmann
Hi,
I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.
I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).
Repository 1: /home/user/src/infotropique/ports
Repository 2: /home/user/src/infotropique/pkgs
The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.
I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
the reository. Both repos worked this way.
After the transition to the new layout, ports works.
pkgs however is throwing errors at me for days. I've tried moving all but one module
out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt
I can not share the actual repo at this moment on this mailinglist, but maybe the
repl message is something that someone could give me a hint, even if it's just a
tiny bit of a "read this manpage" or something like that would help. I've tried
debugging this for days and any help's welcome.

Could this error be coming from the scheme-files procedure in (guix
discovery)? It looks like that's the only place where fold-right is
invoked in that module, and it seems to match up with the stack trace,
unless I'm misreading it (which is possible). You might find it useful
to insert some pk statements into that code to try to figure out what is
being evaluated as "unspecified". My guess, which could be wrong, is
that perhaps scandir* is returning an unspecified value unintentionally
at some point? Someone with more experience debugging Guile code can
probably provide better advice.

[Nils Gillmann]

Post by Chris Marusich

Post by Nils Gillmann
Hi,
I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.
I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).
Repository 1: /home/user/src/infotropique/ports
Repository 2: /home/user/src/infotropique/pkgs
The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.
I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
the reository. Both repos worked this way.
After the transition to the new layout, ports works.
pkgs however is throwing errors at me for days. I've tried moving all but one module
out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt
I can not share the actual repo at this moment on this mailinglist, but maybe the
repl message is something that someone could give me a hint, even if it's just a
tiny bit of a "read this manpage" or something like that would help. I've tried
debugging this for days and any help's welcome.

Could this error be coming from the scheme-files procedure in (guix
discovery)? It looks like that's the only place where fold-right is
invoked in that module, and it seems to match up with the stack trace,
unless I'm misreading it (which is possible). You might find it useful
to insert some pk statements into that code to try to figure out what is
being evaluated as "unspecified". My guess, which could be wrong, is
that perhaps scandir* is returning an unspecified value unintentionally
at some point? Someone with more experience debugging Guile code can
probably provide better advice.
--
Chris

What I found is that: (pkgs sys-kernel) providing common logics for
(pkgs sys-kernel linux linux) [and planned, (pkgs sys-kernel linux-libre)]
and (ports sysutils firmware some-firmware) as well as (pkgs sys-kernel linux-headers)
created a problem. This, moving the definitions from the old linux module I had out to
a common module, was one of the few breaking changes.
I wasn't able to resolve this as amirouche told me on irc. I want to remove some last
bits of these imports tonight. I was able to update my system with the modules in another
place and imports corrected.
What's weird to me is that even after cleaning the ccache of guile in the respective
locations a package that isn't involved in the new change, like for example numactl,
as the only package definition in this place throws errors. In another location it
works. I didn't come up with the name "pkgs" for the root of the repository yesterday,
I have used this for a couple of months in a working state.

Anyways, in which section of the guile Manual would I find "pk"? I've done surprisingly
little debugging with Guile itself so far.

[Chris Marusich]

Post by Nils Gillmann
Anyways, in which section of the guile Manual would I find "pk"? I've
done surprisingly little debugging with Guile itself so far.

Unfortunately, pk isn't (yet?) documented. It apparently stands for
"peek stuff". It prints and returns the value passed to it. It's a
useful little shortcut for printing values. Here are some simple
examples of how to use it:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (pk "hello")

;;; ("hello")
$1 = "hello"
scheme@(guile-user)> (pk (list 1 2 3))

;;; ((1 2 3))
$2 = (1 2 3)
scheme@(guile-user)> (define (1+ n) (+ 1 (pk n)))
scheme@(guile-user)> (1+ 2)

;;; (2)
$3 = 3
scheme@(guile-user)> pk
$4 = #<procedure peek stuff>
--8<---------------cut here---------------end--------------->8---

Note that pk does not print out all values when a procedure returns
multiple values, but it is still quite handy:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (pk (values 1 2))

;;; (1)
$5 = 1
--8<---------------cut here---------------end--------------->8---

I hope that helps!

By the way, if you feel like taking a trip down the rabbit hole, here is
the answer to the somewhat unrelated question of "why did pk only print
the first value?"

https://lists.gnu.org/archive/html/guile-user/2017-06/msg00043.html

[Nils Gillmann]

Post by Chris Marusich

Post by Nils Gillmann
Anyways, in which section of the guile Manual would I find "pk"? I've
done surprisingly little debugging with Guile itself so far.

Unfortunately, pk isn't (yet?) documented. It apparently stands for
"peek stuff". It prints and returns the value passed to it. It's a
useful little shortcut for printing values.

Okay, thanks.

Post by Chris Marusich
Here are some simple
--8<---------------cut here---------------start------------->8---
;;; ("hello")
$1 = "hello"
;;; ((1 2 3))
$2 = (1 2 3)
;;; (2)
$3 = 3
$4 = #<procedure peek stuff>
--8<---------------cut here---------------end--------------->8---
Note that pk does not print out all values when a procedure returns
--8<---------------cut here---------------start------------->8---
;;; (1)
$5 = 1
--8<---------------cut here---------------end--------------->8---
I hope that helps!

Seems like in the case of using (system) it doesn't do anything, I'll
try guile interaction with Guix tonight.

Post by Chris Marusich
By the way, if you feel like taking a trip down the rabbit hole, here is
the answer to the somewhat unrelated question of "why did pk only print
the first value?"
https://lists.gnu.org/archive/html/guile-user/2017-06/msg00043.html
--
Chris

[Ricardo Wurmus]

If you have a file “core/lang-perl/perl.scm” and the module is called
“(core lang-perl perl)” then you should not have “core” itself on
GUIX_PACKAGE_PATH but its parent directory.

In my example (pkgs lang-perl perl) itself had the parent directory "pkgs".
I admit, code examples would've made it more clear..
Since we can point to github.com regularly, I think my private server would
not be considered problematic?

It is better to provide a minimal example that demonstrates the
problem. This is better than providing a full repository and ask other
people to figure out what is relevant and what is not.

[Nils Gillmann]

* except for source code, the modules itself in Guile aren't very
* detailed documented. The documentation is good, but... ... it's
* nowhere mentioned that you can have a module (foo bar baz) and
* possibly also (foo bar baz kim) but (foo baz bar bar) will lead to
* the error I described.
1. Is there a module name maximum length? -> So far I have encountered
very short module names in the wild, 3 the longest.
2. Is this really a module length problem? -> I have a functional set
of packages elsewhere, my non-core packages, named like (ports editors
foo foo) and so forth and they cause no problems. It's just weird to
me that the modules in ports and elsewhere work but in pkgs I can't
dissect the exact problem so far. I'm okay with any namespace, so
I'll simply remove the last element of the module names.
I'd like to improve documentations or other relevant places if
what I experienced is some kind of corner case in module naming,
be it in Guix or Guile... or just to note what to avoid with
regards to modules.

Actually the problem is not really module names I just found out.

I took my perl module which is in "pkgs/lang-perl/perl.scm", which contains our basic perl packages,
and moved ( + renamed ) them into (core lang-perl perl) in a new directory.
If I run GUIX_PACKAGE_PATH="/home/user/src/core" package -s mc I get results for query "mc".
When I run GUIX_PACKAGE_PATH="/home/user/src/core" package --show=perl I get the same message

Sorry, I was wrong (--show=perl works as expected).
However having the same in the old structure with just this module (and the old
name) does throw the old error. ccach of Guile is cleaned.

as initially pointed out in the opening post.

But I'm still curious about any possible guile module naming related limits
and hints from people working longer with Guile.

[Nils Gillmann]

Post by Chris Marusich

Post by Nils Gillmann
Hi,
I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.
I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).
Repository 1: /home/user/src/infotropique/ports
Repository 2: /home/user/src/infotropique/pkgs
The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.
I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
the reository. Both repos worked this way.
After the transition to the new layout, ports works.
pkgs however is throwing errors at me for days. I've tried moving all but one module
out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt
I can not share the actual repo at this moment on this mailinglist, but maybe the
repl message is something that someone could give me a hint, even if it's just a
tiny bit of a "read this manpage" or something like that would help. I've tried
debugging this for days and any help's welcome.

Could this error be coming from the scheme-files procedure in (guix
discovery)? It looks like that's the only place where fold-right is
invoked in that module, and it seems to match up with the stack trace,

Oh, sorry, I skipped this reply. I'll read that module and see if it helps.

Post by Chris Marusich
unless I'm misreading it (which is possible). You might find it useful
to insert some pk statements into that code to try to figure out what is
being evaluated as "unspecified". My guess, which could be wrong, is
that perhaps scandir* is returning an unspecified value unintentionally
at some point? Someone with more experience debugging Guile code can
probably provide better advice.
--
Chris