Using a shared Guix store (was RE: [Bio-packaging] testing out guix)

[Cook, Malcolm]

Ricardo, Pj, et al,


... Hi - I am resending this message as I omitted Guix-devel <guix-***@gnu.org> at first and would like to cast a broader net....


I am interested in understanding details behind Ricardo's observation: "Guix is not designed to be run in a centralised manner. A Guix daemon is supposed to run on each system as root and it listens to RPCs from local users only. In an environment with multiple clusters and multiple workstations this approach requires considerable effort to make it work correctly and securely. Instead we opted to run the Guix daemon on a single dedicated server, writing profile data and store items onto an NFS share. . The cluster nodes and workstations mount this share read-only." (http://elephly.net/posts/2015-04-17-gnu-guix.html)

Can anyone elaborate a little on what are the obstacles to having `/gnu` mounted read-write network wide?

Can it be partially characterized as:

Multi-process contention over un-coordinated access to the store (especially write access necessitated by supporting the `build` action)

If so, might this be mitigated using a variant off of "Using the Offload Facility" (http://www.gnu.org/software/guix/manual/guix.html#Daemon-Offload-Setup) in which builds would still be offloaded (and thus subject to coordination), with the elimination of the need for " Missing prerequisites for the build are copied over SSH to the target machine, which then proceeds with the build; upon success the output(s) of the build are copied back to the initial machine" since they would be done through the shared file system?

Do I understand correctly that in your setup, Ricardo, that absolutely no `guix` commands are executed on any host other than the "single dedicated server". What about `guix environment p1 p2 p3` when p1 p2 p3 are already available in /gnu. If I understand correctly, in such a case, nothing need be written to /gnu... and so should not present any challenge to running guix off a shared mount. Or am I missing an aspect of what is going on?

Lacking the ability to, from any host, dynamically alter the runtime environment to include _already installed_ scientific applications seems like the most important aspect of guix that would be untolerable to my users were I to user guix. I do hope I am mis-understanding the trade-off you made at your site.

I think the second-worst trade-off this compromise takes is that a user cannot even alter their own profile unless connected to the master guix host. For instance, a user switching her default emacs between two already built & installed versions only requires editing the profiles/per-user symlink farm; couldn't such commands be put behind a per-user semaphore, allowing guix/profiles/per-user to be made available +rw on a shared network mount?

I really appreciate everyone's assistance in helping me understand these trade-offs, how guix works, and if there is any new or different thinking about strategies for deploying guix.

Thanks,

Malcolm Cook



Hi Malcolm,

Pleased to e-meet you. It was your
http://elephly.net/posts/2015-04-17-gnu-guix.html that got on this
path. I'm so glad I found it.

Oh, great to read that it reached someone who might benefit from it :)

Great. I have a trove of recipes (in either bash and a
brew-derivative) which I intend to move into guix over time, unless
I'm beaten to any of them (I'm sure I already have been for some):>

[...]

I just went through the list and found a few that looked familiar.
Below are some comments.

gatk
jimKentUtils
igv
igvtools
meme
picard (requires more Java toolchain stuff to build from source)
soapdenovo2
trimmomatic
macs14
tophat
viennaRNA
bamtools
bedtools
blast+ (WIP, it's really big)
bowtie2
cufflinks
fastqc
fastx_toolkit
graphviz
ncbi_sra_toolkit
ngsutils
rsem
star
tabix
bam2fastq
bcftools

[...]

Gonna be a party!

Indeed! I would be very happy if you decide for Guix in the end. There isn't a lot of bioinfo stuff there yet, but that can be fixed by having more people contribute recipes.

Cheers,
Ricardo

[Ricardo Wurmus]

Post by Cook, Malcolm
Can anyone elaborate a little on what are the obstacles to having
`/gnu` mounted read-write network wide?

Yes, the primary problem is that the daemon assumes that it is the only
thing writing to the store and the localstatedir. Any modification of
profiles and the store goes through the daemon.

Post by Cook, Malcolm
If so, might this be mitigated using a variant off of "Using the
Offload Facility"
(http://www.gnu.org/software/guix/manual/guix.html#Daemon-Offload-Setup)
in which builds would still be offloaded (and thus subject to
coordination), with the elimination of the need for " Missing
prerequisites for the build are copied over SSH to the target machine,
which then proceeds with the build; upon success the output(s) of the
build are copied back to the initial machine" since they would be done
through the shared file system?

Something like that has been suggested before: if the daemon were to
accept authenticated connections from the outside rather than to just
listen on a local socket we could have remote guix clients connecting to
the central daemon.

Post by Cook, Malcolm
Do I understand correctly that in your setup, Ricardo, that absolutely
no `guix` commands are executed on any host other than the "single
dedicated server". What about `guix environment p1 p2 p3` when p1 p2
p3 are already available in /gnu. If I understand correctly, in such
a case, nothing need be written to /gnu... and so should not present
any challenge to running guix off a shared mount. Or am I missing an
aspect of what is going on?

This is correct. Our needs are rather simple, so people don’t use any
of the guix commands on their local machines, nor on the cluster nodes.
This is not as inconvenient as this may seem when profiles don’t change
often, but it is of course a serious limitation. My first goal was to
replace traditional software *packaging* (because that’s what affected
my work as a sysadmin); the next goal is to allow users to conveniently
perform software *management*. At the moment this requires users to log
on to the guix management host (or to ask a sysadmin to perform the
changes for them).

Currently our users are okay with that, probably to a large part because
they are not yet aware of all the features of Guix. They are only used
to management by sysadmins or manual compilation, so they are not
inconvenienced.

Ultimately, the correct fix is to allow remote guix clients to
communicate with a central guix daemon. The daemon does not even need
to be aware of remote connections if guix clients can transparently
connect via SSH and send RPCs to the socket. This is not yet
implemented.

Even then, /gnu would be mounted read-only on all but the management
host, because there would still only be one daemon writing to the store
and the localstatedir – even if it would take requests from multiple
remote guix clients.

~~ Ricardo

[Ludovic Courtès]

Post by Ricardo Wurmus
Something like that has been suggested before: if the daemon were to
accept authenticated connections from the outside rather than to just
listen on a local socket we could have remote guix clients connecting to
the central daemon.

[...]

Post by Ricardo Wurmus
Ultimately, the correct fix is to allow remote guix clients to
communicate with a central guix daemon. The daemon does not even need
to be aware of remote connections if guix clients can transparently
connect via SSH and send RPCs to the socket. This is not yet
implemented.

This could be either SSH or a plain direct connection, which is
acceptable on a LAN.

A simple way to test the latter is with ‘socat’, something along the
lines of what I described at:

https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00680.html

It would be great if you could check whether something like this works
for you.

Then we could also have built-in support for that in the daemon and
clients, so that it’s more convenient.

Thanks,
Ludo’.

[Ricardo Wurmus]

Post by Ludovic Courtès
A simple way to test the latter is with ‘socat’, something along the
https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00680.html
It would be great if you could check whether something like this works
for you.
Then we could also have built-in support for that in the daemon and
clients, so that it’s more convenient.

This is still on my list. Maybe I can get around to testing this today
or tomorrow.

~~ Ricardo

[Ludovic Courtès]

Post by Ricardo Wurmus
Even then, /gnu would be mounted read-only on all but the management
host, because there would still only be one daemon writing to the store
and the localstatedir – even if it would take requests from multiple
remote guix clients.

I think $localstatedir (/var/guix by default), or a subset thereof
(namely /var/guix/profiles/per-user) should be shared read-write over
NFS to allow users to create the profiles/per-user/$USER symlinks from
other machines.

Sysadmins might find it more convenient to have $localstatedir under
/gnu, for instance --localstatedir=/gnu/var/guix, so that simply
exporting /gnu over NFS covers everything.

(Of course you still need the ability to talk to the central daemon from
any host, as we discussed.)

Thanks,
Ludo’.

[Cook, Malcolm]

Hi Ludovic and Ricardo,

Post by Ricardo Wurmus
Even then, /gnu would be mounted read-only on all but the management
host, because there would still only be one daemon writing to the
store and the localstatedir – even if it would take requests from
multiple remote guix clients.

I think $localstatedir (/var/guix by default), or a subset thereof (namely
/var/guix/profiles/per-user) should be shared read-write over NFS to allow
users to create the profiles/per-user/$USER symlinks from other machines.

Loduvic, I'm not sure I understand well enough to agree....

Ricardo, I appreciate your fixing my reasoning below....

Certainly per-user profiles must be network readable , since they wind up on PATH. Perhaps within /gnu/guix/profiles/per-user?

However, if `guix` command is enabled (using ssh/pipe/??) to run in networked environment with build and other work continuing to be coordinated by guix-daemon, then updates to per-user profiles similarly would need to be coordinated. For example, if a user wants to `guix package --install somePkg-someVersion` then that package must be protected against any "simultaneous" garbage collection. Running the --install through the network daemon is the guix way of protecting against such a race condition.

On this topic, examining the documentation a bit further makes me wonder whether support for `guix build --root MYROOT` will be impossible when (a) MYROOT is not accessible to the host running guix-daemon. Will this be an issue?

Cheers,

Malcolm

[Ricardo Wurmus]

Post by Cook, Malcolm
Certainly per-user profiles must be network readable , since they wind
up on PATH. Perhaps within /gnu/guix/profiles/per-user?

I have configured guix such that the $localstatedir is /gnu/var, so I
only have to export /gnu.

Post by Cook, Malcolm
However, if `guix` command is enabled (using ssh/pipe/??) to run in
networked environment with build and other work continuing to be
coordinated by guix-daemon, then updates to per-user profiles
similarly would need to be coordinated. For example, if a user wants
to `guix package --install somePkg-someVersion` then that package must
be protected against any "simultaneous" garbage collection. Running
the --install through the network daemon is the guix way of protecting
against such a race condition.

Correct. With a shared store there must be a shared daemon. An
alternative setup would be to have completely separate stores and reduce
build work by using ‘guix publish’.

Unfortunately, I have not yet been able to test a setup where the local
‘guix’ command talks via SSH/socat to a central guix-daemon. (I’ve been
trying to improve the performance in our situation where the store is
located on a central file server and the daemon operates on the NFS
share. This is rather slow here and could only partially be improved by
disabling deduplication.)

~~ Ricardo

[Cook, Malcolm]

Ricardo,

Thanks for your complete and easy-to-understand response to my inquiries.

Post by Cook, Malcolm
Can anyone elaborate a little on what are the obstacles to having
`/gnu` mounted read-write network wide?

Yes, the primary problem is that the daemon assumes that it is the only thing
writing to the store and the localstatedir. Any modification of profiles and
the store goes through the daemon.

Post by Cook, Malcolm
If so, might this be mitigated using a variant off of "Using the
Offload Facility"
(http://www.gnu.org/software/guix/manual/guix.html#Daemon-Offload-

Setu

Post by Cook, Malcolm
p) in which builds would still be offloaded (and thus subject to
coordination), with the elimination of the need for " Missing
prerequisites for the build are copied over SSH to the target machine,
which then proceeds with the build; upon success the output(s) of the
build are copied back to the initial machine" since they would be done
through the shared file system?

Something like that has been suggested before: if the daemon were to
accept authenticated connections from the outside rather than to just listen
on a local socket we could have remote guix clients connecting to the central
daemon.

Hmm. So, for now, I could just teach users to, i.e., `ssh ${USER}@${GUIX_HOST} /path/to/guix package -r lua -i guile guile-cairo` and contrive for GUIX_HOST to be defined for them.

Or, wrap it up even further, and contrive for everyone to have the following in their env:

alias guixpkg='ssh ${USER}@${GUIX_HOST} /path/to/guix package'

??

Post by Cook, Malcolm
Do I understand correctly that in your setup, Ricardo, that absolutely
no `guix` commands are executed on any host other than the "single
dedicated server". What about `guix environment p1 p2 p3` when p1 p2
p3 are already available in /gnu. If I understand correctly, in such
a case, nothing need be written to /gnu... and so should not present
any challenge to running guix off a shared mount. Or am I missing an
aspect of what is going on?

This is correct. Our needs are rather simple, so people don’t use any of the
guix commands on their local machines, nor on the cluster nodes.
This is not as inconvenient as this may seem when profiles don’t change
often, but it is of course a serious limitation. My first goal was to replace
traditional software *packaging* (because that’s what affected my work as a
sysadmin); the next goal is to allow users to conveniently perform software
*management*. At the moment this requires users to log on to the guix
management host (or to ask a sysadmin to perform the changes for them).

I totally get your priorities. Thanks for the reveal! Are these also the priorities of the GnuSD project, do you know?

Currently our users are okay with that, probably to a large part because they
are not yet aware of all the features of Guix. They are only used to
management by sysadmins or manual compilation, so they are not
inconvenienced.
Ultimately, the correct fix is to allow remote guix clients to communicate with
a central guix daemon. The daemon does not even need to be aware of
remote connections if guix clients can transparently connect via SSH and send
RPCs to the socket. This is not yet implemented.

Sounds great. On the roadmap?

Even then, /gnu would be mounted read-only on all but the management
host, because there would still only be one daemon writing to the store and
the localstatedir – even if it would take requests from multiple remote guix
clients.

Agreed.

Fingers crossed, and thanks again,

~Malcolm

~~

[Ludovic Courtès]

Hi,

Post by Cook, Malcolm

Post by Cook, Malcolm
Can anyone elaborate a little on what are the obstacles to having
`/gnu` mounted read-write network wide?

Yes, the primary problem is that the daemon assumes that it is the only thing
writing to the store and the localstatedir. Any modification of profiles and
the store goes through the daemon.

Post by Cook, Malcolm
If so, might this be mitigated using a variant off of "Using the
Offload Facility"
(http://www.gnu.org/software/guix/manual/guix.html#Daemon-Offload-

Setu

Post by Cook, Malcolm
p) in which builds would still be offloaded (and thus subject to
coordination), with the elimination of the need for " Missing
prerequisites for the build are copied over SSH to the target machine,
which then proceeds with the build; upon success the output(s) of the
build are copied back to the initial machine" since they would be done
through the shared file system?

Something like that has been suggested before: if the daemon were to
accept authenticated connections from the outside rather than to just listen
on a local socket we could have remote guix clients connecting to the central
daemon.

??

That’s a possibility.

Another one, which I think Ricardo has been investigating lately, would
be to have users manage their profiles via the Web user interface,
guix-web: <https://git.dthompson.us/guix-web.git>. (The video
<http://audio-video.gnu.org/video/misc/2015-01__GNU_Guix__The_Emacs_of_Distros.webm>
has a demo starting at around 23mn.)

Post by Cook, Malcolm

Currently our users are okay with that, probably to a large part because they
are not yet aware of all the features of Guix. They are only used to
management by sysadmins or manual compilation, so they are not
inconvenienced.
Ultimately, the correct fix is to allow remote guix clients to communicate with
a central guix daemon. The daemon does not even need to be aware of
remote connections if guix clients can transparently connect via SSH and send
RPCs to the socket. This is not yet implemented.

Sounds great. On the roadmap?

Definitely. There are details to be sorted out (SSH vs. plain old
socket), but we should discuss it and “get it done.”

Thanks for your feedback,
Ludo’.

[Cook, Malcolm]

Hi!

Post by Cook, Malcolm

Post by Cook, Malcolm

Post by Ricardo Wurmus

Post by Cook, Malcolm
Can anyone elaborate a little on what are the obstacles to having
`/gnu` mounted read-write network wide?

Yes, the primary problem is that the daemon assumes that it is the
only thing writing to the store and the localstatedir. Any
modification of profiles and the store goes through the daemon.

Post by Cook, Malcolm
If so, might this be mitigated using a variant off of "Using the
Offload Facility"
(http://www.gnu.org/software/guix/manual/guix.html#Daemon-

Offload-

Post by Cook, Malcolm

Post by Ricardo Wurmus
Setu

Post by Cook, Malcolm
p) in which builds would still be offloaded (and thus subject to
coordination), with the elimination of the need for " Missing
prerequisites for the build are copied over SSH to the target
machine, which then proceeds with the build; upon success the
output(s) of the build are copied back to the initial machine"
since they would be done through the shared file system?

Something like that has been suggested before: if the daemon were to
accept authenticated connections from the outside rather than to just
listen on a local socket we could have remote guix clients connecting
to the central daemon.

Hmm. So, for now, I could just teach users to, i.e., `ssh

and contrive for GUIX_HOST to be defined for them.

Post by Cook, Malcolm
Or, wrap it up even further, and contrive for everyone to have the
??

That’s a possibility.

Thanks for the confirmation. I'm thinking of trying this for the "short term" - unless Ricardo's efforts bear fruit sooner.

Post by Cook, Malcolm
Another one, which I think Ricardo has been investigating lately, would be to
have users manage their profiles via the Web user interface,
guix-web: <https://git.dthompson.us/guix-web.git>. (The video
<http://audio-video.gnu.org/video/misc/2015-
01__GNU_Guix__The_Emacs_of_Distros.webm>
has a demo starting at around 23mn.)

Hmm, it is not clear to me how this would play out in a multi-user environment. Having a web server that could alter per-user profiles sounds like a recipe for more confusion, rather than a solution.

In any case, in my opinion, it would be a mistake to have depend on another tool (such as guix-web, or something similar) to implement functionality that could not be gained at using command-line guix. Don't you agree?

Post by Cook, Malcolm

Post by Cook, Malcolm

Post by Ricardo Wurmus
Currently our users are okay with that, probably to a large part
because they are not yet aware of all the features of Guix. They are
only used to management by sysadmins or manual compilation, so they
are not inconvenienced.
Ultimately, the correct fix is to allow remote guix clients to
communicate with a central guix daemon. The daemon does not even
need to be aware of remote connections if guix clients can
transparently connect via SSH and send RPCs to the socket. This is not yet

implemented.

Post by Cook, Malcolm
Sounds great. On the roadmap?

Definitely. There are details to be sorted out (SSH vs. plain old socket), but
we should discuss it and “get it done.”

Are there any signposts on this road-map emerging? Is there any way I can help, such as review proposals for how the CLI would work, I'd be happy to chip in. Not ready to start slinging guile though. Maybe later (the old lisper in me revels at the thought).

Post by Cook, Malcolm
Thanks for your feedback,
Ludo’.

Thanks for GUIX!

~Malcolm

[Ricardo Wurmus]

Post by Cook, Malcolm

Post by Cook, Malcolm
Another one, which I think Ricardo has been investigating lately, would be to
have users manage their profiles via the Web user interface,
guix-web: <https://git.dthompson.us/guix-web.git>. (The video
<http://audio-video.gnu.org/video/misc/2015-
01__GNU_Guix__The_Emacs_of_Distros.webm>
has a demo starting at around 23mn.)

Hmm, it is not clear to me how this would play out in a multi-user
environment. Having a web server that could alter per-user profiles
sounds like a recipe for more confusion, rather than a solution.
In any case, in my opinion, it would be a mistake to have depend on
another tool (such as guix-web, or something similar) to implement
functionality that could not be gained at using command-line guix.
Don't you agree?

Some find a web interface easier to use than the command line. For
those users I suspect the convenience of using a web interface outweighs
the threat of additional confusion. Ultimately, I want both tools to be
available because users differ.

~~ Ricardo

[Ludovic Courtès]

Post by Cook, Malcolm

Post by Cook, Malcolm
Another one, which I think Ricardo has been investigating lately, would be to
have users manage their profiles via the Web user interface,
guix-web: <https://git.dthompson.us/guix-web.git>. (The video
<http://audio-video.gnu.org/video/misc/2015-
01__GNU_Guix__The_Emacs_of_Distros.webm>
has a demo starting at around 23mn.)

Hmm, it is not clear to me how this would play out in a multi-user environment. Having a web server that could alter per-user profiles sounds like a recipe for more confusion, rather than a solution.
In any case, in my opinion, it would be a mistake to have depend on another tool (such as guix-web, or something similar) to implement functionality that could not be gained at using command-line guix. Don't you agree?

I’m just mentioning it as an additional option, for those who like it.

Post by Cook, Malcolm

Post by Cook, Malcolm

Post by Cook, Malcolm

Post by Ricardo Wurmus
Ultimately, the correct fix is to allow remote guix clients to
communicate with a central guix daemon. The daemon does not even
need to be aware of remote connections if guix clients can
transparently connect via SSH and send RPCs to the socket. This is not yet

implemented.

Post by Cook, Malcolm
Sounds great. On the roadmap?

Definitely. There are details to be sorted out (SSH vs. plain old socket), but
we should discuss it and “get it done.”

Are there any signposts on this road-map emerging? Is there any way I can help, such as review proposals for how the CLI would work, I'd be happy to chip in. Not ready to start slinging guile though. Maybe later (the old lisper in me revels at the thought).

Yes, one way to help would be by trying a simple setup using Guix as it
currently exists, coupled with socat:

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20381

If you could test this and provide feedback about the other options
discussed there, that would be great (please email
***@debbugs.gnu.org.)

So no ETA, but once we have feedback from you, Ricardo, and other
interested parties, it will be easier to make a choice and implement it.
:-)

Thanks,
Ludo’.

[Cook, Malcolm]

Ludovic, and Ricardo,

Post by Ludovic Courtès
Yes, one way to help would be by trying a simple setup using Guix as it
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20381
If you could test this and provide feedback about the other options discussed
there, that would be great (please email

Excellent. That definitely looks testable by me, once I'm done with my basic site deployment. I've been moving rather slowly but might be ready to contrast such approach with simply calling guix on guix-master over ssh. I'll plan on it. Maybe a week or two out.

Reading the thread, it sounds like Ricardo has today already had good success with socat. Yay!

In that thread, you suggested "What about installing Guix in /gnu/bin (say) and sharing it over NFS?". Is there a way to `./bootstrap ./configure ` this? Something like -- exec_prefix=/gnu --localstatedir=/gnu/var?

Also, you discussed need for setting NIX_STATE_DIR. This should not be needed if guix was already installed with configuration of --localstatedir=/guix/var (with /guix being nfs mounted), right?

Ricardo also wrote "For ‘guix package’ to work, /gnu/var/guix/profiles/per-user must be shared read-write (over NFS) with correct UID mapping." - I really don't understand this and tried to elicit feedback in http://lists.nongnu.org/archive/html/guix-devel/2015-07/msg00216.html - Can anyone give me clarification on this. I though the guix daemon is going to manage these profiles.

Honestly, much of my understanding of the above comes from reading the manual, archived IRC chats and mail threads, FOSDEM vids, and blog posts - which has been great. But, I've yet to succeed in my own site deploy, which will give me testbed to stretch my own understanding. Today, I had the following failure....

Ricardo, et. al., would you mind sharing some details on your network sharing configuration as regards users/groups and privileges, for I seem to have painted myself into a corner with the following

/gnu is an NFS mount, owned by network user 'guix', group-owned by guix-builder with mod of 1775

guix is being built from git configured with -- exec_prefix=/gnu --localstatedir=/gnu/var

`make check` results are

============================================================================
Testsuite summary for GNU Guix 0.8.3
============================================================================
# TOTAL: 42
# PASS: 32
# SKIP: 0
# XFAIL: 0
# FAIL: 10
# XPASS: 0
# ERROR: 0

With details below.... I can submit all this to bug-***@gnu.org, but I suspect (1) there is something fundamentally wrong with the way I am setting things up (2) your are reading this anyway ;)

Can you see any gotchas I haven't got yet? Can you see the error in my ways and suggest a way through?

FWIW, I try to `make install` anyway, and, I pause.... I cannot complete a make install without "changing hats". I find that it works to do the following

sudo make install-data # as root on localhost
sudo -u guix make install-exec ## since root on localhost is squashed on the NFS mount point,

But I don't know if this is complete or not (the Makefile is complex!).

Anyway, "changing hats" feels very wrong, and I can feel more trouble coming when I want to run the daemon....

Thanks so much everyone, and, keep on guixing,

~Malcolm


PS - in case it helps - below are test results and tarred logs are at http://research.stowers.org/mec/guixtests.tar

I have not yet taught myself how to read them with understanding.... - (have a great weekend!)

PASS: tests/base32.scm
PASS: tests/base64.scm
FAIL: tests/cpio.scm
PASS: tests/hash.scm
PASS: tests/pk-crypto.scm
PASS: tests/pki.scm
PASS: tests/sets.scm
PASS: tests/substitute.scm
FAIL: tests/builders.scm
FAIL: tests/derivations.scm
PASS: tests/ui.scm
PASS: tests/records.scm
PASS: tests/utils.scm
PASS: tests/build-utils.scm
FAIL: tests/packages.scm
PASS: tests/snix.scm
PASS: tests/hackage.scm
PASS: tests/elpa.scm
FAIL: tests/store.scm
PASS: tests/monads.scm
PASS: tests/gexp.scm
FAIL: tests/nar.scm
FAIL: tests/union.scm
PASS: tests/profiles.scm
FAIL: tests/syscalls.scm
PASS: tests/gremlin.scm
PASS: tests/lint.scm
PASS: tests/publish.scm
PASS: tests/size.scm
PASS: tests/guix-build.sh
PASS: tests/guix-download.sh
PASS: tests/guix-hash.sh
PASS: tests/guix-package.sh
FAIL: tests/guix-package-net.sh
PASS: tests/guix-system.sh
PASS: tests/guix-archive.sh
PASS: tests/guix-authenticate.sh
FAIL: tests/guix-environment.sh
PASS: tests/guix-lint.sh
PASS: tests/guix-register.sh
PASS: tests/guix-gc.sh
PASS: tests/guix-daemon.sh

Post by Ludovic Courtès
So no ETA, but once we have feedback from you, Ricardo, and other
interested parties, it will be easier to make a choice and implement it.

[Ricardo Wurmus]

Post by Cook, Malcolm
Also, you discussed need for setting NIX_STATE_DIR. This should not
be needed if guix was already installed with configuration of
--localstatedir=/guix/var (with /guix being nfs mounted), right?

NIX_STATE_DIR is used to override localstate at runtime. If you are
happy with the localstatedir (defined at configure time) you do not need
NIX_STATE_DIR. After my vacation I’ll take some time to think about how
to allow users to run “guix” on cluster nodes to manipulate their
profiles — and how to reliably set this all up. I’m planning on
documenting this, both in a specialised form for my blog and in a more
generic fashion for the Guix manual.

Post by Cook, Malcolm
Ricardo also wrote "For ‘guix package’ to work,
/gnu/var/guix/profiles/per-user must be shared read-write (over NFS)
with correct UID mapping." - I really don't understand this and tried
to elicit feedback in
http://lists.nongnu.org/archive/html/guix-devel/2015-07/msg00216.html
- Can anyone give me clarification on this. I though the guix daemon
is going to manage these profiles.

I don’t remember writing this or the context in which I wrote this
(could it be that Ludovic wrote this?) — looking at the code for
guix/scripts/package.scm it seems that the client performs some of the
symlink switching when switching generations (e.g. after installing a
new package). For that to work it would need write permission to the
user’s profile directory.

(I could be completely wrong here. That’s not how it’s set up here and
I haven’t yet tested a configuration like this.)

~~ Ricardo

[Cook, Malcolm]

Post by Cook, Malcolm
Also, you discussed need for setting NIX_STATE_DIR. This should not
be needed if guix was already installed with configuration of
--localstatedir=/guix/var (with /guix being nfs mounted), right?

NIX_STATE_DIR is used to override localstate at runtime. If you are happy
with the localstatedir (defined at configure time) you do not need
NIX_STATE_DIR. After my vacation I’ll take some time to think about how to
allow users to run “guix” on cluster nodes to manipulate their profiles — and
how to reliably set this all up. I’m planning on documenting this, both in a
specialised form for my blog and in a more generic fashion for the Guix
manual.

Great - thanks for clarifying - I am looking forward to reading about this further in both places.

I will eventually care about the cluster use case, but for now my first aim is to ensure that users at my institute can expect an identical experience in using guix regardless of which of our multiple shared-used computational servers they are logged into.

Post by Cook, Malcolm
Ricardo also wrote "For ‘guix package’ to work,
/gnu/var/guix/profiles/per-user must be shared read-write (over NFS)
with correct UID mapping." - I really don't understand this and tried
to elicit feedback in
http://lists.nongnu.org/archive/html/guix-devel/2015-07/msg00216.html
- Can anyone give me clarification on this. I though the guix daemon
is going to manage these profiles.

I don’t remember writing this or the context in which I wrote this (could it be
that Ludovic wrote this?) — looking at the code for guix/scripts/package.scm
it seems that the client performs some of the symlink switching when
switching generations (e.g. after installing a new package). For that to work it
would need write permission to the user’s profile directory.

Ricardo I am sorry, you are correct, I was quoting Ludovic in a thread where you replied https://lists.gnu.org/archive/html/bug-guix/2015-07/msg00042.html

(I could be completely wrong here. That’s not how it’s set up here and I
haven’t yet tested a configuration like this.)

I hope to chime in further with a big WOOHOO once I have this configuration working....

Thanks again for

[Pjotr Prins]

Great Malcolm,

One thing I was advocating today (compared to, for example, easybuild,
modules or homebrew) is that once you create a GNU Guix package it
will live on *all* future Linux systems (and perhaps beyond). It being
a GNU project I can only imagine it becoming ubiquous.

Pj.

Post by Cook, Malcolm

Post by Cook, Malcolm
Also, you discussed need for setting NIX_STATE_DIR. This should not
be needed if guix was already installed with configuration of
--localstatedir=/guix/var (with /guix being nfs mounted), right?

NIX_STATE_DIR is used to override localstate at runtime. If you are happy
with the localstatedir (defined at configure time) you do not need
NIX_STATE_DIR. After my vacation I’ll take some time to think about how to
allow users to run “guix” on cluster nodes to manipulate their profiles — and
how to reliably set this all up. I’m planning on documenting this, both in a
specialised form for my blog and in a more generic fashion for the Guix
manual.

Great - thanks for clarifying - I am looking forward to reading about this further in both places.
I will eventually care about the cluster use case, but for now my first aim is to ensure that users at my institute can expect an identical experience in using guix regardless of which of our multiple shared-used computational servers they are logged into.

Post by Cook, Malcolm
Ricardo also wrote "For ‘guix package’ to work,
/gnu/var/guix/profiles/per-user must be shared read-write (over NFS)
with correct UID mapping." - I really don't understand this and tried
to elicit feedback in
http://lists.nongnu.org/archive/html/guix-devel/2015-07/msg00216.html
- Can anyone give me clarification on this. I though the guix daemon
is going to manage these profiles.

I don’t remember writing this or the context in which I wrote this (could it be
that Ludovic wrote this?) — looking at the code for guix/scripts/package.scm
it seems that the client performs some of the symlink switching when
switching generations (e.g. after installing a new package). For that to work it
would need write permission to the user’s profile directory.

Ricardo I am sorry, you are correct, I was quoting Ludovic in a thread where you replied https://lists.gnu.org/archive/html/bug-guix/2015-07/msg00042.html

(I could be completely wrong here. That’s not how it’s set up here and I
haven’t yet tested a configuration like this.)

I hope to chime in further with a big WOOHOO once I have this configuration working....
Thanks again for your helps...
~Malcolm

~~ Ricardo

--

[Cook, Malcolm]

Post by Cook, Malcolm
I hope to chime in further with a big WOOHOO once I have this configuration working....

Not quite WOOHOO yet. Getting close.... Here is where I am:

I have built guix-0.8.2 on centOS7 passing all checks

However

./pre-inst-env guix package -i hello

Gives me

guix package: error: build failed: the build users group `guix-builder' has no members

But, but, I do have a group `guix-builder`

I think the problem may be that my guix-builder group was defined in NIS whereas its members are not in NIS.

I don't know the method that guix is using to detect group membership, but my configuration seems to not respect it.

I am going to try putting guix-builder1-10 in NIS also and see if that helps, but, while I wait for my sysadmin to help me, I thought I'd ask all y'all.


My setup now is like this:


/gnu is network shared mount rw everywhere
owned by NIS user guix
group owned by NIS group guix-builder

I configured the build with --prefix=/gnu

All tests pass.

The daemon is running as user `guix` on host ${GUIXHOST} (I welcome a better suggestion for this host. Maybe just GUIX is better? Or GUIXMASTER?)

Post by Cook, Malcolm
guix build hello

...
The following files will be downloaded:
/gnu/store/ydj867rdlz3p6lnnmbqn7jhvdf3i0dvi-hello-2.10
/gnu/store/hy2hi0zj5hrqkmkhpdxf04c9bcnlnsf9-glibc-2.21
...
603 operations

And confirmed:

/gnu/store/ydj867rdlz3p6lnnmbqn7jhvdf3i0dvi-hello-2.10/bin/hello
Hello, world

However trying to --install is not successful

/gnu/bin/guix package --install hello
accepted connection from pid 26597, uid 1232
error: while creating directory `/gnu/var/guix/profiles/per-user/mec': Permission denied
Please create the `/gnu/var/guix/profiles/per-user/mec' directory, with you as the owner.
Try "info '(guix) Invoking guix package'" for more information.

Which leads me to think the documentation is in error where it says "The per-user directory is created when guix-daemon is started".

Creating it by hand (owned by guix, with write access to all) :

mkdir /gnu/var/guix/profiles/per-user ## the doc says this should happen by the daemon but not! FIXME! BUG?
chmod a+rwx /gnu/var/guix/profiles/per-user

gets me further, and I did find the rest of the sentence to be correct: " and the user sub-directory is created by guix package."

But --installing only gets a little further:
/gnu/bin/guix package --install hello
accepted connection from pid 31645, uid 1232
The following package will be installed:
hello 2.10 /gnu/store/ydj867rdlz3p6lnnmbqn7jhvdf3i0dvi-hello-2.10

The following derivations will be built:
/gnu/store/6cs8g6kq96zdwl25929l5s5y6gxk7hxs-profile.drv
/gnu/store/b1f6jwficry84qgigvkldkfyhlc2xhg2-ca-certificate-bundle.drv
/gnu/store/n2ks4dzmrcg9ghhrp6c6wsp57nib5rqf-info-dir.drv
The following files will be downloaded:
/gnu/store/kbf46vds6f7lp723xn62kgdaxmwc1jvi-module-import-compiled
/gnu/store/41wzbfhmb6j5yha508y30kwycivhpx9b-module-import
/gnu/store/ki6mvjqbf3nc8lfavwqfsrxqzi7qhf50-module-import
/gnu/store/jhbkpizpakgk5xzzpyrh15vaprqcarb8-module-import-compiled
guix package: error: build failed: the build users group `guix-builder' has no members


Huh? Why
a) is it going to download and build derivations AGAIN !?!?
b) does it tell me "the build users group `guix-builder' has no members"

So, I am stymied, and welcome your kind assistance and observations.

I feel a YAHOO WAHOO comi

[Ludovic Courtès]

Post by Cook, Malcolm
However
./pre-inst-env guix package -i hello
Gives me
guix package: error: build failed: the build users group `guix-builder' has no members
But, but, I do have a group `guix-builder`
I think the problem may be that my guix-builder group was defined in NIS whereas its members are not in NIS.

What does ‘getenv group guix-builder’ return? It should show something
like:

guixbuild:x:30000:guixbuilder01,guixbuilder02,guixbuilder03,guixbuilder04,guixbuilder05,guixbuilder06,guixbuilder07,guixbuilder08,guixbuilder09,guixbuilder10

Each build account must have ‘guix-builder’ in its supplementary group list.

Post by Cook, Malcolm
The daemon is running as user `guix` on host ${GUIXHOST}

The daemon really needs to run as root, if possible:

http://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html

Post by Cook, Malcolm
However trying to --install is not successful
/gnu/bin/guix package --install hello
accepted connection from pid 26597, uid 1232
error: while creating directory `/gnu/var/guix/profiles/per-user/mec': Permission denied
Please create the `/gnu/var/guix/profiles/per-user/mec' directory, with you as the owner.
Try "info '(guix) Invoking guix package'" for more information.
Which leads me to think the documentation is in error where it says "The per-user directory is created when guix-daemon is started".

Well, /var/guix/profiles/per-user is created by the daemon, but .../mec
is created by ‘guix package’.

Post by Cook, Malcolm
mkdir /gnu/var/guix/profiles/per-user ## the doc says this should happen by the daemon but not! FIXME! BUG?
chmod a+rwx /gnu/var/guix/profiles/per-user

Perfect.

The daemon really does create /gnu/var/guix/profiles/per-user, making it
world-writable (see nix/libstore/local-store.cc:254.)

However, you mentioned guix-daemon was *not* running as root, in which
case it does not attempt to create this directory. That could be the
reason.

Thanks for your report,
Ludo’.

[Claes Wallin (韋嘉誠)]

Post by Cook, Malcolm
However
./pre-inst-env guix package -i hello
Gives me
guix package: error: build failed: the build users group

`guix-builder' has no members

Post by Cook, Malcolm
But, but, I do have a group `guix-builder`
I think the problem may be that my guix-builder group was defined in

NIS whereas its members are not in NIS.

What does ‘getenv group guix-builder’ return? It should show something

Typo: getent, not getenv.
guixbuild:x:30000:guixbuilder01,guixbuilder02,guixbuilder03,guixbuilder04,guixbuilder05,guixbuilder06,guixbuilder07,guixbuilder08,guixbuilder09,guixbuilder10

Each build account must have ‘guix-builder’ in its supplementary group

list.

Post by Cook, Malcolm
The daemon is running as user `guix` on host ${GUIXHOST}

http://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html

[Andreas Enge]

Post by Ludovic Courtès

Post by Cook, Malcolm
But, but, I do have a group `guix-builder`

guixbuild:x:30000:guixbuilder01,guixbuilder02,guixbuilder03,guixbuilder04,guixbuilder05,guixbuilder06,guixbuilder07,guixbuilder08,guixbuilder09,guixbuilder10

The documentation on the names of the group and the build users changed
recently; maybe your set-up corresponds to the former documentation.
Then you should start the daemon with
guix-daemon --build-users-group=guixbuild

Andreas

[Cook, Malcolm]

Post by Cook, Malcolm

Post by Cook, Malcolm
However
./pre-inst-env guix package -i hello
Gives me
guix package: error: build failed: the build users group
`guix-builder' has no members
But, but, I do have a group `guix-builder`
I think the problem may be that my guix-builder group was defined in NIS

whereas its members are not in NIS.
What does ‘getenv group guix-builder’ return? It should show something
guixbuild:x:30000:guixbuilder01,guixbuilder02,guixbuilder03,guixbuilder04,gu
ixbuilder05,guixbuilder06,guixbuilder07,guixbuilder08,guixbuilder09,guixbuild
er10

Answered below....

Post by Cook, Malcolm
Each build account must have ‘guix-builder’ in its supplementary group list.

Post by Cook, Malcolm
The daemon is running as user `guix` on host ${GUIXHOST}

http://www.gnu.org/software/guix/manual/html_node/Build-
Environment-Setup.html

I guess my issues are arising from challenging this (soft?) requirement.

My SA has provided me /gnu as NFS network share owned by network user `guix` and group owned by `guix-builder`.

However it is 'root squashed' on all servers, including my GUIX_HOST. In other works, the root account cannot write to it.

For this reason, I have tried

./configure --prefix=/gnu

rather than the suggested

./configure --localstatedir=/gnu/var --exec-prefix=/gnu

Since this suggestion results in `make install` needed to write to both the root-squashed network share and /usr/local which can only be done by root, an impossibility.

Or am I thinking about this wrong?

--prefix=/gnu has some additional advantage of getting the /share directory network wide, as presumably should be /etc/bashy_completion.d

I am making guix from fresh `git pull` master passing all tests (except for SKIPPING the container test thanks to recent commit - last week it was failing this one)

However, with this configuration, after install, and running the daemon as user 'guix', I now get:

bash-4.2$ guix build hello
guix build: error: build failed: acquiring/releasing lock: No locks available

Oh, right, and, I am still doing by hand:
su -c 'mkdir /gnu/var/guix/profiles/per-user; chmod a+rwx /gnu/var/guix/profiles/per-user' - guix

So, I think if I knew a little more about guix internals, I would expect that I could figure out a ./configuration that allows /gnu to reside on root-squashed network share.

Ricardo, you seem to have something _like_ this working. I'm guessing that your /gnu was local to your GUIXHOST, and so you did not need to solve this root squash issue.

Ricard & Ludo, do you mind thinking this through a little further with me?

Here is current my account setup

getent group guix-builder
guix-builder:!:20302:guix-builder1,guix-builder2,guix-builder3,guix-builder4,guix-builder5,guix-builder6,guix-builder7,guix-builder8,guix-builder9,guix-builder10

getent passwd guix
guix:$1$E5Ru3NpE$wZZY.cM8TwbRMHBI1UP110:3036:20302:Guix build user:/var/empty:/bin/bash

getent passwd guix-builder1 ## and all the other guix-builder
guix-builder1:!!:3048:20302:Guix build user 1:/var/empty:/sbin/nologin


~Malcolm

Post by Cook, Malcolm

Post by Cook, Malcolm
However trying to --install is not successful
/gnu/bin/guix package --install hello
accepted connection from pid 26597, uid 1232
error: while creating directory `/gnu/var/guix/profiles/per-

user/mec': Permission denied

Post by Cook, Malcolm
Please create the `/gnu/var/guix/profiles/per-user/mec' directory,

with you as the owner.

Post by Cook, Malcolm
Try "info '(guix) Invoking guix package'" for more information.
Which leads me to think the documentation is in error where it says "The

per-user directory is created when guix-daemon is started".
Well, /var/guix/profiles/per-user is created by the daemon, but .../mec is
created by ‘guix package’.

Post by Cook, Malcolm
mkdir /gnu/var/guix/profiles/per-user ## the doc says this should

happen by the daemon but not! FIXME! BUG?

Post by Cook, Malcolm
chmod a+rwx /gnu/var/guix/profiles/per-user

Perfect.
The daemon really does create /gnu/var/guix/profiles/per-user, making it
world-writable (see nix/libstore/local-store.cc:254.)
However, you mentioned guix-daemon was *not* running as root, in which
case it does not attempt to create this directory. That could be the reason.
Thanks for your report,

[Cook, Malcolm]

Hi,

I have resolved a main issue by having /gnu mounted without root squashing and running gnu-daemon as root, as strongly suggested after building with `./configure --localstatedir=/gnu/var --exec-prefix=/gnu`

And .... I am proceeding largely happily.

So.... WOOHOO and WHOOWHEE and general applause, cheers, and well-wishing from all quarters.

Thanks for everyone's advice and help.

I will start new threads with other issues.

Ludo, did I tell you, as you asked, I did build guix from master also without any FAILS and only SKIPPED one test, containers.scm.

~Malcolm

-----Original Message-----
From: Cook, Malcolm
Sent: Monday, July 20, 2015 5:38 PM
To: 'Ludovic Courtès'
Cc: 'Ricardo Wurmus'; 'Guix-devel'; 'Pjotr Prins'; 'bio-
Subject: RE: Using a shared Guix store (was RE: [Bio-packaging] testing out
guix)

Post by Cook, Malcolm

Post by Cook, Malcolm
However
./pre-inst-env guix package -i hello
Gives me
guix package: error: build failed: the build users group
`guix-builder' has no members
But, but, I do have a group `guix-builder`
I think the problem may be that my guix-builder group was defined in NIS

whereas its members are not in NIS.
What does ‘getenv group guix-builder’ return? It should show something
guixbuild:x:30000:guixbuilder01,guixbuilder02,guixbuilder03,guixbuilde
r04,gu
ixbuilder05,guixbuilder06,guixbuilder07,guixbuilder08,guixbuilder09,gu
ixbuild
er10

Answered below....

Post by Cook, Malcolm
Each build account must have ‘guix-builder’ in its supplementary group list.

Post by Cook, Malcolm
The daemon is running as user `guix` on host ${GUIXHOST}

http://www.gnu.org/software/guix/manual/html_node/Build-
Environment-Setup.html

I guess my issues are arising from challenging this (soft?) requirement.
My SA has provided me /gnu as NFS network share owned by network user
`guix` and group owned by `guix-builder`.
However it is 'root squashed' on all servers, including my GUIX_HOST. In
other works, the root account cannot write to it.
For this reason, I have tried
./configure --prefix=/gnu
rather than the suggested
./configure --localstatedir=/gnu/var --exec-prefix=/gnu
Since this suggestion results in `make install` needed to write to both the
root-squashed network share and /usr/local which can only be done by root,
an impossibility.
Or am I thinking about this wrong?
--prefix=/gnu has some additional advantage of getting the /share directory
network wide, as presumably should be /etc/bashy_completion.d
I am making guix from fresh `git pull` master passing all tests (except for
SKIPPING the container test thanks to recent commit - last week it was failing
this one)
However, with this configuration, after install, and running the daemon as
bash-4.2$ guix build hello
guix build: error: build failed: acquiring/releasing lock: No locks available
su -c 'mkdir /gnu/var/guix/profiles/per-user; chmod a+rwx
/gnu/var/guix/profiles/per-user' - guix
So, I think if I knew a little more about guix internals, I would expect that I
could figure out a ./configuration that allows /gnu to reside on root-squashed
network share.
Ricardo, you seem to have something _like_ this working. I'm guessing that
your /gnu was local to your GUIXHOST, and so you did not need to solve this
root squash issue.
Ricard & Ludo, do you mind thinking this through a little further with me?
Here is current my account setup
getent group guix-builder
guix-builder:!:20302:guix-builder1,guix-builder2,guix-builder3,guix-
builder4,guix-builder5,guix-builder6,guix-builder7,guix-builder8,guix-
builder9,guix-builder10
getent passwd guix
guix:$1$E5Ru3NpE$wZZY.cM8TwbRMHBI1UP110:3036:20302:Guix
build user:/var/empty:/bin/bash
getent passwd guix-builder1 ## and all the other guix-builder
guix-builder1:!!:3048:20302:Guix build user
1:/var/empty:/sbin/nologin
~Malcolm

Post by Cook, Malcolm

Post by Cook, Malcolm
However trying to --install is not successful
/gnu/bin/guix package --install hello
accepted connection from pid 26597, uid 1232
error: while creating directory `/gnu/var/guix/profiles/per-

user/mec': Permission denied

Post by Cook, Malcolm
Please create the `/gnu/var/guix/profiles/per-user/mec' directory,

with you as the owner.

Post by Cook, Malcolm
Try "info '(guix) Invoking guix package'" for more information.
Which leads me to think the documentation is in error where it says "The

per-user directory is created when guix-daemon is started".
Well, /var/guix/profiles/per-user is created by the daemon, but
.../mec is created by ‘guix package’.

Post by Cook, Malcolm
mkdir /gnu/var/guix/profiles/per-user ## the doc says this should

happen by the daemon but not! FIXME! BUG?

Post by Cook, Malcolm
chmod a+rwx /gnu/var/guix/profiles/per-user

Perfect.
The daemon really does create /gnu/var/guix/profiles/per-user, making
it world-writable (see nix/libstore/local-store.cc:254.)
However, you mentioned guix-daemon was *not* running as root, in

which

Post by Cook, Malcolm
case it does not attempt to create this directory. That could be the reason.
Thanks for your repo

[Ricardo Wurmus]

Post by Cook, Malcolm
I have resolved a main issue by having /gnu mounted without root
squashing and running gnu-daemon as root, as strongly suggested after
building with `./configure --localstatedir=/gnu/var
--exec-prefix=/gnu`
And .... I am proceeding largely happily.
So.... WOOHOO and WHOOWHEE and general applause, cheers, and well-wishing from all quarters.

Woo! That’s great!
Glad it’s working for you.

~~ Ricardo

[Ludovic Courtès]

Post by Cook, Malcolm
So.... WOOHOO and WHOOWHEE and general applause, cheers, and well-wishing from all quarters.

Yay! :-)

Post by Cook, Malcolm
Ludo, did I tell you, as you asked, I did build guix from master also without any FAILS and only SKIPPED one test, containers.scm.

Excellent.

Thanks for the joyful feedback!

Ludo’.

[Cook, Malcolm]

Ludovic, and Ricardo,

Post by Ludovic Courtès
Yes, one way to help would be by trying a simple setup using Guix as it
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20381
If you could test this and provide feedback about the other options discussed
there, that would be great (please email

Excellent. That definitely looks testable by me, once I'm done with my basic site deployment. I've been moving rather slowly but might be ready to contrast such approach with simply calling guix on guix-master over ssh. I'll plan on it. Maybe a week or two out.

Reading the thread, it sounds like Ricardo has today already had good success with socat. Yay!

In that thread, you suggested "What about installing Guix in /gnu/bin (say) and sharing it over NFS?". Is there a way to `./bootstrap ./configure ` this? Something like -- exec_prefix=/gnu --localstatedir=/gnu/var?

Also, you discussed need for setting NIX_STATE_DIR. This should not be needed if guix was already installed with configuration of --localstatedir=/guix/var (with /guix being nfs mounted), right?

Ricardo also wrote "For ‘guix package’ to work, /gnu/var/guix/profiles/per-user must be shared read-write (over NFS) with correct UID mapping." - I really don't understand this and tried to elicit feedback in http://lists.nongnu.org/archive/html/guix-devel/2015-07/msg00216.html - Can anyone give me clarification on this. I though the guix daemon is going to manage these profiles.

Honestly, much of my understanding of the above comes from reading the manual, archived IRC chats and mail threads, FOSDEM vids, and blog posts - which has been great. But, I've yet to succeed in my own site deploy, which will give me testbed to stretch my own understanding. Today, I had the following failure....

Ricardo, et. al., would you mind sharing some details on your network sharing configuration as regards users/groups and privileges, for I seem to have painted myself into a corner with the following

/gnu is an NFS mount, owned by network user 'guix', group-owned by guix-builder with mod of 1775

guix is being built from git configured with -- exec_prefix=/gnu --localstatedir=/gnu/var

`make check` results are

============================================================================
Testsuite summary for GNU Guix 0.8.3
============================================================================
# TOTAL: 42
# PASS: 32
# SKIP: 0
# XFAIL: 0
# FAIL: 10
# XPASS: 0
# ERROR: 0

With details below.... I can submit all this to bug-***@gnu.org, but I suspect (1) there is something fundamentally wrong with the way I am setting things up (2) your are reading this anyway ;)

Can you see any gotchas I haven't got yet? Can you see the error in my ways and suggest a way through?

FWIW, I try to `make install` anyway, and, I pause.... I cannot complete a make install without "changing hats". I find that it works to do the following

sudo make install-data # as root on localhost
sudo -u guix make install-exec ## since root on localhost is squashed on the NFS mount point,

But I don't know if this is complete or not (the Makefile is complex!).

Anyway, "changing hats" feels very wrong, and I can feel more trouble coming when I want to run the daemon....

Thanks so much everyone, and, keep on guixing,

~Malcolm


PS - in case it helps - below are test results and attached are tarred logs - I have not yet taught myself how to read them with understanding.... - (have a great weekend!)

PASS: tests/base32.scm
PASS: tests/base64.scm
FAIL: tests/cpio.scm
PASS: tests/hash.scm
PASS: tests/pk-crypto.scm
PASS: tests/pki.scm
PASS: tests/sets.scm
PASS: tests/substitute.scm
FAIL: tests/builders.scm
FAIL: tests/derivations.scm
PASS: tests/ui.scm
PASS: tests/records.scm
PASS: tests/utils.scm
PASS: tests/build-utils.scm
FAIL: tests/packages.scm
PASS: tests/snix.scm
PASS: tests/hackage.scm
PASS: tests/elpa.scm
FAIL: tests/store.scm
PASS: tests/monads.scm
PASS: tests/gexp.scm
FAIL: tests/nar.scm
FAIL: tests/union.scm
PASS: tests/profiles.scm
FAIL: tests/syscalls.scm
PASS: tests/gremlin.scm
PASS: tests/lint.scm
PASS: tests/publish.scm
PASS: tests/size.scm
PASS: tests/guix-build.sh
PASS: tests/guix-download.sh
PASS: tests/guix-hash.sh
PASS: tests/guix-package.sh
FAIL: tests/guix-package-net.sh
PASS: tests/guix-system.sh
PASS: tests/guix-archive.sh
PASS: tests/guix-authenticate.sh
FAIL: tests/guix-environment.sh
PASS: tests/guix-lint.sh
PASS: tests/guix-register.sh
PASS: tests/guix-gc.sh
PASS: tests/guix-daemon.sh

Post by Ludovic Courtès
So no ETA, but once we have feedback from you, Ricardo, and other
interested parties, it will be easier to make a choice and implement it.
:-)
Thanks,
Ludo’.

[Ludovic Courtès]

(Stripping the list of recipients.)

Post by Cook, Malcolm
In that thread, you suggested "What about installing Guix in /gnu/bin (say) and sharing it over NFS?". Is there a way to `./bootstrap ./configure ` this? Something like -- exec_prefix=/gnu --localstatedir=/gnu/var?

Yes, --exec-prefix=/gnu --localstatedir=/gnu/var (note “--exec-prefix”
with a hyphen.)

Post by Cook, Malcolm
Also, you discussed need for setting NIX_STATE_DIR. This should not be needed if guix was already installed with configuration of --localstatedir=/guix/var (with /guix being nfs mounted), right?

Right.

Post by Cook, Malcolm
Ricardo also wrote "For ‘guix package’ to work, /gnu/var/guix/profiles/per-user must be shared read-write (over NFS) with correct UID mapping." - I really don't understand this and tried to elicit feedback in http://lists.nongnu.org/archive/html/guix-devel/2015-07/msg00216.html - Can anyone give me clarification on this. I though the guix daemon is going to manage these profiles.

‘guix package’ makes remote procedure calls (RPCs) to the daemon and the
daemon does most things on its behalf.

However, there’s one thing that ‘guix package’ does directly by itself:
updating the profile symlinks, ~/.guix-profile and its target, which is
in $localstatedir.

For that reason, ‘guix package’ must have write access to
$localstatedir.

Does that clarify things?

Post by Cook, Malcolm
============================================================================
Testsuite summary for GNU Guix 0.8.3
============================================================================
# TOTAL: 42
# PASS: 32
# SKIP: 0
# XFAIL: 0
# FAIL: 10
# XPASS: 0
# ERROR: 0

[...]

Post by Cook, Malcolm
FAIL: tests/cpio.scm

What does ‘cpio --version’ returns? Could you email me (privately) the
file ‘t.cpio’ made with these commands:

cd /path/to/guix
ln -s chbouib /tmp/somelink
(for i in / $PWD /tmp/somelink $PWD/guix.scm \
$PWD/guix/build/syscalls.scm $PWD/guix/packages.scm ; \
do echo $i ; done ) | \
cpio -o -O t.cpio -H newc

Post by Cook, Malcolm
FAIL: tests/builders.scm
FAIL: tests/derivations.scm

These two show errors like:

--8<---------------cut here---------------start------------->8---
output path `/home/mec/project/sce/guix/test-tmp/store/2l1h7am3q3c6sd4z5425mp3kjady6nla-hello-2.8.tar.gz should be a non-executable regular file
--8<---------------cut here---------------end--------------->8---

Which makes it sound like files are automatically made executable on
this file system.

What does ‘umask’ return?

What about ‘mount | grep /home’?

Post by Cook, Malcolm
FAIL: tests/packages.scm

Ditto.

Post by Cook, Malcolm
PASS: tests/snix.scm
PASS: tests/hackage.scm
PASS: tests/elpa.scm
FAIL: tests/store.scm

This one shows this:

--8<---------------cut here---------------start------------->8---
;;; (c #<condition &nix-protocol-error [message: "path `/home/mec/project/sce/guix/test-tmp/stor?/vlryjv621i7xj4sr8ajzl9k9w5bmkv5c-text' is not in the Nix store" status: 1] 24a61e0>)
tests/store.scm:560: FAIL import corrupt path

;;; (verify1 #t)

;;; (verify2 #f)

;;; (verify3 #t)
tests/store.scm:620: FAIL verify-store + check-contents
--8<---------------cut here---------------end--------------->8---

This particular test is not designed to work with long $top_builddir
like this (the test corrupts an archive, but in this case the corruption
occurs in the middle of the file name, which confuses the test logic.)
I’ll see how to fix it.

Post by Cook, Malcolm
FAIL: tests/nar.scm

--8<---------------cut here---------------start------------->8---
tests/nar.scm:172: FAIL write-file puts file in C locale collation order
--8<---------------cut here---------------end--------------->8---

Post by Cook, Malcolm
FAIL: tests/union.scm

“Non-executable” error as before.

Post by Cook, Malcolm
PASS: tests/profiles.scm
FAIL: tests/syscalls.scm

Harmless clone/setns/pivot-root test failures probably due to the fact
that the host kernel is too old. Can you confirm that this is fixed in
‘master’ (commit b62a3eb)? Simply run:

make check TESTS=tests/syscalls.scm

Post by Cook, Malcolm
FAIL: tests/guix-package-net.sh

“Non-executable” error again.

Post by Cook, Malcolm
FAIL: tests/guix-environment.sh

Likewise.

Thanks for your help!

Ludo’.

[Cook, Malcolm]

Ludo,

As you observed, the majority of my problems were due to building guix on a network mount having some odd nfs4_facl setting that was making all the files be executable, which is explicitly tested against in many tests, and thus they were all failing.

Moving off that file system, and building in /tmp, I am now building guix-0.8.2 on centOS7 passing all tests.

Thanks so much for reviewing these logs and your guidance.

I hope to be cooking with guix soon.

~Malcolm

-----Original Message-----
Sent: Monday, July 13, 2015 11:46 AM
To: Cook, Malcolm
Cc: Guix-devel
Subject: Test suite failures
(Stripping the list of recipients.)

Post by Cook, Malcolm
In that thread, you suggested "What about installing Guix in /gnu/bin (say)

and sharing it over NFS?". Is there a way to `./bootstrap ./configure ` this?
Something like -- exec_prefix=/gnu --localstatedir=/gnu/var?
Yes, --exec-prefix=/gnu --localstatedir=/gnu/var (note “--exec-prefix”
with a hyphen.)

Post by Cook, Malcolm
Also, you discussed need for setting NIX_STATE_DIR. This should not be

needed if guix was already installed with configuration of --
localstatedir=/guix/var (with /guix being nfs mounted), right?
Right.

Post by Cook, Malcolm
Ricardo also wrote "For ‘guix package’ to work, /gnu/var/guix/profiles/per-

user must be shared read-write (over NFS) with correct UID mapping." - I
really don't understand this and tried to elicit feedback in
http://lists.nongnu.org/archive/html/guix-devel/2015-07/msg00216.html -
Can anyone give me clarification on this. I though the guix daemon is going to
manage these profiles.
‘guix package’ makes remote procedure calls (RPCs) to the daemon and the
daemon does most things on its behalf.
updating the profile symlinks, ~/.guix-profile and its target, which is in
$localstatedir.
For that reason, ‘guix package’ must have write access to $localstatedir.
Does that clarify things?
==========================================================
============

Post by Cook, Malcolm
======
Testsuite summary for GNU Guix 0.8.3

==========================================================
============

Post by Cook, Malcolm
======
# TOTAL: 42
# PASS: 32
# SKIP: 0
# XFAIL: 0
# FAIL: 10
# XPASS: 0
# ERROR: 0

[...]

Post by Cook, Malcolm
FAIL: tests/cpio.scm

What does ‘cpio --version’ returns? Could you email me (privately) the file
cd /path/to/guix
ln -s chbouib /tmp/somelink
(for i in / $PWD /tmp/somelink $PWD/guix.scm \
$PWD/guix/build/syscalls.scm $PWD/guix/packages.scm ; \
do echo $i ; done ) | \
cpio -o -O t.cpio -H newc

Post by Cook, Malcolm
FAIL: tests/builders.scm
FAIL: tests/derivations.scm

--8<---------------cut here---------------start------------->8---
output path `/home/mec/project/sce/guix/test-
tmp/store/2l1h7am3q3c6sd4z5425mp3kjady6nla-hello-2.8.tar.gz should be a
non-executable regular file --8<---------------cut here---------------end----------
----->8---
Which makes it sound like files are automatically made executable on this file
system.
What does ‘umask’ return?
What about ‘mount | grep /home’?

Post by Cook, Malcolm
FAIL: tests/packages.scm

Ditto.

Post by Cook, Malcolm
PASS: tests/snix.scm
PASS: tests/hackage.scm
PASS: tests/elpa.scm
FAIL: tests/store.scm

--8<---------------cut here---------------start------------->8---
;;; (c #<condition &nix-protocol-error [message: "path
`/home/mec/project/sce/guix/test-
tmp/stor?/vlryjv621i7xj4sr8ajzl9k9w5bmkv5c-text' is not in the Nix store"
status: 1] 24a61e0>)
tests/store.scm:560: FAIL import corrupt path
;;; (verify1 #t)
;;; (verify2 #f)
;;; (verify3 #t)
tests/store.scm:620: FAIL verify-store + check-contents --8<---------------cut
here---------------end--------------->8---
This particular test is not designed to work with long $top_builddir like this
(the test corrupts an archive, but in this case the corruption occurs in the
middle of the file name, which confuses the test logic.) I’ll see how to fix it.

Post by Cook, Malcolm
FAIL: tests/nar.scm

--8<---------------cut here---------------start------------->8---
tests/nar.scm:172: FAIL write-file puts file in C locale collation order --8<-------
--------cut here---------------end--------------->8---

Post by Cook, Malcolm
FAIL: tests/union.scm

“Non-executable” error as before.

Post by Cook, Malcolm
PASS: tests/profiles.scm
FAIL: tests/syscalls.scm

Harmless clone/setns/pivot-root test failures probably due to the fact that
the host kernel is too old. Can you confirm that this is fixed in ‘master’
make check TESTS=tests/syscalls.scm

Post by Cook, Malcolm
FAIL: tests/guix-package-net.sh

“Non-executable” error again.

Post by Cook, Malcolm
FAIL: tests/guix-environment.sh

L

[Ludovic Courtès]

Post by Cook, Malcolm
As you observed, the majority of my problems were due to building guix on a network mount having some odd nfs4_facl setting that was making all the files be executable, which is explicitly tested against in many tests, and thus they were all failing.
Moving off that file system, and building in /tmp, I am now building guix-0.8.2 on centOS7 passing all tests.

That’s good news. It would be great though if you could run the tests
on ‘master’ as well, like you did before, to make sure all the issues
really vanished.

Thanks,
Ludo’.