core-updates, let’s go!

[Ludovic Courtès]

Hello Guix!

‘core-updates’ is doing OK!

https://hydra.gnu.org/jobset/gnu/core-updates

The remaining issues are GCC 4.8 and 4.9 failing to build, which are
nothing serious and left as an exercise to the reader.

What about starting a new evaluation of all the packages now so we can
merge it early 2018?

Efraim, can you confirm that everything’s alright on aarch64?

We’re lacking a couple of glibc commits from the ‘release/2.26/master’
branch (see below), but Florian Weimer marked them as non-critical
provided we have the previous fixes. Should we include them anyway and
rebuild the whole thing? (I’m inclined to answer “no”.)

Thanks,
Ludo’.


commit 633e2f7f3d88df6427aa3a7a984d3a6b796d9611 (HEAD -> release/2.26/master, origin/release/2.26/master)
Author: Florian Weimer <***@redhat.com>
Date: Thu Dec 14 15:05:57 2017 +0100

elf: Count components of the expanded path in _dl_init_path [BZ #22607]

(cherry picked from commit 3ff3dfa5af313a6ea33f3393916f30eece4f0171)

commit 43b3cb59b2288953efc26e70fe7c6eb437513b1f
Author: Florian Weimer <***@redhat.com>
Date: Thu Dec 14 15:18:38 2017 +0100

elf: Compute correct array size in _dl_init_paths [BZ #22606]

(cherry picked from commit 8a0b17e48b83e933960dfeb8fa08b259f03f310e)

[Efraim Flashner]

Post by Ludovic Courtès
Hello Guix!
‘core-updates’ is doing OK!
https://hydra.gnu.org/jobset/gnu/core-updates
The remaining issues are GCC 4.8 and 4.9 failing to build, which are
nothing serious and left as an exercise to the reader.
What about starting a new evaluation of all the packages now so we can
merge it early 2018?

I've merged master into core-updates and I'm running 'make' against it
as we speak to make sure I didn't make any obvious errors, and then I'm
about to push it.

Post by Ludovic Courtès
Efraim, can you confirm that everything’s alright on aarch64?

I've been keeping mesa in good shape on aarch64, I'm pretty sure gtk+,

Post by Ludovic Courtès
We’re lacking a couple of glibc commits from the ‘release/2.26/master’
branch (see below), but Florian Weimer marked them as non-critical
provided we have the previous fixes. Should we include them anyway and
rebuild the whole thing? (I’m inclined to answer “no”.)
Thanks,
Ludo’.
commit 633e2f7f3d88df6427aa3a7a984d3a6b796d9611 (HEAD -> release/2.26/master, origin/release/2.26/master)
Date: Thu Dec 14 15:05:57 2017 +0100
elf: Count components of the expanded path in _dl_init_path [BZ #22607]
(cherry picked from commit 3ff3dfa5af313a6ea33f3393916f30eece4f0171)
commit 43b3cb59b2288953efc26e70fe7c6eb437513b1f
Date: Thu Dec 14 15:18:38 2017 +0100
elf: Compute correct array size in _dl_init_paths [BZ #22606]
(cherry picked from commit 8a0b17e48b83e933960dfeb8fa08b259f03f310e)

[Danny Milosavljevic]

I'd like to get the patch

[bug#29856] [PATCH core-updates] guix: python-build-system: Modify ".py" files in-place.

into this core-updates cycle. What do you all think?

If we don't do it then a lot of Python programs (the ones who use "console_scripts" in setup.py as they should) will not work (because we emit a shell script wrapper and Python tries to import it - that's going to end badly). On the other hand, I don't know the effect on other programs (read: I didn't build the world with it - but I did build quite a few Python programs). You can try it with the guix package ledger-agent and try to invoke "ledger-agent".

The patch only affects files whose names end in ".py" since Python's importer will only find those in any case. So its possible ill effect is limited.

Hartmut, what do you think?

[Danny Milosavljevic]

On Sun, 31 Dec 2017 12:14:16 +0100

Post by Danny Milosavljevic
I'd like to get the patch
[bug#29856] [PATCH core-updates] guix: python-build-system: Modify ".py" files in-place.
into this core-updates cycle. What do you all think?
If we don't do it then a lot of Python programs (the ones who use "console_scripts" in setup.py as they should) will not work (because we emit a shell script wrapper and Python tries to import it - that's going to end badly). On the other hand, I don't know the effect on other programs (read: I didn't build the world with it - but I did build quite a few Python programs). You can try it with the guix package ledger-agent and try to invoke "ledger-agent".
The patch only affects files whose names end in ".py" since Python's importer will only find those in any case. So its possible ill effect is limited.

^^^ Correction: whose names end in ".py" and that end up in "bin" or "sbin". Otherwise it would be common.

Post by Danny Milosavljevic
Hartmut, what do you think?

[Marius Bakke]

Post by Danny Milosavljevic
I'd like to get the patch
[bug#29856] [PATCH core-updates] guix: python-build-system: Modify ".py" files in-place.
into this core-updates cycle. What do you all think?

We already have a substantial amount of Python changes on core-updates,
so we could save ourselves some trouble by adding this when there's less
"congestion" in case of any problems.

[Ricardo Wurmus]

Post by Marius Bakke

Post by Danny Milosavljevic
I'd like to get the patch
[bug#29856] [PATCH core-updates] guix: python-build-system: Modify ".py" files in-place.
into this core-updates cycle. What do you all think?

We already have a substantial amount of Python changes on core-updates,
so we could save ourselves some trouble by adding this when there's less
"congestion" in case of any problems.

I agree with Marius.

There’s also a change to the wrap-program procedure that does without
the shell wrappers and instead prepends a short Guile program, which is
read as a comment in the target language. It’s better to play with this
in the next core-updates cycle.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net

[Hartmut Goebel]

Post by Ricardo Wurmus
There’s also a change to the wrap-program procedure that does without
the shell wrappers and instead prepends a short Guile program, which is
read as a comment in the target language. It’s better to play with this
in the next core-updates cycle.

I assume, this is what we discussed in
<https://lists.gnu.org/archive/html/guix-devel/2017-11/msg00041.html>.

I just wonder whether this will keep the encoding-lines in place, like
these:

#!/usr/bin/python
# -*- coding: latin-1 -*-
from __future__ import print_function

[Ricardo Wurmus]

Hi Hartmut,

Post by Hartmut Goebel

Post by Ricardo Wurmus
There’s also a change to the wrap-program procedure that does without
the shell wrappers and instead prepends a short Guile program, which is
read as a comment in the target language. It’s better to play with this
in the next core-updates cycle.

I assume, this is what we discussed in
<https://lists.gnu.org/archive/html/guix-devel/2017-11/msg00041.html>.
I just wonder whether this will keep the encoding-lines in place, like
#!/usr/bin/python
# -*- coding: latin-1 -*-
from __future__ import print_function

In the current implementation that I have here it would not.

According to the documentation[1], the regular expression
“coding[=:]\s*([-\w.]+)” must match for a comment on the second line of
a script, so I’d think that the following would still be valid:

--8<---------------cut here---------------start------------->8---
#!/gnu/store/…/bin/guile --no-auto-compile
#!#; -*- coding: latin-1 -*-
#\-(setenv …)
#\-(apply execl …)
#!/gnu/store/…/bin/python3
# -*- coding: latin-1 -*-
…
--8<---------------cut here---------------end--------------->8---

All we need to do is read the second line of the script and see if it is
a coding comment; in that case we’d place the content on the second
line. I’ll add a special case for this and a test case. Thanks for
bringing it up!

[1]: https://docs.python.org/3/reference/lexical_analysis.html#encoding-declarations

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net

[Hartmut Goebel]

Post by Ricardo Wurmus
I’ll add a special case for this and a test case.

Great! Thanks.

[Ricardo Wurmus]

Post by Hartmut Goebel

Post by Ricardo Wurmus
I’ll add a special case for this and a test case.

Great! Thanks.

FYI: I sent an initial implementation to guix-patches and got #29951.
We can further discuss things there.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net

[Marius Bakke]

Post by Ludovic Courtès
Hello Guix!
‘core-updates’ is doing OK!
https://hydra.gnu.org/jobset/gnu/core-updates
The remaining issues are GCC 4.8 and 4.9 failing to build, which are
nothing serious and left as an exercise to the reader.
What about starting a new evaluation of all the packages now so we can
merge it early 2018?
Efraim, can you confirm that everything’s alright on aarch64?
We’re lacking a couple of glibc commits from the ‘release/2.26/master’
branch (see below), but Florian Weimer marked them as non-critical
provided we have the previous fixes. Should we include them anyway and
rebuild the whole thing? (I’m inclined to answer “no”.)

I agree, let's not hold up this branch any longer. Godspeed!

[Ludovic Courtès]

Hello Guix!

I’ve started a new evaluation, let’s see what happens!

We’ll have to stay focused in the coming days to fix everything as
quickly as we can.

Thanks, and happy new year where applicable! :-)

Ludo’.

[Efraim Flashner]

Post by Ludovic Courtès
Hello Guix!
I’ve started a new evaluation, let’s see what happens!
We’ll have to stay focused in the coming days to fix everything as
quickly as we can.
Thanks, and happy new year where applicable! :-)
Ludo’.

I've tracked down the ***@4.[89] build failures, I'll try to fix ***@4.9
without affecting libstdc++-boot0

[Leo Famulari]

Post by Ludovic Courtès
We’ll have to stay focused in the coming days to fix everything as
quickly as we can.

Libtiff failed the first build attempt when the upstream FTP server
timed out. It has succeeded since then, but this will have caused a lot
of dependent packages to fail.

Should I "Restart all dependency failed builds" now?

I changed the libtiff source URL to use HTTP. Hopefully it will be more
reliable.

[Leo Famulari]

Post by Leo Famulari
Libtiff failed the first build attempt when the upstream FTP server
timed out. It has succeeded since then, but this will have caused a lot
of dependent packages to fail.
Should I "Restart all dependency failed builds" now?

I went ahead and did this because it was too hard to failed packages
that did not fail due to the earlier libtiff problem.

[Leo Famulari]

Post by Ludovic Courtès
We’ll have to stay focused in the coming days to fix everything as
quickly as we can.

FYI everyone, I merged master -> core-updates and updated Fossil because
the latest Fossil requires a new SQLite.

[Leo Famulari]

Post by Ludovic Courtès
We’ll have to stay focused in the coming days to fix everything as
quickly as we can.

The latest core-updates evaluation is coming together [0], but there are
still a few notable issues:

* Pandas is failing to build:

https://hydra.gnu.org/job/gnu/core-updates/python2-pandas-0.19.2.x86_64-linux

* CLISP is failing to build, which breaks SBCL:

https://hydra.gnu.org/job/gnu/core-updates/clisp-2.49-60.x86_64-linux

* Several packages are failing due to changes in ICU related to type
definitions. For example, evolution-data-server is failing to build,
which breaks GNOME:

https://hydra.gnu.org/job/gnu/core-updates/evolution-data-server-3.24.3.x86_64-linux

* ant-bootstrap failed to build, breaking most (all?) of the Java
packages. This is being discussed in <https://bugs.gnu.org/30107>. I
can't retry the ant-bootstrap build because the package is not public,
so there is no corresponding job on Hydra. An example:

https://hydra.gnu.org/build/2443807

* And more!

Why not check if the packages you rely on are working on core-updates?
These are the packages that fail compared to the master branch:

[0]
https://hydra.gnu.org/eval/109888?compare=master&full=1#tabs-now-fail

[Leo Famulari]

Post by Leo Famulari
The latest core-updates evaluation is coming together [0], but there are

Some of the SELinux packages are also failing to build:

https://hydra.gnu.org/build/2414231

I tried updating libsepol to 2.7, and that fixes some things, but it
breaks other packages...

If you use SELinux, please help :)

[Ricardo Wurmus]

Post by Leo Famulari

Post by Leo Famulari
The latest core-updates evaluation is coming together [0], but there are

https://hydra.gnu.org/build/2414231
I tried updating libsepol to 2.7, and that fixes some things, but it
breaks other packages...
If you use SELinux, please help :)

I updated to 2.7 in master and core-updates. It built fine on x86_64,
but according to Mark it broke on i686 and armhf. I’m going to take a
look at this today.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net

[Kei Kebreau]

Post by Leo Famulari

Post by Ludovic Courtès
We’ll have to stay focused in the coming days to fix everything as
quickly as we can.

The latest core-updates evaluation is coming together [0], but there are
https://hydra.gnu.org/job/gnu/core-updates/python2-pandas-0.19.2.x86_64-linux
https://hydra.gnu.org/job/gnu/core-updates/clisp-2.49-60.x86_64-linux
* Several packages are failing due to changes in ICU related to type
definitions. For example, evolution-data-server is failing to build,
https://hydra.gnu.org/job/gnu/core-updates/evolution-data-server-3.24.3.x86_64-linux
* ant-bootstrap failed to build, breaking most (all?) of the Java
packages. This is being discussed in <https://bugs.gnu.org/30107>. I
can't retry the ant-bootstrap build because the package is not public,
https://hydra.gnu.org/build/2443807
* And more!
Why not check if the packages you rely on are working on core-updates?
[0]
https://hydra.gnu.org/eval/109888?compare=master&full=1#tabs-now-fail

I have a patch for CLISP that addresses the source of the build
failure. I'm currently testing to see if the resulting binary can build
SBCL properly. Expect to see the patch on the guix-patches list soon!