[PATCHES] gnu: linux-libre: Update to 4.16

[Mark H Weaver]

Hello Guix,

Here are preliminary patches to update to linux-libre-4.16. It works
well on my x86_64 GuixSD system on the 'core-updates' branch, but it's
not convenient for me to test it on 'master'.

I would be grateful to hear reports of testing this on 'master', and
also on 'armhf' systems.

One more thing: I had to add 'flex' and 'bison' as native-inputs because
the build system in 4.16 now concludes that they need to be re-run for
some reason. This could perhaps be avoided by adding a phase to touch
the generated files, but I didn't try it.

I think I'd prefer to add a phase to *delete* the generated files and
therefore _always_ run flex and bison, so that we are using source code
instead of trusting generated code from upstream.

What do you think?

Anyway, here are the patches. Test reports encouraged.

Mark

[Leo Famulari]

Post by Mark H Weaver
One more thing: I had to add 'flex' and 'bison' as native-inputs because
the build system in 4.16 now concludes that they need to be re-run for
some reason. This could perhaps be avoided by adding a phase to touch
the generated files, but I didn't try it.

I noticed this too while building 4.16 with Debian's tooling (`make
bindeb-pkg`).

Post by Mark H Weaver
Anyway, here are the patches. Test reports encouraged.

Thanks, I'm testing this on x86_64 (it's what I have access to).

I noticed this warning at the beginning of the build phase:

------
starting phase `build'
scripts/kconfig/conf --silentoldconfig Kconfig
Makefile:976: "Cannot use CONFIG_STACK_VALIDATION=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel"
------

The warning goes away when elfutils is available in the build
environment.

[Mark H Weaver]

Post by Leo Famulari

Post by Mark H Weaver
One more thing: I had to add 'flex' and 'bison' as native-inputs because
the build system in 4.16 now concludes that they need to be re-run for
some reason. This could perhaps be avoided by adding a phase to touch
the generated files, but I didn't try it.

I noticed this too while building 4.16 with Debian's tooling (`make
bindeb-pkg`).

That's interesting. I guess the timestamps in your unpacked source
directory were probably not zeroed, so I guess that timestamps are not
the issue here.

Post by Leo Famulari

Post by Mark H Weaver
Anyway, here are the patches. Test reports encouraged.

Thanks, I'm testing this on x86_64 (it's what I have access to).
------
starting phase `build'
scripts/kconfig/conf --silentoldconfig Kconfig
Makefile:976: "Cannot use CONFIG_STACK_VALIDATION=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel"
------
The warning goes away when elfutils is available in the build
environment.

Ah, good catch! I've now added to my queue another patch which adds
elfutils to native-inputs in 'make-linux-libre'. I'll push it to master
along with the 4.16 update after a bit more testing.

Thanks,
Mark

[Vincent Legoll]

Hello,

Post by Mark H Weaver

Post by Leo Famulari

Post by Mark H Weaver
One more thing: I had to add 'flex' and 'bison' as native-inputs because
the build system in 4.16 now concludes that they need to be re-run for
some reason. This could perhaps be avoided by adding a phase to touch
the generated files, but I didn't try it.

I noticed this too while building 4.16 with Debian's tooling (`make
bindeb-pkg`).

That's interesting. I guess the timestamps in your unpacked source
directory were probably not zeroed, so I guess that timestamps are not
the issue here.

This is not specific to guix / guixsd building the kernel.

I think that's intentional, the kbuild system has been modified to always
regenerate the files from their sources.

See lkml discussion:
https://lkml.org/lkml/2017/8/19/49

See commit:
29c833061c1d8c2d1d23a62e7061561eadd76cdb

Merged in :
562f36ed28e6faa4245ea2ca1392d90ab98ebbe8

Hope that helps

[Mark H Weaver]

Hi Vincent,

Post by Vincent Legoll

Post by Mark H Weaver

Post by Leo Famulari

Post by Mark H Weaver
One more thing: I had to add 'flex' and 'bison' as native-inputs because
the build system in 4.16 now concludes that they need to be re-run for
some reason. This could perhaps be avoided by adding a phase to touch
the generated files, but I didn't try it.

I noticed this too while building 4.16 with Debian's tooling (`make
bindeb-pkg`).

That's interesting. I guess the timestamps in your unpacked source
directory were probably not zeroed, so I guess that timestamps are not
the issue here.

This is not specific to guix / guixsd building the kernel.
I think that's intentional, the kbuild system has been modified to always
regenerate the files from their sources.
https://lkml.org/lkml/2017/8/19/49
29c833061c1d8c2d1d23a62e7061561eadd76cdb
562f36ed28e6faa4245ea2ca1392d90ab98ebbe8

Thank you for getting to the bottom of this, Vincent!

Note that although the original proposed patch would have removed _all_
*.c_shipped files in the tree, the merged patch removed only the ones
for scripts/kconfig/zconf.{l,y}. The following generated files still
remain in the upstream 4.16 tarball:

***@jojen ~$ find linux-4.16 | egrep '\.tab\.'
linux-4.16/scripts/dtc/dtc-parser.tab.c_shipped
linux-4.16/scripts/dtc/dtc-parser.tab.h_shipped
linux-4.16/scripts/genksyms/parse.tab.c_shipped
linux-4.16/scripts/genksyms/parse.tab.h_shipped

So, we still have a decision to make: whether to delete these generated
files (possibly in a snippet) to avoid using pre-generated non-source
files in our build. I would be in favor of it.

I'd like to hear opinions on this.

Mark

[Chris Marusich]

Post by Mark H Weaver
So, we still have a decision to make: whether to delete these generated
files (possibly in a snippet) to avoid using pre-generated non-source
files in our build. I would be in favor of it.
I'd like to hear opinions on this.

Perhaps we can consider our existing packages as a precedent. Many
packages include files in their source distribution that were
auto-generated by the Autotools. For example, consider the "configure"
script that Autoconf generates. Is there a significant difference
between the "configure" script and the "pre-generated non-source files"
you're talking about?

[Ludovic Courtès]

Hello,

Post by Chris Marusich

Post by Mark H Weaver
So, we still have a decision to make: whether to delete these generated
files (possibly in a snippet) to avoid using pre-generated non-source
files in our build. I would be in favor of it.
I'd like to hear opinions on this.

I’d be in favor of removing these, especially since that seems to be an
easy change, but…

Post by Chris Marusich
Perhaps we can consider our existing packages as a precedent. Many
packages include files in their source distribution that were
auto-generated by the Autotools. For example, consider the "configure"
script that Autoconf generates. Is there a significant difference
between the "configure" script and the "pre-generated non-source files"
you're talking about?

Indeed, there’s a long tradition in GNU to ship generated code to
facilitate bootstrapping. There’s configure, Makefile.in, etc., and
there’s also Bison- and Flex-generated files often.

I have mixed feelings about this. I think it’s great to be able to use
these pre-generated files; our bootstrap graph would be much more
complicated or even out of reach if we were to re-generate everything.
OTOH, it’s true that this is the elephant in the room in terms of
bootstrapping.

Maybe it’s a can of worms we’d rather leave aside. :-)

Thoughts?

Ludo’.