wip-bootstrap updated

[Jan Nieuwenhuizen]

I've updated the wip-bootstrap branch[0] for Mes[1] 0.13. It has new
mes-boot and tcc-boot packages. mes-boot is a bootstrap version of
Mes; it only depends on mescc-tools and a previously compiled mes.M1
seed. Likewise, tcc-boot depends on a precompiled tcc-seed. Also,
tcc-boot uses a heavily patched version of the tcc sources.

Mes 0.13 is the first release that can bootstrap a fairly functional
tcc-boot. This bootstrapped tcc passes 67/68 C tests that were created
for MesCC. It can compile a version if itself where float, long long
and bitfield are patched out...but linking fails. This amazing compiler
can now be played with by doing something like

--8<---------------cut here---------------start------------->8---
git checkout wip-bootstrap
make
./pre-inst-env guix build tcc-boot # may take ~2h
./pre-inst-env guix environment --ad-hoc tcc-boot
mes-tcc --help #duck and run
--8<---------------cut here---------------end--------------->8---

The next big effort will be to make this mes-tcc fully functional and
integrate this with GuixSD. To give you a taste of that,
here's latest bug I'm currently looking at (pretty printed comments
are only added when Guile runs MesCC, the problem is in LEA)

--8<---------------cut here---------------start------------->8---
$ diff -u ../mes-seed/mes.M1 src/mes.M1
--- ../mes-seed/mes.M1 2018-05-01 18:49:37.312162270 +0200
+++ src/mes.M1 2018-05-01 19:49:40.774770406 +0200
@@ -35805,12 +33091,11 @@
call32 %strcpy
add____$i8,%esp !0x8
test___%eax,%eax
- # strcpy(buf + strlen(buf), "/mes/");
- push___$i32 &_string_reader_read_list_266
+ push___$i32 &_string_reader_read_list_265
mov____%ebp,%eax
- add____$i32,%eax %0x-200
+ add____$i32,%eax %0x-800
push___%eax
- lea____0x32(%ebp),%eax %0x-200
+ lea____0x32(%ebp),%eax %0x-800
push___%eax
call32 %strlen
add____$i8,%esp !0x4
--8<---------------cut here---------------end--------------->8---

We also need to remove some shortcuts that we took, most notably:
mes-seed[3]. This seed consists of 1MB of M1 code. mes.M1 is
produced by compiling mes.c using MesCC, the C compiler written in
(Guile) Scheme that comes with Mes. Although that's really terrible,
it's probably a big step forward: currently GuixSD uses ~250MB of
binary seed: the bootstrap binaries.

The plan is to replace the mes.M1 seed with mes.M2 and compile this
new mes.M2 seed using the brand new M2-Planet[2]. M2 is basically
simple C with structs, without preprocessor. This will reduce the
seed size by a factor of 10 while making it much more readable.

An excerpt of the TODO I keep in Mes' BOOTSTRAP document

--8<---------------cut here---------------start------------->8---
* TODO
** have tcc-boot's mes-tcc compile a fully functional tcc
*** mescc: fix unknown bug.
*** mescc: support function-static.
*** mescc: support/grok global static.
*** mescc: support unsigned comparison, arithmetic.
*** mescc: support long long (do we need long long to get long long in tcc)?
*** mescc: support bitfield (do we need bitfield to get bitfield in tcc)?
*** mescc: support float (do we need float to get float in tcc)?
** have bootstrapped tcc compile gcc-4.7
** remove or upstream patches from tcc-boot
** prepare src/mes.c for M2-Planet[2] transpiler
** integrate with GuixSD
** x86_64, arm, the Hurd
--8<---------------cut here---------------end--------------->8---

Greetings,
janneke

[0] http://git.savannah.gnu.org/cgit/guix.git/log/?h=wip-bootstrap
[1] https://gitlab.com/janneke/mes
[2] https://github.com/oriansj/m2-planet
[3] https://gitlab.com/janneke/mes-seed

[Ludovic Courtès]

Hello!

Post by Jan Nieuwenhuizen
I've updated the wip-bootstrap branch[0] for Mes[1] 0.13. It has new
mes-boot and tcc-boot packages. mes-boot is a bootstrap version of
Mes; it only depends on mescc-tools and a previously compiled mes.M1
seed. Likewise, tcc-boot depends on a precompiled tcc-seed. Also,
tcc-boot uses a heavily patched version of the tcc sources.

Very nice!

Post by Jan Nieuwenhuizen
Mes 0.13 is the first release that can bootstrap a fairly functional
tcc-boot. This bootstrapped tcc passes 67/68 C tests that were created
for MesCC. It can compile a version if itself where float, long long
and bitfield are patched out...but linking fails. This amazing compiler
can now be played with by doing something like

[...]

Post by Jan Nieuwenhuizen
The next big effort will be to make this mes-tcc fully functional and
integrate this with GuixSD. To give you a taste of that,
here's latest bug I'm currently looking at (pretty printed comments
are only added when Guile runs MesCC, the problem is in LEA)

Ouch. :-)

At some point people were wondering whether using tcc as a base was the
best course of action, after all, given that it needs to be heavily
patched and that MesCC is making good progress. What’s your take on
this now?

Also, does M1 support all the architectures Guix currently supports?

Post by Jan Nieuwenhuizen
mes-seed[3]. This seed consists of 1MB of M1 code. mes.M1 is
produced by compiling mes.c using MesCC, the C compiler written in
(Guile) Scheme that comes with Mes. Although that's really terrible,
it's probably a big step forward: currently GuixSD uses ~250MB of
binary seed: the bootstrap binaries.

Definitely. I think we can improve things incrementally, we don’t have
to go straight to the ultimate solution. So if we can make small
changes to our bootstrap graph and already remove 10MB out of these
250MB, let’s just do that.

Post by Jan Nieuwenhuizen
The plan is to replace the mes.M1 seed with mes.M2 and compile this
new mes.M2 seed using the brand new M2-Planet[2]. M2 is basically
simple C with structs, without preprocessor. This will reduce the
seed size by a factor of 10 while making it much more readable.

Exciting.

Thanks for the status update and for all the great work!

Ludo’.

[Jan Nieuwenhuizen]

Post by Ludovic Courtès

Post by Jan Nieuwenhuizen
I've updated the wip-bootstrap branch[0] for Mes[1] 0.13.

Very nice!

Thanks!

Post by Ludovic Courtès
At some point people were wondering whether using tcc as a base was the
best course of action, after all, given that it needs to be heavily
patched and that MesCC is making good progress. What’s your take on
this now?

Yes, I'm wondering about that quite often. At the reproducible-builds
conference, Ricardo was inspired to try using Mes to bootstrap Guile
instead of tcc. That did really work yet, mainly because Mes didn't
recursively expand macros. Recursive macro expansion was added in Mes
0.12, I'd love to revisit the direct Guile bootstrap path.

At the moment, tcc seems the best of the worst. Tcc is the only
somewhat simple C compile that I know of that can build gcc. I'd be
very happy to hear any alternatives.

Post by Ludovic Courtès
Also, does M1 support all the architectures Guix currently supports?

Currently, Mes only supports x86. The low level tools (hex0, hex1, M0
M1 and M2-Planet) become gradually more architecture independent, with
implementations for x86, amd64, amd and Knight VM.

My first aim is to have a working gcc for x86, a pragmatic choice for
me.

Post by Ludovic Courtès
Definitely. I think we can improve things incrementally, we don’t have
to go straight to the ultimate solution. So if we can make small
changes to our bootstrap graph and already remove 10MB out of these
250MB, let’s just do that.

Yes, that's my strategy...however I do not see how to replace anything
until we have bootstrapped a working Gcc [or Guile?]. So all effort is
focussed on that now.

The past months I've been working to improve Mes Scheme, to make it
a viable option for compiling tcc. Now that's done, I'll be looking
at the tcc and MesCC compiler again.

Post by Ludovic Courtès

Post by Jan Nieuwenhuizen
The plan is to replace the mes.M1 seed with mes.M2

Exciting.

Yes, and it's so much fun over at #bootstrappable. M2-Planet just
got anonymous structs and function pointers -- that should be enough
for mes.c :-)

Greetings,
janneke