Python applications that are also libraries

[Ricardo Wurmus]

Hi Guix,

we have a bunch of packages that are used both as applications and as
Python libraries. An example is “deeptools”.

As a library we need to propagate other Python inputs; as an application
this is not necessary because we have wrappers.

I wonder how to deal with this. Should we assume that these packages
are used as libraries and default to propagating all Python inputs? Or
should we have package variants (or outputs?) that propagate inputs as a
side-effect?

--
Ricardo

[Hartmut Goebel]

Post by Ricardo Wurmus
I wonder how to deal with this. Should we assume that these packages
are used as libraries and default to propagating all Python inputs? Or
should we have package variants (or outputs?) that propagate inputs as a
side-effect?

If this is a "pure" application, I'd install it with*out* propagated
inputs. This might not be easy to determine, though.

https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html

[Ricardo Wurmus]

Post by Hartmut Goebel

Post by Ricardo Wurmus
I wonder how to deal with this. Should we assume that these packages
are used as libraries and default to propagating all Python inputs? Or
should we have package variants (or outputs?) that propagate inputs as a
side-effect?

If this is a "pure" application, I'd install it with*out* propagated
inputs. This might not be easy to determine, though.

It is both. It is often used just as an application, but the procedures
that make up the application are just as often used in an interactive
Python session or as a library.

Post by Hartmut Goebel
https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html

This is about wrapping (or not) using virtual envs. I don’t really see
how this relates to this problem, but maybe I’m missing something
obvious.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net

[Hartmut Goebel]

Post by Ricardo Wurmus

Post by Hartmut Goebel
If this is a "pure" application, I'd install it with*out* propagated
inputs. This might not be easy to determine, though.

It is both. It is often used just as an application, but the procedures
that make up the application are just as often used in an interactive
Python session or as a library.

So I'm afraid, we need to install it with propagated inputs.

Stimulated by your question I rethought whether we might come around
propagated inputs, and I did not find a solution. There must only be one
version of a library in each profile, otherwise we'd get conflicts. We
could provide our own implementation of site.py (or site-customize.py)
to avoid *propagating*, but this would not avoid the *conflicts* but
only hide the cause of the conflicts and make them hard to find (as we
already discusses a year or two ago).

Post by Ricardo Wurmus

Post by Hartmut Goebel
https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html

This is about wrapping (or not) using virtual envs. I don’t really see
how this relates to this problem, but maybe I’m missing something
obvious.

Sorry for the confusion, this link shouldn't have been there. I had
pasted it in since I thought it is related and I'm going to refer to is,
but it is not.

[Chris Marusich]

Post by Ricardo Wurmus
we have a bunch of packages that are used both as applications and as
Python libraries. An example is “deeptools”.

I took a brief peek at deeptools. It looks like there are programs in
bin, and libraries in lib. Why can't we just split them into two
outputs? For example, put the libraries into the default "out" output
and the programs into the "bin" output.

[Hartmut Goebel]

Post by Chris Marusich
Why can't we just split them into two
outputs? For example, put the libraries into the default "out" output
and the programs into the "bin" output.

For consistence, we should then do this for all other python packages
including a script, e.g. sphinx and a lot others.

[Ricardo Wurmus]

Post by Chris Marusich

Post by Ricardo Wurmus
we have a bunch of packages that are used both as applications and as
Python libraries. An example is “deeptools”.

I took a brief peek at deeptools. It looks like there are programs in
bin, and libraries in lib. Why can't we just split them into two
outputs? For example, put the libraries into the default "out" output
and the programs into the "bin" output.

The programs in “bin” use the libraries in “lib”. The binaries aren’t
big, so separating them from the lib output provides hardly any benefit.

The point here is that in the library case inputs must be propagated,
whereas in the case where only the executables are used the inputs don’t
have to be propagated.

Separating them in two outputs doesn’t address this problem as far as I
can see.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net

[Hartmut Goebel]

Post by Ricardo Wurmus
The point here is that in the library case inputs must be propagated,
whereas in the case where only the executables are used the inputs don’t
have to be propagated.
Separating them in two outputs doesn’t address this problem as far as I
can see.

Using the venv-based wrapper I describe in
<https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html
could avoid propagating for the scripts - but not for the libs.

IMHO it does not make much sense having "script" packages without
propagation, as we still need "libs" packages with propagation. Even if
"deeptools:script" would not propagate its inputs, "somepackage:lib"
(required by "deeptools:script") would still need to propagate its
inputs. Thus nothing gained.