[gobolinux-devel] USE flags & Xorg

[Michael Homer]

On 5/5/07, Jonas Karlsson <jonka750 at student.liu.se> wrote:
> On Fri, 04 May 2007 21:42:14 +0200, Hisham Muhammad <hisham.hm at gmail.com>
> wrote:
>
> > On 5/3/07, Michael Homer <gobo-users-dufus at wotfun.com> wrote:
> >> On 5/4/07, hisham.hm at gmail.com <hisham.hm at gmail.com> wrote:
> >> > On 5/3/07, Michael Homer <gobo-users-dufus at wotfun.com> wrote:
> >> > > > My main comment is to reiterate that USE flags are welcome. I
> >> have an
> >> > > > abstract sentiment that part of it could be automated with
> >> optional
> >> > > > dependencies (ie, "don't build the GTK+ interface if I don't have
> >> GTK+
> >> > > > installed") but I don't have a concrete image of how everything
> >> would
> >> > > > fit together implementation-wise.
> >> > > That would be useful too - but it doesn't help for hardware-required
> >> > > programs (unless you examine /proc/config.gz really closely, and
> >> > > that's likely to be difficult to do flawlessly).
> >> >
> >> > I didn't think about going this far; I agree it's probably not worth
> >> the effort.
> >> >
> >> > > One component of my original idea was to have installed programs
> >> > > automatically become enabled flags, which would have that effect.
> >> >
> >> > That's basically what I had in mind too.
> >> >
> I can see the use of this, but I don't agree. This should not be flags,
> but automatic settings. If an application optionally depend on say gtk+ to
> be built with gui and I choose not to install it when Compile ask me, this
> setting should be passed to configure.
>
> Flags should be more generic and be of value to the user. For example a
> '--with-gtk' flag would automatically install gtk+ for me (if I didn't
> have it installed) in the example above. The gui setting for the
> application I want to install has little to do with flags. It's just a
> "symptom".
>
> I don't think the problem lies in if I want to build with gui or not, but
> to me the issue is if I want to have gtk+ installed or not, hence the flag
> should be about that, not the configure options.
The flag would be "GTK", not "GUI", since one program could have
several different GUI options. So it would be basically like you
describe, I imagine.

It would still be necessary to have a way of setting independent flags
to cover the original case I brought up: there's no "i810" or "Cyrus"
program to look for to see whether to compile support for the
appropriate video card.
> >> > > There would still need to be another, parallel system to deal with
> >> the
> >> > > other situations, and a way to disable the flags created this way. I
> >> > > don't know whether that would be more trouble than it's worth; it
> >> > > might be confusing to have behaviour changes because of an unrelated
> >> > > install.
> >> >
> >> > Right.
> >> >
> Flags shouldn't be about configure options, therefore this isn't needed.
> In the example above, is it really important to be able to specify if I
> want to build with gui or not, if I already have gtk+ installed? And if we
> design this right, we don't have to use these micro controlling mechanisms.
I think it is, yes. Especially when there are multiple options.
Ideally as much of it as possible would happen by magic for the common
case, so you wouldn't actually have to do anything.
> >> > > It might also be necessary to allow per-program flags (as in,
> >> "enable
> >> > > the GTK+ interface to Foo, but don't build the Qt one, even thought
> >> I
> >> > > have both installed"). I don't know how would be best to go about
> >> >
> I can't see why this is useful.
Compile time? Mutual exclusivity? It's not necessarily the case that
something wouldn't -work- because it you couldn't do that, but there
is utility in it.
> >> > A flag to compile overriding the system useflags? As in
> >> > Compile foobar --with gtk --without qt
> >> > where --with and --without are lists of useflags.
> >> I'd prefer things to be automatic as far as possible, so that things
> >> don't end up broken because you forgot to use the same flags this
> >> time.
> >
> > Yeah. I was thinking about the case when a user wants to do a one-off
> > change to their usual set of flags.
> >
> We still have '--configure-options', which I think we should save
> somewhere, when the user specifies his/her own settings for an application.
>
> Flags should be system wide, like '--use-gtk', '--prefer-gnutls' (over
> OpenSSL) etc. For controlling applications '--configure-options' should be
> used.
Broadly, yes, but I would still like things to be automatic wherever
possible, rather than relying on people doing it the same way manually
every time. People will forget, and it'd be a pain to have to look
them up each time.
> >> > > that. Gentoo uses text files; we might prefer a filesystem-based
> >> > > approach. I would quite like to have some way to find out which
> >> > > programs were compiled with which flags, in both directions. That
> >> >
> >> > The set of used flags at the time of compilation could be stored in a
> >> > Resources/ file.
> >> > But what do you mean by both directions?
> >> In the sense that it would be simple to get a list of everything
> >> compiled with flag X on, so you could find out what to recompile if
> >> you turned it off. You could just search through all the Resources
> >> files, but that's not really reasonable when there are hundreds of
> >> them. Of course, it could just be that that's the domain of another
> >> tool and it really does work that way.
> >
> > I see. Anyway, scanning 200 Resources files is not *that* much work,
> > anyway. Tools could do that, and with a little caching it would work
> > nicely.
> >
> Resource file is ok, but to provide readability, shouldn't configure
> options used be there as well, not just flags. After all, users shouldn't
> have to use tools to find out information about an application in
> GoboLinux.
> >> > > could be symlinks, I'm not sure whether it'd ever actually be useful
> >> > > to traverse a tree like that.
> >> > >
> >> > > Post-014 I would like to put some work into getting something like
> >> > > this up and running; the specifics of which method would be best I
> >> > > don't know. My original idea had dependency files like `Flag_On i810
> >> > > && require XF86-Video-i810 1.7.4`, and similar functions for adding
> >> > > configure parameters in the recipes, but I think that might get a
> >> > > little ugly for something complicated. It's probably necessary on
> >> the
> >> > > recipes' side, since things could get arbitrarily complicated there.
> >> >
> >> > A number of proposed implementations floated around in the list over
> >> > the years (IIRC, one from Carlo, one from Andreas Koehler, and even
> >> > one mine if I'm not mistaken) and they were all variations on this
> >> > basic theme. Right now I think for Compile the cleanest would be to
> >> > have "sections" (actually shell functions) like this:
> >> >
> >> > using_gtk() {
> >> >    add_flag configure_options "--with-gtk"
> >> >    something_else="foo"
> >> > }
> >> > not_using_ssl() {
> >> >    add_flag configure_options "--without-ssl"
> >> > }
> >> >
> >> > We could have smart functions to cover common cases such as "when
> >> > using gtk, add --with-gtk, but remove it otherwise":
> >> >
> >> > when_using_gtk configure_options "--with-gtk"
> >> That is very slick. I like it. What about situations that require
> >> testing multiple flags?
> >
> > I think we could come up with these features as the needs arise, but
> > with some general functions basically everything could be solved
> > programatically. I think functions like described above would be
> > prettier but something like:
> >
> > if using ssl && not_using kerberos
> > then
> >    something=foo
> > fi
> >
> > would work too. (The idea of recipe files degenerating into shell
> > scripts is a painful one though. I'd rather have shell code limited to
> > the interior of functions, like
> >
> > using_ssl() {
> >    not_using kerberos && something=foo
> > }
> >
> > to keep recipes more "statically verifiable" (in other words, to make
> > RecipeLint's life easier).
> >
> I agree on keeping recipes static. There should be arrays, like we use now
> with for example configure_options, in the recipes instead of functions.
> The these should be parsed by Compile.
>
> with_gtk=(
>     '--build-ui'
> )
>
> with_openssl=(
>     '--enable-foo'
>     '--with-bar'
> )
>
> without_kerberos=(
>     '--without-bar'
> )
>
> As configure options can be stacked, and the last one defined will be the
> one that is used, this should work. Anything within 'without_foo' will be
> parsed after 'with_foo' and will take precedence.
I prefer the other way, myself. It seems tidier, and it's more powerful.
> >> It's not really plausible to have functions
> >> for every combination, and you can't nest them. I suppose the smart
> >> functions cover the common case even there, so it might not be worth
> >> worrying about. I am thinking of complex programs here, like PHP (107
> >> flags in portage, many of which are codependent).
> >
> > One fear I always had about use flags is that they could make recipes
> > more complicated and less maintainable... I'd rather add flags only
> > where people need them, and not map all ./configure options of an app
> > in a recipe "just because". This is more of a policy/guideline than a
> > technical issue, but I believe that even with use flags we should
> > strive to keep recipes as small as we can. ("Somebody might want to
> > enable xyz" is not a good enough argument to me.)
> >
I agree. A lot of those flags are redundant (mysql vs mysqli). It's
still plausible that something could get dozens of them legitimately,
although it's not going to be too common. PHP is unusually
complicated.
> As stated above, flags should be system wide settings. For enabling "just
> that feature" we have '--configure-options'.

> >> Dependency files are still a complication though, unless they're
> >> changed into a shell-sourceable format. Ideally it would be compatible
> >> with existing files though, so I guess that means using a comment. "#
> >> :gtk" for "when gtk is enabled"? That would let actual comments still
> >> exist, if they're useful to have.
> >
> > Dependency files are currently parsed by Python code and it already
> > uses a custom syntax (with operators, etc.) -- extending it to put
> > flag information, annotation such as "optional", etc., is doable.
> >
> optional/recommended/required is really needed. And the idea of adding
> options/changing status of a dependency depending on flags is a good idea,
> the question is how to implement it. Perhaps 'optional :with_gtk' to have
> it enabled when 'with_gtk' is used, or even more fine grained: 'optional
> with_openssl:recommended with_gnutls:disabled' to have different status
> depending on different flags.
I would just go with flags enabling/disabling things themselves, and
have everything enabled be mandatory. Unless there's a common case
where you'd actually want to ignore a dependency even though you had
the flag for it enabled. And if there is, per-program disabling would
do the same thing for you.
> >> I guess that this would also make all enabled dependencies mandatory,
> >> which was another issue that came up a while back.
> >
> > Looking back, having dependencies accepted/rejected by the user was a
> > primitive way of having something like use flags. I'd still like to
> > have a power-user-friendly "force" mode, but the default behavior
> > could switch to not compile/install if dependencies can't be matched.
> >
> >> > > I also don't know how this would interplay with binary packages. So
> >> >
> >> > I think we should go the simple route there: for binary packages,
> >> > define a fixed set of use flags, and use them consistently in all
> >> > packages from the repository. If people want to tweak their system at
> >> > use flag level, then they should follow the ways of the source.
> >> Yes. But that does mean people will end up having to have all-binary
> >> or all-source systems, since there's no way to guarantee they'll be
> >> compatible. Gentoo does it that way; there are binary packages you can
> >> install provided you haven't changed your USE flags or compiled
> >> anything since initial installation. They would be useful as far as
> >> providing things that don't fit on the CD, but not much further.
> >
> > Somewhat. Turns out that dependency chains between packages tend to
> > form "clusters", and often a user is interested in tweak part of their
> > system, but not the entire thing. For example, I might want to
> > recompile all my audio apps to use Jack, low-latency, etc., but I'd
> > still use GCC from the binary package. When using a package as a
> > dependency, comparing their flags to the flags being used can serve as
> > a good hint for compatibility. We might need to specify flags in
> > dependency files ("I need a libfoo compiled with the bar flag").
> >
> I can't see why binaries should be that much problem. As I said above,
> both flags and options passed to configure should be saved somewhere.
> Building all my systems from source I have only found very few
> applications that needed something enabled in another application, which
> wasn't enabled by default. If we make enough stable default settings
> almost everything should play nicely and users shouldn't have to change
> anything, just like today (what will be different from today?).
> For users that like to play with the flags I think that adding used flags
> to the BuildDependencies and then compare them with the used flags for the
> applications is a good way to solve that problem.
It wouldn't be as much of an issue as for Gentoo, I imagine, but it
does come up occasionally. You genuinely can't install the binary
packages after you change things around, though. Hisham makes a good
point about the clusters, though, so it wouldn't be as drastic as I
thought. So OpenOffice packages would probably still be fine no matter
what, for example.

So from what's been said so far, I think it should be possible to
build a system that is simple for the large majority of cases without
cluttering things too much, or losing functionality from binary
packages and such. I like the system Hisham put forward for it.
-Michael