[gobolinux-users] Re: Compiling Glibc 2.3.6

[Sean E. Russell]

On Saturday 13 May 2006 00:17, Carlo Calica wrote:
> First off, I'd like to say any failure to build/install Glibc from
> recipes/packages is a problem.  ChrootCompile looks to be the best
> solution.

Am I understanding that there are two, separate, unrelated problems?  One is 
compiling glibc itself, and the other is breakages due to a glibc upgrade?

> On 5/12/06, Sean E. Russell <gobo at ser1.net> wrote:
> > Ok.  I'm an end-user, and my opinion is that, no matter what, I don't
> > want to upgrade something and have it make my computer entirely unusable,
...
> Good baseline :-).  Now how to accomplish it in an imperfect world.
> In fairness, a rescue disk isn't too extreme.  The install media
> should qualify.

I can't believe that resorting to a rescue disk is ever a good solution to 
something that is foreseeable and occurs during normal system maintenance.  
The user hasn't done something wrong, or mis-configured something; there's 
been no hardware failure.

I'm not opposed to rescue disks; I just don't think that they should be a 
standard tool for upgrading software.

> > My *preferred* solution, as I've mentioned elsewhere, is some sort of
> > magical mechanism by which I can have numerous versions of glibc and be
> > (mostly) ignorant about what links to which version.
>
> Not asking for much :-).

Yeah, I think that'd require a different link loader.  I don't really expect 
that to happen -- but it would be nice.

> ChrootCompile should provide a sandboxed build.  Then it is a package
> install which "should" be easy.  A "who uses" script is easy.
> Something like "fgrep $package /P/*/*/R/Dependencies".  Then scan
> those package through ldd to narrowdown the version.

This sounds pretty reasonable.  Will there be a mechanism in the recipe to 
keep people from accidentally installing glibc the "wrong" way?

> > I can see a broader use for this.  Gentoo recently had a lot of problems
> > with a libxml upgrade, which broke -- well, almost everything, but
...
> Wow.  That sounds like an ABI breakage.  Here's the problem:

I don't know; I think it was more along the lines that libxml wasn't a slotted 
package, so the upgrade installed libxml.so.1 and deleted libxml.so.0, which 
everything linked to.  Different, and more easily solvable, problem than an 
ABI issue.

> Foo > depends > Bar > depends > Baz.
>
> Baz changes ABI.  So Bar gets rebuilt.  Does Bar's ABI change?  The
> closer to the root of the dep tree to more things need to be rebuilt.
> For glibc it is the root.  I'm not sure how a "who_uses" would help.
> The deps tree is already known.

Hm. There's also the toolchain problem with glibc, which we've talked about.  
If the deps tree is already known, and is being *used* to maintain a working 
state, then upgrading glibc shouldn't be a problem -- as long as you can work 
around the toolchain problem.  Is that accurate?

> > Will they?  I've got some 693 packages installed on my Gentoo laptop, and
> > I certainly don't keep track of most of them.  I was completely sidelined
> > by the libxml issue, and was only saved from a glibc disaster by the
> > pitiful cries of others who, too eagerly, ate from the forbidden fruit.
>
> Why did you upgrade libxml?  Was it a critical fix or "oh, new libxml,
> gotta get it".  I'd argue Gentoo is a bleeding edge distro (at least
> how most users use it) which is prone to those issues.

No, it wasn't an explicit upgrade.  I upgraded something like Inkscape, which 
uses libxml, so the new libxml got pulled in through the dependency 
resolution.

When I do a *system* upgrade -- those other machines I was talking about -- I 
generally use "emerge world", which upgrades any packages for which there are 
new releases.  This is generally a good thing, because it captures bug and 
security fixes on a lot of packages, and it is more or less easy to do with 
Gentoo.  However, it doesn't recompile *everything*, so things like the 
libxml snafu occur occasionally.

The current Gentoo trend is to do modular builds -- KDE is broken up into a 
hundred packages, Xorg is broken up into a hundred packages -- so even 
without a system-wide upgrade, upgrading one of these often pulls in a lot of 
dependencies, some of which may break other software on the system.  There's 
no warning that the libxml upgrade is going to bork other things on the 
system; unless the person is a Linux enthusiast, they probably won't even 
realize that a seemingly *minor* upgrade to glibc (it didn't go to glibc 4.0, 
or something) would case a serious problem.

> I see the situation as a compromise between a "version mix n match"
> and "managed, QA'd deployment".

By managed, you mean bundled system upgrades?

> For a version mix'n match, Compile works well now.  It is impossible
> to have ABI stability so don't try.  Dangerous recipes should be
> labeled "expert_only".

You probably have more knowledge in this area than I, so excuse me if I don't 
understand why managed and mix-n-match are so different.  I think that the 
impact of ABI breakages can be managed with 

Somebody writes a new recipe. Presumably, that person is actually paying 
attention to the package that they're writing the recipe for, and presumably 
they'll be reading changelogs and will be aware of ABI breakages.  In that 
case, they should be able to set a flag in the recipe that says: BREAKS_ABI.  
Since we track dependencies, an upgrade to that package could/should trigger 
a re-compile of all packages that depend on that new version, and this should 
cascade to all of their dependants.  And this should be calculated, and the 
user should be warned about this and asked if they want to continue.

Yes, some ABI breakages will slip through, either through mistakes by recipe 
writers or by upstream changes which fail to warn about breakages, but I do 
think that the problem can be minimized, and that this would catch most of 
the cases.  Where it fails most grossly is when the BREAKS_ABI flag is in, 
say, version 4.0, and the user is going from 3.5 -> 4.1.  A possible solution 
for this to to include in the recipe a _sparse_ history of every revision 
that breaks the ABI.  Compile could then do a simple check to see if the 
version delta includes one of these breakages.  Users who write manage their 
own recipes will be no worse off, and users who rely entirely on "official" 
recipes wouldn't have to worry about ABI issues.  As much.

This is a lot of work to do, although I think it's much more do-able than 
rewriting Linux's link loader.  I'm aware that I'm being one of those people 
who are doing a lot of demanding, and not significantly contributing, and I 
apologize for that.

-- 
### SER   
### Deutsch|Esperanto|Francaise|Linux|XML|Java|Ruby|Aikido|Iaido
### http://www.ser1.net  jabber.com:ser  ICQ:83578737 
### GPG: http://www.ser1.net/Security/ser_public.gpg