[gobolinux-users] Source repositories and other suggestions

Michael Homer gobo-users-dufus at wotfun.com
Sat Dec 2 21:48:37 UTC 2006


I replied to all three of your emails in here.
On 12/3/06, Martin Baldan <martinobal at gmail.com> wrote:
>
>
> On 12/2/06, Michael Homer <gobo-users-dufus at wotfun.com> wrote:
> >
> > Patching the linker was an either/or with the rest of it; you could do
> > it just by patching the linker with some cleverness (and dubious
> > reliability), or by patching everything else. The problem in the
> > former is in dealing with shared libraries and performing the
> > necessary guesswork; the problem with the latter is in finding the
> > appropriate places to change and in making the changes correctly.
>
> It looks like patching the linker would be cleanest. Can you think of ways
> in which the guesswork could fail? I mean, if foo.so.3 is safe, doen't it
> mean that for all X, foo.so.3.X is also safe?
The main problem guessing that I saw was that it's quite difficult to
tell what program you're actually linking from inside the linker, and
I don't want to be putting that sort of complex logic into it. Doing
it by patching the linker could actually introduce instability, which
pretty much defeats the purpose.
> >
> >
> > I don't think this is logistically possible, because it would involve
> > a high degree of manual intervention (if you think it can be fully
> > automated, go ahead and prove me wrong; I think there's too much
> > variation). The benefit is also absolutely minuscule - even for
> > somebody with the resources to make it happen, it wouldn't be
> > worthwhile. It'd be a lot easier to manage for a fully binary-based
> > distribution of course, and I could see one of the very security-heavy
> > ones doing it for the paranoia value.
>
> Why do you insist that the benefit would be small? It's not just security
> paranoia, shared libraries can be a royal PITA because of compatibility
> issues (minor versions being accidentally incompatible). If minor versioning
> worked so well in practice, things like PBIs would not exist. In Linux,
> installing a new application can (and often does) break other applications
> you installed before. They always say you can choose between the rock-solid
> stability of statically linked libraries, or the memory benefits of shared
> libraries. What if you could have both?
I think it would be small because that very rarely happens when
compiling from source (binary compatibility is easier to break).
There's also not the problem of installing one application breaking
the rest in Gobo because they can't overwrite each other's files at
will, only those of different versions of themselves.

You really don't have huge memory benefits from shared libraries if
every program is linked against different versions.

I don't think the benefit is substantial enough for anybody but one of
the security-heavy distros to bother with the work involved. For
anybody else, how often does a compatible upgrade actually break
things? I'm not sure I've ever had that happen - emerge.log on my
Gentoo system shows me running revdep-rebuild only after upgrading
OpenSSL and after downgrades.
>
> > There may also be problems if a dependent library of Foo is linked
> > against a different version of Bar than Foo is. That means library
> > compatibility couldn't be updated until every dependent (and
> > co-reliant) program was verified as well.
> >
> > To be honest I'm pretty dubious about how the compatibility database
> > would really work, but I'm having difficulty voicing why. I guess it's
> > partly because versions would be marked compatible on an
> > at-first-sight basis. I might have more to say on that subject, but
> > I'll have to give it some more thought.
>
> I'm not sure what you mean here, but notice that every "atom" of
> information that is uploaded to the database would only refer (recursively)
> to real names, not to sonames. I don't know what you mean by "at first
> sight", but in the worst case, it would always be at least as reliable as
> the official minor version criterion, since that would be the maximum
> compatibility allowed in tests.
By "at first sight" I meant that people would certify a version as
working if nothing appeared to be wrong as soon as they ran the
program, while (in the situations where this sort of thing is
necessary) there could be problems below the surface. I only think
it's plausible if there's an actual team of people dedicated to
testing every package and combination; that's why I think it'd be
better suited for a large, binary distribution.
>
> > Again, I don't think this is an impossible or bad idea in itself, I
> > just don't think it's plausible or fitting for Gobolinux. A very
> > security-conscious, high-resourced, binary-based distribution could
> > well want to take it on, if they don't already.
> >
>
> If it can't be automated, clean and optional, in a  very gobo-like spirit, I
> don't want it. It should work as an option in Compile, something like
> choosing between "Compile --realnames appfoo" or "Compile --sonames appfoo"
> and having a global variable to state which is the default behavior. The
> compatibility database would be just an aid for safe update automation, but
> it would be easily overriden by the user without breaking anything (in fact
> that's the way the database would grow).
>
> I will be busy for a few days, but when I have the time, I'd like to look
> further into the matter, read more about this, ask a few more questions and
> then ask for something like this as an optional feature, unobstrusive with
> the general Gobolinux workings.
If you can manage patches to do it, I doubt they'd be rejected. I
don't think automatic is really possible though, and anything less
probably wouldn't make it in.
On 12/3/06, Martin Baldan <martinobal at gmail.com> wrote:
> On 12/2/06, Michael Homer <gobo-users-dufus at wotfun.com> wrote:
> >
> > There may also be problems if a dependent library of Foo is linked
> > against a different version of Bar than Foo is. That means library
> > compatibility couldn't be updated until every dependent (and
> > co-reliant) program was verified as well.
> I think I've just seen this point of yours, and the answer is that, when a
> library itself depends on other libraries, its real name is not enough for
> compatibility purposes. You also need to know what it has been linked
> against in your system. This information should be stored locally and
> uploaded to the database as well. So, if a library's real name is
> libfoo.so.3.1 , then we could say that its "recursively real name" is
> something like libfoo.so.3.1-linkedagainst-libbar.so.4.2-
> -linkedagainst-libwatever.so.5.3. Knowing the recursively real names of
> every library an application has been tested with should be safe.
Yes. The database will grow pretty quickly, though, and of course all
this detail reduces the chances of a "hit" for somebody installing a
program. A lot of the time you'd breaking new ground a tiny bit
because you had slightly different versions somewhere far down the
line.

On 12/3/06, Martin Baldan <martinobal at gmail.com> wrote:
> How about this:
>
> We leave the linking policy as it is now (the linker links against sonames,
> each soname points to a real name) BUT there a table in the system that maps
> real names to recursively-real names. This table is updated each time you
> compile, recompile or relink something. When you want to update a library,
> the system takes the real name of the proposed new minor version, it looks
> up the corresponding R-R name (generated during compilation of the new
> minor vesion library) and then checks in the compatibility database that the
> new R-R name has been tested with all the applications currently installed
> in your system. If not, you may ask which applications might break, and then
> decide what to do. If you compile and run and get no problems, that's more
> info for the online database. It wouln't be as slow as it sounds, since most
> of the times this information would be cached by the database ("are all
> currently installed apps tested-compatible with this R-R name? yes/no").
That's basically what I suggested before, shifting the logic into the
compile/install stage. It's both plausible and possible, and I don't
see any reason you couldn't do it. There are still a few potential
problems, but they're not technical ones now: that online database is
going to be pretty big, and grow pretty quickly as people upgrade, and
the system only works if enough people (a critical mass) bother to
mark versions as compatible. That can't really be automated unless you
install wrappers for everything.
-Michael


More information about the gobolinux-users mailing list