[gobolinux-devel] Symlinking of metarecipes

[Jonas Karlsson]

On Wed, 06 Dec 2006 00:25:10 +0100, Hisham Muhammad <hisham.hm at gmail.com>  
wrote:

> On 12/5/06, Jonas Karlsson <jonka750 at student.liu.se> wrote:
>> On Tue, 05 Dec 2006 23:39:47 +0100, Hisham Muhammad  
>> <hisham.hm at gmail.com>
>> wrote:
>>
>> > On 12/5/06, André Detsch <detsch at gobolinux.org> wrote:
>> >> On 12/5/06, Jonas Karlsson <jonka750 at student.liu.se> wrote:
>> >> > I'm just trying to build Xorg 7.1 and one thing that strikes me is
>> >> that I
>> >> > think that Compile spend more time at symlinking (and running
>> >> ldconfig)
>> >> > after each sub app has compiled then actully compiling the apps.
>> >> Wouldn't
>> >> > it be possible to set PATH and LD_LIBRARY_PATH to include the  
>> actual
>> >> > directory, /Programs/Foo/1.2/bin and /Programs/Foo/1.2/lib
>> >> respectively,
>> >> > during the compilation of a meta recipe, instead of running
>> >> symlinkprogram
>> >> > each time?
>> >>
>> >> Sounds a good idea. Can you implement this and test with the Xorg
>> >> recipe? Please ensure the changes do not affect regular (non-meta)
>> >> recipes.
>> >
>> > Won't work. PATH and LD_LIBRARY_PATH won't be enough. The proto stuff
>> > installs headers. Then you'll have to handle C_INCLUDE_PATH. And if
>> > something uses C++, then CPLUS_INCLUDE_PATH and so on and so on and so
>> > on. It's hackish. And then some other recipe in the future may look
>> > for share/ stuff which won't work unless properly symlinked. Doing
>> > this is asking for meta-recipes to break in mysterious ways.
>> >
>> "and so on and so on and so on."
>> I can only see that there are six variables to take care of (PATH,
>> MANPATH, INFOPATH, LD_LIBRARY_PATH, C_INCLUDE_PATH and  
>> CPLUS_INCLUDE_PATH)
>> of which two have to have two entries bin/sbin and lib/libexec.
>
> No, the set of variables is unbounded: think of PkgConfig, Python,
> etc. Potentially, any program can create an environment variable that
> establishes a path dependency that can be necessary in a later step.
>
I still don't see the problem. The only problem I see with not symlinking  
is that one has to update environmental variables so that the files  
installed (and not symlinked) are found. If an app creates environmental  
variables to resources outside of that dir, I can't see how that would  
break things. Maybe I'm missing something.

>> Of cource shared messes things up, but one can choose only to symlink  
>> that.
>>
>> > Instead, add a new flag "do_symlink=no" and set that on sub-recipes
>> > that you can test and ensure that they don't need to be symlinked in
>> > an intermediate step. This is the safest approach and will result in
>> > the desired speedups. For example, you can skip symlinking on all
>> > proto's and only symlink in the last one, so that when libs are
>> > compiled, the proto headers are linked (yes, this depends on a proper
>> > order for includes in the main recipe, but order is already relevant).
>> >
>> You're funny. :)
>> You want me to go through 200+ recipes one by one and see which don't  
>> need
>> symlinking and which needs it? Perhaps if I got paid ;)
>
> No, I don't. Sorry if I phrased it badly -- I should have said "the
> best approach would be for one to set that on sub-recipes (etc.)". I
> didn't mean to say "go and do it", I meant to say "if one were to do
> it, this is how I think it's the right way to be done".
>
Hehe, it's ok. But "Instead, add a new flag..." sounded like it was  
directed directly at me. :)

> In another email, you suggested for sub-recipes to have 200+
> hand-maintained Dependencies files, while we removed Dependencies
> files from sub-recipes precisely because of the maintenance burden.

I still think that dependencies should be in every sub-recipe. Look at my  
other thread, where one of the components of Xorg 7.1 needed another  
component of Xorg 7.1 (but the latter wasn't installed due to an error by  
me). I would like to be able to update sub apps separatly, when it's  
possible.

The difference, the way I see it, is that dependencies fail once in a  
while, but your idea of add do_symlink=no had the possibility to fail  
200+ -#_of_proto times.

> That's the difference between a "wouldn't it be nice if" idea and
> something that can be managed in practical terms. Working on the Xorg
> recipe set is a huge amount of work and I'm well aware of that -- I
> hold memories of writing custom scripts that read the HTML from
> x.org's directory listings and created the original recipes. André has
> also put a lot of work on them.
Me too - unnecessarily. I accidently used an old version of the Xorg 7.1  
recipe I had lying (I made an effort to make an update whan 7.1 was  
released but something came in the way and then time went by), so I pretty  
much did the Xorg 7.1 recipe manually from 7.0 plus that I did some tweaks  
(which I will post in my new 7.1 version, based on Andrés recipe, after I  
have made a successful build).

Now with the new modular build Xorg has a shellscript that can be used for  
building the complete Xorg and I used that for build ordering, which was  
somewhat different to the order we used.

> Speeding up the compilation, while not
> essential, would be a nice addition, but any tweak made to those
> recipes really is a lot of work. The do_symlink=no feature I suggested
> could be used even if it was not applied to the maximal valid set of
> subrecipes (using it in protos alone could be a noticeable speed-up).
> But like I said, it's a non-essential optimization (and the
> variable-tweaking approach, on the other hand, is error-prone). Your
> general observation that some of the symlinking can be skipped,
> though, remains a valid one.
I'll see what time can bring on do_symlink=no. :)

I had some ideas concerning meta recipes (I don't like the fact that I  
don't have control over the sub packages) and I'll make a new thread about  
that.

-- 
/Jonas

Using Opera's revolutionary e-mail client: http://www.opera.com/mail/