Howto speed-compiling in every version of GentooX & MCE

Talk about anything related to Gentoox or Gentoo.
Orpheus
Newbie
Posts: 5
Joined: Wed Apr 06, 2005 1:53 pm

Howto speed-compiling in every version of GentooX & MCE

Post by Orpheus »

Hi folks!

While talking to some Linux gurus last night we figured out that it should be possible to do all the compiling on a PC.
Just install any version of GentooX/MCE and then copy the rootfs over to a PC (one with a fast CPU and much RAM) running Linux (I used an up to date Gentoo with same compiler-version as the GentooX).

YOU WILL DEFINITIVELY NEED AT LEAST A PENTIUM3 COMPATIBLE MACHINE, THAT IS ABLE TO RUN THE BINARIES COMPILED FOR THE XBOX. A PENTIUM3, PENTIUM4 AND ATHLON XP WILL WORK!!!

I guess you need "some kind of other Software" you have to run on the xbox for getting ftp access. :wink:
On the PC mount the rootfs loopback (as root on the PC!)

Code: Select all

 mkdir /mnt/xboxroot
mount -o loop /where/you/copied/the/rootfs /mnt/xboxroot
Then just go ahead and chroot into the Gentoox just mounted.

Code: Select all

chroot /mnt/xboxroot /bin/bash
and update your environment variables:

Code: Select all

env-update
source /etc/profile
now you are logged in as root into your GentooX.
here you can run magic, emerge whatever you want and do all the things that take so much time on the xbox.
JUST REMEMBER: YOU ONLY HAVE 4GB SPACE ON THIS ROOTFS!!!
when you are finished type

Code: Select all

exit
to leave the GentooX environment and umount the Xbox-rootfs

Code: Select all

umount /mnt/xboxroot
now just transfer the rootfs back to your xbox.
Voila! Now you can start GentooX/MCE on the xbox and use all the nice things you have compiled and/or done on the rootfs when you used it on your PC.
ShALLaX
Site Admin
Posts: 1973
Joined: Sun Aug 10, 2003 9:25 pm
Location: England
Contact:

Post by ShALLaX »

Yep, this is certainly do-able... or you could just install Gentoo on your pc then compile the packages with "emerge --build-pkg whatever"... then transfer the package over to the xbox and install it ;p
The original Xbox adaptation of Gentoo
Orpheus
Newbie
Posts: 5
Joined: Wed Apr 06, 2005 1:53 pm

Post by Orpheus »

Yes, you could also do this, but it is quite slow (just my opinion) to install the precompiled packages... atm i can just beam the rootfs over to my workstation, compile all the stuff i want with distcc on 3 PCs REALLY fast and beam it back to the xbox. One transfer take about 9 minutes... This is ok, i think.
Even if i loose 20 Minutes just for transfering the rootfs, it is sooo much faster to compile anything.
Even with using distcc with my 3 2GHz and more PCs KDE-Compilation on the pro edition took more then 25hours, just because the xbox was not able to distribute the compile-jobs and write the comilations to the disk. So every of the 3 Machines was recievong a compile job ever 20-30 seconds, just because the Xbox beeing totally f***ed up with only distributing, not compiling anything by itself...

btw: have you ever thought of making the rootfs larger by doing a raid0 over 2 loopback filesystem? this way you are able to get a larger rootf without needing to do a native installation...

and: sorry... I just came home from playing darts and having one or ten "Koelsch" (Cologne Beer) too much... :oops:
ShALLaX
Site Admin
Posts: 1973
Joined: Sun Aug 10, 2003 9:25 pm
Location: England
Contact:

Post by ShALLaX »

I have thought of doing that, but i was informed that it isnt possible.
The original Xbox adaptation of Gentoo
angrypants
Newbie
Posts: 4
Joined: Fri Aug 06, 2004 12:37 am

Post by angrypants »

Wow, I have been having alot or problems with gentoox. I simply want to unmerge ked and xfce and a few other programs and emerge fluxbox ad mythfront end. I have had alot of problems with things crashing or just taking days to compile. I started from scratch and unmerged kde and xfce and installed fluxbox in about an hour rather than multipule hours. I ftp'd the rootfs back to the xbox and it booted into flux with no problems. I'm emerging mythfrontend right now so we'll see how long that takes. The compile time on my desktop machine is a huge diffrence but it is also nice to be able to use the xbox for playing games and music in the meantime. Thanks for the helpful hint I would have never thought of doing this otherwise.

Update: It took me less than an hour to emerge mythfrontend and it's working like a champ. Gentoox is great but if you have alot of apps to install this is really the way to do it. Now to emerge ettercap and some other fun stuff.
orochi
Gentoox Guru!
Posts: 606
Joined: Sun Oct 24, 2004 8:11 am

Post by orochi »

patience young jedi, the xbox is no supercomputer (tho if i could afford it, id expand my 2 box cluster heh)
plewright
Newbie
Posts: 3
Joined: Sat May 14, 2005 5:07 pm

Post by plewright »

I'll try to give some background before I get to the question, I havn't dealt much with linux kernal, but I downloaded and installed Gentoox Pro, pretty smooth operation so far. Shallax, you've earnt my respect on this. Quite excited about being able to set up some games and educational apps for the kids to play with. They love playing with my laptop with mspaint and typing their name in big letters etc, - learning the basics a 4 yr old can about computers. Anyway, installing linux onto the xbox opens up the opportunity for me to give them all sorts of (free) educational stuff to use with the xbox and TV, so they dont tie up (or muck up) my laptop or pc.

Having quickly realised the limitations of the Xbox hardware for compiling all the stuff, trying to do the "magic" thing, I decided to try and do the trick by using my pc - 3.2GHz 1.0GB beasty, should be a bit quicker. Kinda like what was discussed in this thread.

Just of interest, to do this I happen to be copying the gentoox rootfs to my pc, then mounting it using the coLinux 2.6 kernal as a process running under windows XP. This worked with normal pc-Gentoo, and has been successful so far (today) with gentoox also.

OK, getting to the point, could their be a problem with using a slightly different kernel and hardware than gentoox expects? The gentoox rootfs booted up succuessfully with coLinux kernal, but there were some complaints about hardware this and that, something I noticed was the /dev/i2c/0 error need to be root...

Networking - no problems (shock horror.) Could ping xbox even though it was running inside XP. (In and Out.) Lovely when something goes right.

Anyway, i persisted and started magic, had a lot of "unable to open /dev/i2c/0 errors", but it seemed to keep working despite the problems.

Apart from that seemed to be running and compiling happily for a couple of hours. Then it stopped, error regarding out-of-memory, seen in the coLinux console, the gentoox monitor was frozen in the middle of compiling something, then did ctrl-c and it come back to the magic recieved ctrl-c prompt and recorded it as an error, or something of that nature. (Realised I had only given coLinux/gentoox process max 64mb.) Didn't remember to give it heaps more. Next time I will.

Anyway, do you think there will be more problems caused by i2c etc? Am I wasting my time if anyone else knows that I am heading down the wrong path. I've thought about compiling a special kernal for coLinux, that provides what Gentoox is looking for, but is that necessary?

Is the "env-update" and "source profile" enough to prevent problems?

Or are there some known problems already?

Appreciate you responses, cause it might save me a lot of time.

Thanks.
Christopher
orochi
Gentoox Guru!
Posts: 606
Joined: Sun Oct 24, 2004 8:11 am

Post by orochi »

plewright wrote:I'll try to give some background before I get to the question, I havn't dealt much with linux kernal, but I downloaded and installed Gentoox Pro, pretty smooth operation so far. Shallax, you've earnt my respect on this. Quite excited about being able to set up some games and educational apps for the kids to play with. They love playing with my laptop with mspaint and typing their name in big letters etc, - learning the basics a 4 yr old can about computers. Anyway, installing linux onto the xbox opens up the opportunity for me to give them all sorts of (free) educational stuff to use with the xbox and TV, so they dont tie up (or muck up) my laptop or pc.

Having quickly realised the limitations of the Xbox hardware for compiling all the stuff, trying to do the "magic" thing, I decided to try and do the trick by using my pc - 3.2GHz 1.0GB beasty, should be a bit quicker. Kinda like what was discussed in this thread.

Just of interest, to do this I happen to be copying the gentoox rootfs to my pc, then mounting it using the coLinux 2.6 kernal as a process running under windows XP. This worked with normal pc-Gentoo, and has been successful so far (today) with gentoox also.

OK, getting to the point, could their be a problem with using a slightly different kernel and hardware than gentoox expects? The gentoox rootfs booted up succuessfully with coLinux kernal, but there were some complaints about hardware this and that, something I noticed was the /dev/i2c/0 error need to be root...

Networking - no problems (shock horror.) Could ping xbox even though it was running inside XP. (In and Out.) Lovely when something goes right.

Anyway, i persisted and started magic, had a lot of "unable to open /dev/i2c/0 errors", but it seemed to keep working despite the problems.

Apart from that seemed to be running and compiling happily for a couple of hours. Then it stopped, error regarding out-of-memory, seen in the coLinux console, the gentoox monitor was frozen in the middle of compiling something, then did ctrl-c and it come back to the magic recieved ctrl-c prompt and recorded it as an error, or something of that nature. (Realised I had only given coLinux/gentoox process max 64mb.) Didn't remember to give it heaps more. Next time I will.

Anyway, do you think there will be more problems caused by i2c etc? Am I wasting my time if anyone else knows that I am heading down the wrong path. I've thought about compiling a special kernal for coLinux, that provides what Gentoox is looking for, but is that necessary?

Is the "env-update" and "source profile" enough to prevent problems?

Or are there some known problems already?

Appreciate you responses, cause it might save me a lot of time.

Thanks.

I believe i2c is the onboard sensor
Telek
Linux User
Posts: 15
Joined: Tue May 24, 2005 6:30 am
Contact:

Post by Telek »

I²C is a very simple bus (inter integrated circuit) that requires only 3 wires (clock/data/ground) and can allow several different chips to talk to each other. SMBUS in your PC is somewhat similar to this. The LPC bus in your xbox is similar too.

PCs don't have an I²C bus so any drivers that try to access is will of course fail. I'm not sure exactly what is controlled over this bus, but this should only effect things like the power LED, thermal sensors, etc. All of the main core of the system is just like in a PC. Thus you shouldn't have any problems. I'd suspect that frequent I²C errors could be caused by attempts to control the power led or something like that.
plewright
Newbie
Posts: 3
Joined: Sat May 14, 2005 5:07 pm

Post by plewright »

I understand what you mean, that the I2C is one of the hardware modules suddenly without its hardware, so would normally spit the dummy when it finds itself as a ghost having been ripped from its body, I would spit the dummy too if some higher being separated my body from my head. Spitting the dummy would be about all I could do wouldn't it? At least I would still have a mouth, to use! Just like an I2C module complaining to the terminal. Somehow that draws my thoughts to Hitch Hiker's Guide to the Galaxy!?

So when its spitting the dummy, are you saying that it's NOT related to the "Build Environment"? Is it just a coincidince that it comes up then at the same time? Or is there more to this baby?

The thing that I am worried about, stemming from not having dealt much with the Build Environment, which is used by Magic and Portage. After "mounting" and "chrooting", do the "env-update" and "source profile" commands mentioned by Orpheus, do they accomplish EVERYTHING that is required? Are ALL possible settings for the build environment IDENTICAL under the CoLinux Kernal and PC hardware, as they are on the Gentoox kernal and XBox hardware?

Is there anything used by the build environment that might be different?

Can we trust that the given "profile" covers everything, especially if we are intentionally doing the opposite of what it's meant for, by purposely running it on a different hardware and with a different Kernal? Does Magic/Portage/Emerge look at the profile only???

For example, on my P4, HypedupThreading is noticed when CoLinux starts up, as it outputs "multi-threading" stuff to the screen during start-up. I don't know if this information is found in the same place as for example the "uname" command which tells me that it's a 686 under Gentoo. I havn't yet checked what "uname" reports under the Gentoox rootfs running from the CoLinux kernal because that wouldn't really answer my question because I dont know if that is the same place that the build environement finds what CPU is being used. For all I know, the build process potentially "probes" for info "outside" for info that isn't covered in the "profile."

Continuing with this example scenario, now what happens when the "Magic" system downloads a package that could make use of the multi-threading, e.g. as some compression programs do? Does it mistakingly include it into the package when building it, because it automatically probes and finds a multi-threading CPU, only to consequently crash when later run on my Xbox? Or does it not look passed a certain point, because it is completely 100% restricted within the pre-definded boundary when "env-update" and "source profile" are called?

My current (lack of) understanding limits me to expect one of two answers: either, a, b (or c),

a) DIFFERENT: the build environment could be different, because of x,y,z (cool if you could point me to where to start looking for the list of settings, probings etc.) Perhaps someone could offer some experience or examples that would at least let me know then that I have to spend time learning all about the Build Environment vs the Profile to be 100% satisfied.

b) SAME: the build environment is the same, because ALL POSSIBLE settings are defined in the "Profile", given inside the rootfs, which CAN NOT change automatically. If this is true however, then how does the Profile completely restrict the range of variables that might not have even been thought of, when new packages are developed?

Following from my example above, unfortunately it is most likely covered, but just for the analogy because there are bound to be extra/new features that are created for new Profiles that couldn't possibly be covered by existing profiles. So, given that the zipping Package was prepared with the option to utilise the hyperthreading to be included during the build process, then imagine that the corresponding variable was not defined or even thought of when the Gentoox profile was made, say something like the variable MULTITHRD (I don't know what the real variable name is, or if there is one), but say that is what is looked for in the Profile. So what does the Magic/Emerge/Portage system after not finding in the Profile the value? Does it then try to probe outside, and find out for itself what CPU it there? Or, how does it choose weather or not to use that feature?

One solution I imagine (but have no clue about "Profiles", so don't quote me) is that the potential package must have some sort of interface, that the host Magic/Emerge/Portage system looks at, which defines the complete list of all of the features that it would need to use in the host's Profile when building. The host system could compare its own list of variables and ensure that there are not any unknown ones that the potential package would be looking for. Note also, that users can change/tweak their own build "Profile" so the package system couldn't easily rely on using a controlled pre-defined simple set of expected "Profiles." Unfortunaltey, every single element of the profile would need to be checked, but only then would it be possible to cover everything, ensuring 100% compatibility, and the only way I can think of that ensures there is no need for a package to look beyond the Profile.

c) ANOTHER TOPIC? Has this been covered? Is their somewhere else a discussion of the internals and workings of Magic? and Xbox Profile? Or does this thread continue and follow through, because however much benefit using a faster PC to compile appears to be, moving the build environment to another (different) one raises concerns. (for us yet to be educated anyway.)

I should apologise for the repeatedness of the question presented here, but I reckon this time it might be more clear as to what I was looking for, than from my earlier post.
Christopher
l03wn3
Newbie
Posts: 1
Joined: Thu Aug 04, 2005 2:37 pm

distcc

Post by l03wn3 »

I don't get it, why whould you transfer rootfs to another computer, compile stuff from there with three (other?) computers via distcc, and then transfer back, instead of just compiling from the xbox with distcc and the (now four?) other computers from the beginning?
raid517
Novice
Posts: 40
Joined: Fri Jul 02, 2004 6:00 am

Post by raid517 »

Hi, I tried this method - but after I mount the image, I don't seem to be able to ping anything. (I can't do emerge sync).

Is this because my network settings have somehow changed? If so how do I enable networking?

If config says:

linux1 /

Code: Select all

# ifconfig
Warning: cannot open /proc/net/dev (No such file or directory). Limited output.
Warning: cannot open /proc/net/dev (No such file or directory). Limited output.
lan0      Link encap:Ethernet  HWaddr 00:13:8F:64:13:F3
          inet addr:169.254.150.23  Bcast:169.254.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:6 Base address:0x8800

Warning: cannot open /proc/net/dev (No such file or directory). Limited output.
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1

GJ
dragon_788
Newbie
Posts: 6
Joined: Sun Sep 03, 2006 5:59 pm

Post by dragon_788 »

plewright: as I understand it, magic is ShallaX's custom built updater for Gentoox, that being said it includes specific use flags for the Xbox that restrict the packages to certain use flags and so running it on a PC with HT shouldn't affect it. ShallaX would hafta clear it up if I'm out of line.

l03wn3: the reason for transferrring it to a PC is that the xbox as mentioned earlier in the thread really sucks at controlling distcc compilations.
orochi
Gentoox Guru!
Posts: 606
Joined: Sun Oct 24, 2004 8:11 am

Post by orochi »

raid, this method implies you have a working linux system to then chroot to your rootfs.

in your case the 169....... ip address implies you dont have internet access (or a cable plugged in to recieve DHCP requests.

emerge, magic etc. should all work through a chroot
gnif
Novice
Posts: 42
Joined: Wed Sep 13, 2006 2:07 pm

Post by gnif »

PCs don't have an I2C bus so any drivers that try to access is will of course fail.
Just like to point out that they do infact have a i2c bus... those little chips in the corner of a slice of memory are i2c eeproms that contain the memory timings... also quite often sensor data is tied to the same bus.

You can pickup the i2c bus lines in the computer (easiest access is the memory) and add extra devices to it (extra temp probe for example).
Post Reply