IRC logs for #baserock for Thursday, 2014-07-24

*** fay [~fay@82-68-191-81.dsl.posilan.com] has joined #baserock07:02
fay is now known as Guest1171907:02
*** tiagogomes [~tiagogome@access.ducie-dc1.codethink.co.uk] has joined #baserock07:12
*** tiagogomes [~tiagogome@access.ducie-dc1.codethink.co.uk] has quit [Remote host closed the connection]07:59
*** tiagogomes [~tiagogome@access.ducie-dc1.codethink.co.uk] has joined #baserock08:00
*** flatmush [~flatmush@82-68-191-81.dsl.posilan.com] has quit [Ping timeout: 240 seconds]08:26
*** jonathanmaw [~jonathanm@82-68-191-81.dsl.posilan.com] has joined #baserock08:29
*** flatmush [~flatmush@82-68-191-81.dsl.posilan.com] has joined #baserock08:29
*** ssam2 [~ssam2@82-68-191-81.dsl.posilan.com] has joined #baserock08:53
liw-orcpedroalvarez, there's potentially a lot to configure in a Trove -- would it make sense to be able to provide all of lorry-controller.conf instead of breaking it into many small variables?09:42
persiaThat makes more sense to me: reduced comprehension overhead for the user.09:42
*** Guest11719 [~fay@82-68-191-81.dsl.posilan.com] has quit [Quit: Leaving]09:45
*** fay [~fay@82-68-191-81.dsl.posilan.com] has joined #baserock09:45
pedroalvarezliw-orc: no sur about how to implement your suggestion09:45
*** fay [~fay@82-68-191-81.dsl.posilan.com] has quit [Client Quit]09:45
*** Guest76659 [~fay@82-68-191-81.dsl.posilan.com] has joined #baserock09:45
*** Guest76659 [~fay@82-68-191-81.dsl.posilan.com] has quit [Client Quit]09:45
liw-orcpedroalvarez, I'd sugest something like TROVE_LORRY_CONTROLLER_CONFIG=path/to/lorry-controller.conf09:45
liw-orcand if that's not given, use a default file instead09:46
liw-orc(a default that uses http to mirror git.baserock.org, to avoid having to set up ssh keys and things; alternatively, a default that doesn't mirror anything)09:47
pedroalvarezthat change will make more complicated the things I've made simple :/09:50
* pedroalvarez ponders adding a cloud-devel system to the release cluster10:52
pedroalvarezcloud= wih initramfs and cloud-init enabled10:53
persiaMaybe a trove?10:56
richard_mawhmm, I'm tempted to suggest we make the normal systems cloud compatible10:56
persiacloud-init being configured can ease getting it started in libvirt environments also10:56
persiarichard_maw: The problem with enabling cloud-init by default is that it does odd things for folk not working in virtual or highly-networked environments.10:57
pedroalvarezindeed10:57
richard_mawand the initramfs means you don't need to hunt for the OS type without a virtio rootfs when importing a VM into virt-manager10:57
ssam2pedroalvarez: would that not be a cluster morph rather than a system ?10:58
pedroalvarezagree with using initramfs in all of them10:58
pedroalvarezssam2: yes, I meant, add an entry in the release cluster morph10:58
ssam2pedroalvarez: ah, OK. Certainly it's worth adding an example, I think10:58
ssam2i.e. something we could easily paste into release.morph if we wanted to at a later date10:59
* pedroalvarez nods11:00
pedroalvarezpersia: yes, I also want to add the trove to the cluster :)11:01
persiarichard_maw: Using an initramfs by default seems sensible that change needn't get stuck in the cloud-init discussions.11:01
persiapedroalvarez: So add a cloud-trove to the release set.  It's independent from the others, so not likely to cause issues, and it makes it easy for folk to try a local trove, without lots of fussy setup.11:02
persia(plus having a trove in the release set makes it easier to test if a given change in definitions causes behaviour changes for the associated trove).11:03
pedroalvarezthis is my proposal: http://pastebin.com/uUcq3sSE11:21
persiapedroalvarez: I thought you had proposed putting something in release.morph11:42
pedroalvarezsince I don't know the consecuences of adding things in the release.morph, I decided to add them in an example for now11:43
persiaI think you're running into the same duplication trap as the potential for creating the cloud-specific devel system.12:03
persiaIf you're uncertain, ask for someone who does releases to give an opinion here, or if none are about, ask on the mailing list.12:04
persiaIf there is any negative impact from adding directly to release.morph, this will be discussed in review, and then it can be split out.12:04
persiaIf you split it out, the review goes quickly, but then we have to do it all over again with a new review of a patch series to add to release.12:04
ssam2I disagree. The second review would be super quick, if it simply involves moving a definition that already exists and has been used into a different cluster morph12:06
ssam2The first review would also be super quick, since it can't possibly break anything that exists12:07
ssam2I don't know if patch review is the best way to discuss what we should release12:07
ssam2it might be, I guess12:07
persiaEasier to discuss a concrete patch than some vague commentary (all of my email aside)12:12
pedroalvarezthen I'm going to send a patch for what I really want to happen12:22
persiaThat's usually the best plan :)12:27
*** fay [~fay@82-68-191-81.dsl.posilan.com] has joined #baserock12:28
*** fay [~fay@82-68-191-81.dsl.posilan.com] has quit [Client Quit]12:28
pedroalvarezI found a problem with baserock and volumes creation in openstack. I need to reboot the system to mount a volume :/13:40
pedroalvarezcan be solved with CONFIG_HOTPLUG?13:41
pedroalvarez(kernel)13:41
pedroalvarezor it's something more complicated?13:42
richard_mawpedroalvarez: to mount the volume?13:58
richard_mawso, newly added hardware in OpenStack isn't showing up immediately, or it's not auto-mounting?13:59
pedroalvarezrichard_maw: the fomrer13:59
pedroalvarezs/fomer/former/13:59
richard_mawpedroalvarez: hm, I'm not sure what normal behaviour for this is, since I could add a USB attached storage to a local VM, but virtio didn't show up the first time I tried it, and the second time it failed to attach14:05
persiaDoes it work in cirros?14:07
pedroalvarezpersia: yes14:07
persiaHmm.  Complicated.14:08
persiaAnd with Baserock, nothing in the system log?14:08
pedroalvareznope :/14:13
liw-orcI suspect that pedroalvarez's analysis is correct and that our kernel doesn't notice new (virtual) hard disks via virtio or SATA, after boot, but that is just a guess14:14
doffmI'm having trouble building trove-system-x86:14:17
doffmERROR: Failed to update cached version of repo git://git.baserock.org/delta/luxio14:17
KinnisonInteresting.  Can you get some more log details and pastebin them?14:18
* pedroalvarez waits the log, and assumes is not a prblem with the trove itself14:19
liw-orcalso, details of the host (which release of Baserock and which version of morph?)14:19
persiaIt's more complicated than CONFIG_HOTPLUG though: that seems not to even exist anymore.14:24
jjardonI think this would be on interest here (just made the GNOME/Wayland smoke test succeeded all the way to taking a screenshot of the session) http://blogs.gnome.org/mclasen/2014/07/23/continuous-testing-and-wayland/14:30
liw-orcjjardon, nice14:32
jjardoncurrent image here if someone fancy a try ;) http://build.gnome.org/continuous/buildmaster/images/z/current/14:32
pedroalvarezjjardon: using baserock? :)14:35
jjardonpedroalvarez: nope (yet ;)) the current instance of build.gnome.org uses ostree14:37
rjekSadface: http://pastebin.com/zdGdRsV214:39
rjek(During stage 3 native)14:39
* rjek wonders why it doesn't work now.14:40
pedroalvarezSee `config.log' for more details.14:40
pedroalvarezanything there?14:41
rjekThere are dozens of config.logs; just looking through them14:42
rjek(not helped that the environment's 'less' appears to be cat)14:42
persiarjek: You want /gcc.inst/gcc.build/o/config.log14:43
pedroalvarezCould be the -j2?14:43
rjekpersia: It worked fine with the -j2 before.14:43
liw-orcrjek, that sounds like busybox's less, alas14:43
rjekThe only real difference here is that this is gcc 4.7 not 4.6.14:43
rjek(I imagine)14:43
Kinnisonrjek: I'd start by dropping back to not -j2 for gcc just in case14:46
rjekWe'll have to wait until tomorrow, then.14:54
robtaylorrjek: can you paste the config.log persia metnioned?14:54
rjekNot easily, no.14:55
robtaylorboh :(14:56
rjekAnd when I tailed that one, it looked like it existed with success14:56
persiaclearly pastebinit or equivalent should be part of the devel system14:56
persiaYes, it seems to have configured sucessfully, but incorrectly.14:56
Kinnisonrjek: unfortunately the bit you really need in config.log is usually nowhere near the end of it :-(14:57
Kinnisonrjek: If you're currently mid-bootstrap it's possible the only tool to get content off your device usefully might be the busybox built-in nc14:57
Kinnisonrjek: if you check for .../o/libgcc/config.log and there is one, that might be the one we need14:58
* Kinnison really hates that the toolchain parts (binutils, gcc, glibc) seem to be stuffed full of configures run at build time14:59
rjekxgcc: internal compiler error: Segmentation fault (program cc1)15:02
rjekKinnison: Quick chat with robtaylor, I mentioned I would now like different architectures to have different compilers.  We can't do this, because there's only one build-essential.morph.  He says this is a good argument for manifests.15:06
rjekI'm abandoning hope on this again for the day.  It is too late to be going down that particular rabit hole.15:06
pedroalvarezAdded a "Instantiate a generic Trove" section on the wiki! http://wiki.baserock.org/guides/deploying-a-trove/#index3h115:07
Kinnisonrjek: my understanding of manifests wouldn't solve that.  I wanted conditionals in strata, so I could say "repo foo sha bar for arch baz, everything else, sha wibble"15:08
Kinnisonrjek: but that's a whole n'other kettle of flesh eating worms15:09
* rjek thinks it's pretty important to be able to control what GCC gets used to build things.15:10
rjekAs basically only x86 is guarenteed to work in any given release, IME.15:11
rjekI'm now basically at a complete loss for what to do apart from revert the gcc 4.7 upgrade.15:11
robtaylorrjek: rpobably best thing is to find out what's making it blow up15:12
rjekrobtaylor: I have no gdb or anything.15:12
* robtaylor is suddenly reminded of star trekkin15:12
jjardonrjek: try to upgrade to 4.8?15:12
robtaylorrjek: gah!15:12
rjekjjardon: Not sure if trolling or not.15:13
rjekgcc 4.8 will be a world of pain because it requires a C++ compiler to bootstrap.15:13
rjekIIRC15:13
robtaylorrjek: do you have a working 4.6 based devel system?15:13
rjekrobtaylor: nope.15:13
robtaylorrjek: as it 'not at this time' or 'its not possible to have one' ?15:14
jjardonrjek: not trolling :) I think that was for 4.9?15:14
rjekrobtaylor: I had a completed 4.6-based stage 3 bootstrap which then couldn't build anything for MIPS because it used upstream morph rather than my patched version, which is how all this started...15:14
robtaylorrjek: i say, make a branch which builds a working system15:14
rjek(the pushing of needed changes to git.br.org, and discovering gcc had moved on)15:14
robtaylorrjek: then use that to debug why your stage 3 doesnt work with 4.715:15
rjekI can of course just try that successful stage 3 build with a locally-patched morph rather than the one in my branch.15:15
robtayloryep15:15
* rjek is not sure if that might cause provenence problems.15:15
jjardonrjek: you are rigth, 4.8 need a c++ compiler, nevermind15:16
Kinnisonrjek: It'd be good enough to get us going, but overall it still hilights that we have the 4.6 vs. 4.7 discrepancy15:16
robtaylorrjek: well, just push a developer branch with your changes in15:16
robtaylorrjek: and use that in your branch15:16
rjekrobtaylor: I was more thinking of just checking that branch out inside the bootstrapped system, and setting PATH...15:16
Kinnisonrjek: backing the gcc sha down to your 4.6 one would let us get closer to something reviewable for merge15:17
robtaylorrjek: i understand, but that won't build you a stage three with a morph with your pachtes15:17
Kinnisonrjek: even if we can't merge it yet15:17
rjekrobtaylor: I have a built stage three.  It's just got a morph which doesn't know what mips is.15:17
robtaylorrjek: indeed, which is why i'm suggesting what im suggesting15:17
* robtaylor wonders if rjek is reading the same things robtaylor thinks he's writing15:18
rjekI don't understand why running a locally-patched morph won't let me build a devel system.15:18
rjekI don't understand much :)15:18
robtayloroh, i see, you're saying boot your stage 3 and then check out a new morph15:18
robtaylorsure that'd work15:18
rjekyeah15:18
robtaylorbetter to do what i said though15:19
rjekI don't understand what it is you said :)15:19
rjek"just push a developer branch with your changes in and use that in your branch"15:19
robtaylorrjek: do you have a branch with your cahnegs to morph15:19
robtaylor?15:19
rjekyo dawg, etc.15:19
robtaylorrjek: and have you pushed that branch up to git.baserock.org?15:19
rjekhttp://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/log/?h=baserock/robkendrick/mips6415:20
robtaylorrjek: ok, so all you need to do is update the ref for morph in you defintions.git15:20
robtaylorrjek: to point at that sha15:20
robtaylorand run your bootstrap15:20
rjekAnd rerun the whole stage1 and stage2?15:20
robtayloryup15:21
rjekWhat that actually boils down to is what Kinnison suggested: change the gcc refs to when it was still 4.615:21
robtaylorheh, yep15:21
rjek(Rather than change the morph ref, which obviously I already have :)15:21
rjekI will try to work out how to do that without giving up all hope, and kicking off a build to run overnight, then :)15:22
robtaylorah i was confused where you were at exactly..15:22
rjekI could write a book describing the story so far :)15:22
paulsherwoodrjek: i'd read it :)15:23
robtaylorI think that#s not a terrible plan at all15:23
rjekIf Baserock had a mechanism that let me say "When building for architecture X, use this specific compiler instead", I think a lot of the friction I've had in the past three weeks would have not occured.15:24
robtaylormm, we discussion variable and conditionals many many moons ago but did nothing on it15:25
* robtaylor feels theres still a lot of basics not right in morph still15:26
rjekIOError: [Errno 28] No space left on device15:40
rjekBah!15:40
rjekWhat's the incantation for clearing the cache, but not too much?15:40
paulsherwoodmorph gc15:41
rjekta15:41
liw-orcrjek, also, 'cause I don't think morph gc does that, rm *.system.* removes the system artifacts, which are huge, and not particularly interesting most of the time15:42
paulsherwoodyou could consider rm /src/cache/artifacts/*rootfs too15:42
liw-orc(morph gc keeps the few latest ones, I believe)15:42
rjekThanks15:42
jjardonShould we add a note to the wiki?15:43
* jjardon has the same problem as rjek the other day15:43
paulsherwoodi'll do it15:43
liw-orcthanks, paulsherwood15:43
rjekI was just thinking that even with a top-shelf SSD, this is slow.  Then I remembered how slow OE is.15:48
*** tiagogomes [~tiagogome@access.ducie-dc1.codethink.co.uk] has quit [Quit: Leaving]15:50
paulsherwooddone: Build failures15:56
paulsherwood==============15:56
paulsherwoodhttp://wiki.baserock.org/tips-and-tricks15:56
* rjek runs his native-bootstrap script fixing script16:04
paulsherwooddoes cache-key calculation depend on the version of morph, now?16:20
pedroalvarezpaulsherwood: I don't think so16:20
pedroalvarezbut I think that one of the last commits in morph changes the cache-keys16:21
KinnisonI think the chunks-in-definitions might have16:21
richard_mawchunks in definitions did16:21
paulsherwoodwas that unavoidable?16:23
KinnisonI believe so.16:24
KinnisonBecause previously the chunk morphology content was encoded as part of the chunk tree SHA but with it definitions, the content needed to be part of the key16:24
pedroalvarezoh yes, because of that16:25
pedroalvarezwithout the change, any change in the chunk morpholgy won't trigger a rebuild16:25
paulsherwoodright, thanks, i get it now16:26
KinnisonI *think* this was covered in Richard's mails on baserock-dev16:26
Kinnisonif you need more detail16:26
paulsherwoodthanks16:26
*** jonathanmaw [~jonathanm@82-68-191-81.dsl.posilan.com] has quit [Quit: Leaving]16:29
paulsherwoodis there any version of definitions artifacts cached at g.b.o based on the new calculation?16:29
KinnisonI don't think so.  gbo's cache tends to only contain artifacts for releases16:29
paulsherwoodok16:29
pedroalvarezSeems like youtube is taking its time today, but here will appear a video about configuring a generic Trove  in OpenStack using a tiny cloud-init script: https://www.youtube.com/watch?v=JmH4cCjs5-M17:00
pedroalvarezaround 3 minutes, some mouse clicks, and some easy instrucitons, a Trove configured in the cloud :)17:01
*** ssam2 [~ssam2@82-68-191-81.dsl.posilan.com] has quit [Quit: Leaving]17:08
paulsherwoodsuper! what would it take to get that same soln ito AWS?17:23
persiaA set of AWS clients, to start.17:24
paulsherwoodpedroalvarez: could that be put onto the vimeo channel?17:24
pedroalvarezpaulsherwood sure! 17:35
pedroalvarezI think the YouTube video is working now :) 17:36
jjardonHi, would be possible to replace the busybox sed for the real one?21:02
jjardonIm getting a compilation error because the -E option is not supported21:03
persiaLooks like it would need lorrying (I don't see it lorried).21:07
persiaSo for now, you'd have to have a local chunk.21:07
persiaBut once you have that, you can set whatever needs real sed to build-depend on real sed, and you should be fine.21:07
persiaThere is some way to manage overlays to replace busybox sed, but I don't really know how that works.21:08
jjardonI have a similar problem, so maybe it would be good replace all the commands for the real ones21:22
jjardons/similar problem/similar problem with diff/21:23
persiaIf you're building a system that needs GNU utilities, then use them.22:05
persiaI doubt the default will change, because lots of folk seem to be striving for very small systems.22:05
persiaBut there's not reason that if you're making, say, a GNOME desktop environment, you couldn't use the GNU utilities rather than busybox.22:06
pedroalvarezI had a similar problem with cloudinit, and I persuaded upstream to support busybox :) 22:06
* persia suspects its an easier sell to convince folk working on appliance enablement code to support simple stuff, and that folk building a full-featured desktop system probably don't find the difference in size between busybox and GNU signifcant :)22:07
jjardonwhat is our version of busybox?22:16
* jjardon checks22:16
jjardonSeems its 1.20.1, more than 2 years old ... Maybe in the new version the functionality has been added in the previous commands22:21
jjardonwith more problems than expected but ... gnome-shell master compiled with baserock! :)  time to go home, I will setup a vitual machine and deploy tomorrow when travelling to GUADEC and see if everything works correctly22:34
persiaNice!23:01

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!