IRC logs for #baserock for Friday, 2015-02-27

*** zoli__ has quit IRC01:20
*** SotK has quit IRC01:28
*** SotK has joined #baserock01:28
*** petefoth has joined #baserock06:47
*** pacon has joined #baserock07:24
*** pacon has quit IRC07:32
*** a1exhughe5 has joined #baserock08:12
*** gfinney_ has joined #baserock08:18
*** gfinney has joined #baserock08:20
*** mdizzle has joined #baserock08:27
*** rdale has joined #baserock08:34
*** rdale has quit IRC08:38
*** rdale has joined #baserock08:43
*** bashrc has joined #baserock09:03
*** jonathanmaw has joined #baserock09:12
*** CTtpollard has joined #baserock09:20
*** tiagogomes_ has joined #baserock09:28
*** gary_perkins has joined #baserock09:38
tiagogomes_hey, could someone else review my patches for Moonshot deployment?10:03
*** ssam2 has joined #baserock10:04
*** ChanServ sets mode: +v ssam210:04
* ssam2 begins operation "why does master of definitions not build on my Jetson"10:05
pedroalvarezgood luck :/10:06
franredssam2, good luck :S10:07
pedroalvareztiagogomes_: I have a nitpick - you are installing the m400-1003.dtb file using INSTALL_FILES and also you are modifying the kernel in a moonshot configure extension. Would it  make sense to install the dtb file also in the configure extension given that you  always need it?10:10
Kinnisonpedroalvarez: We're only installing the .dtb in the install-files because we're still waiting for a kernel patch to get it installed by-default from the kernel build10:11
Kinnisonpedroalvarez: sadly we don't have the redistributable source yet10:11
pedroalvarezok, so this is because we expect it to dissapear from definitions.git10:12
KinnisonWe do.10:13
*** Krin has joined #baserock10:17
ssam2some of the artifacts that fail to build were cached, before I hid my artifact cache, so either this system image is broken, or Morph is broken10:18
* tiagogomes_ is still puzzled why morph starting building everything from scratch yesterday10:20
ssam2I have a suspicion that my breakage is because my shared artifact cache (not has a cached version of stage1-binutils that links against /lib/ instead of /lib/
ssam2this ABI break in GLIBC has highlighted a problem in our bootstrap process, I guess... if I'm correct that this is the problem, it shows that we should have tracked the ABI break in cache keys somehow10:33
ssam2bootstrap chunks should have some kind of 'bootstrap abi' field perhaps...10:34
ssam2we could include the cache key of the host Baserock system, but this would greatly decrease the usefulness of Mason, because artifacts from would only be usable by people also running that exact version of Mason...10:34
ssam2i reckon the reason nobody else has had this problem is that I'm using a different, quite old shared artifact cache, where most people use cache.baserock.org10:35
pedroalvarezssam2: ah! so you are not building it from scratch10:35
ssam2i've not yet build stage1-binutils from scratch, I'm going to try that now10:36
pedroalvarezssam2: indeed, this is a problem in our bootstrap process..10:47
pedroalvarezwe sitll have another one, when a crossbootsraped stage1 gcc has the same cache key as a normal strage1 gcc10:48
pedroalvarezbut that one is easy to fix10:48
Zaraso, I've been slowed down a lot this last week by various conflicts between master of definitions and morph. I'm wondering if there's any way to improve testing before merge to master.10:49
ssam2right now, the solution is test new branches when it's sent for review, and give it a -1 if it breaks stuff for you10:49
ssam2s/it's/they are/10:50
ssam2the future goal, which tlsa is working on at the moment, is to run Mason's tests on candidate branches of definitions before they get merged, instead of testing master10:51
Zaraah, cool, I was going to ask if there was any way to use Mason beforehand. I'm not unwilling to test branches, but there are so many that it's not feasible for all of them, and sometimes it takes a while to tell that things are broken.10:52
ssam2yes, it takes a bit of knowledge to know which ones are the dangerous ones :)10:53
ssam2ok, i built stage1-binutils from scratch this time, and it's still broken (presumably linked against the wrong again)10:54
ssam2but gcc on this system definitely doesn't link against the wrong, so....10:54
*** paulsherwood has joined #baserock10:58
ssam2I notice that config.guess has detected the system type as armv7l-unknown-linux-gnueabihf but TARGET_STAGE1 is armv7lhf-bootstrap-linux-abi .... maybe this is breaking things?11:11
ssam2sorry, TARGET_STAGE1 is armv7lhf-bootstrap-linux-gnueabi11:15
ssam2but the abi field doesn't end in 'hf', and maybe things expect it to now... i don't know11:15
jjardonHey, does anyone here have a rp2? Does someone try to put baserock on it?11:42
rjekIt's ARMv7, should just work11:42
rjekStill only 1GB of RAM, so I wouldn't want to build on it11:43
straycatwon't the bootloader on the rp2 be something we'd need to do a bit of work on?11:46
*** CTtpollard has quit IRC11:57
pedroalvarezjjardon: I almost buy one just to do that, but then I though that for builds they won't be faster than the wandboards, and I don't know what else I could put on it12:05
Kinnisonpedroalvarez: Did you ever finish the port to the RPi 1?12:06
radiofreestraycat: we could provide a rootfs and leave the bootloader/kernel config as an exercise for the reader12:06
pedroalvarezKinnison: nope, I gave up when I found that morph couldnt build with the kernel I was using in the RPI (raspbian kernel)12:08
ssam2I don't think i'll be able to do a 15.09 release today, still debugging this bootstrap breakage on Jetson12:08
ssam2it is totally mindbending12:08
pedroalvarezKinnison: are you interested?12:08
KinnisonI have an rpi which was given to me for xmas12:08
KinnisonI was going to use it to write a smart snake vivarium monitor12:08
Kinnisonbut I can use raspbian12:09
Kinnisonit's fine12:09
straycatmy friend has an rpi2 and wanted to try baserock on it12:09
straycatradiofree, hrm, i guess :)12:09
radiofreessam2: what bootstrap breakage?12:11
ssam2I get a build failure in stage1-gcc when building master12:13
ssam2i'm interested if you see the same thing12:13
ssam2I've previously built the same commit successfully beyond master, but that was using cached artifacts12:13
ssam2if I hide cache/artifacts I see the build failure (which is caused by 'armv7lhf-bootstrap-linux-gnueabi-as' being linked against instead of
ssam2i'm totally at a loss to how it's happening so far12:14
radiofreedoes stage1-gcc use the hosts compiler?12:16
radiofreessam2: what version of gcc do you have on the host12:17
ssam2the host is a really recent build of master12:17
ssam2if I compile binutils on the host outside of morph, it works. And if I unpack the binutils chunk from the cache, it works12:17
ssam2but the version that makes it into the staging area doesn't workj12:18
ssam2obviously I'm missing something, but no idea what12:18
radiofreei could try compiling master on my br machine12:18
ssam2if you could run a build with a clean artifact cache, and --artifact-cache-server and --trove-host set to none i'd much appreciate it12:19
radiofreewell i don't really want to clean the artefact cache? can i just add "echo HELLO!" to binutils morph and build from there?12:20
pedroalvarezradiofree: that should be enough :)12:21
ssam2radiofree: just move it out of the way12:22
radiofree/dev/sda1               109.9G     93.2G     11.1G  89% /src :\12:22
radiofreeoh well, i'll just build from master over the weekend12:23
* radiofree nukes his artifacts12:23
pedroalvarezmv /src/cache/artifacts /src/cachhe/artifacts-bkp?12:23
radiofreeoh wait, maybe i can't rebuild from master over the weekend if this doesn't work12:23
radiofreeyeah, i'll mv it12:23
radiofreeshould i clean out /src/cache?12:25
radiofree/src/cache/ccache even12:25
ssam2i forgot about ccache, I think it's not used12:25
ssam2but maybe that's the problem, who knows12:25
radiofreewell i mv it12:26
*** pacon has joined #baserock12:29
radiofreessam2: how long into the build does it take for it to fail?12:42
radiofree2015-02-27 12:39:29 [Build 2/192] [stage1-gcc] Running build-commands12:42
radiofreeby "master" did you mean master of definitions?12:43
ssam2not sure. i'm running another build and it seems to be taking longer than it did before12:43
ssam2yes, master of definitions12:43
ssam2oh, my build just failed again12:44
ssam2i keep thinking "Oh it's worked this time" just as it fails12:44
ssam2error building target libgcc12:44
*** zoli__ has joined #baserock12:50
ssam2radiofree: to actually answer your question, about 13 minutes12:50
radiofreei'm 13 minutes in now, this is exciting12:52
jjardonhi, seems there is something wrong with the indentation of the mailing list archives?
radiofreejjardon: that seems to be a common problem with mailman?12:59
radiofreei have seen archives like that before13:00
radiofreessam2: well i'm > 20 minutes in now13:01
ssam2what sha1s of morph and what sha1 of definitions is it? and how old is your devel system?13:02
radiofreessam2: how do i find out how old my devel system is?13:02
ssam2 /baserock/*-rootfs.meta13:03
radiofree"sha1": "83db755ad2d104af9084e567962bc6d0078dd4d1",13:04
ssam2ok, thanks13:04
radiofreeas for morph, i'm still using your "Hack petrify command into working again" branch13:04
radiofreeso i guess the last "real" morph commit there is 1d7e39b955f4da58230f9c5a06ff1b05f8bc020b13:04
radiofreeit might still fail13:07
radiofreei've booted into the wrong kernel13:07
radiofreeso this thing might still fail13:08
ssam2what do you mean?13:08
ssam2oh bum! I forgot about the chunk hardlink cache!!!13:08
radiofreemy jetson is probably only running ~ 200MHz13:08
* ssam2 looks forward to SotK killing it forever with OStree13:08
radiofreei'll abort and tree again13:09
ssam2I think my original assessment of the problem may have been correct (ABI break in glibc on ARM that the cache key doesn't track), i'll run a build with $tempdir/chunks deleted and see13:10
*** zoli__ has quit IRC13:11
radiofreetiagogomes_: you can ignore my 50 minute systemd benchmark (i think)13:13
radiofreei was booting into the wrong kernel on my jetson13:13
*** zoli__ has joined #baserock13:16
tiagogomes_radiofree are you saying that you built systemd in 50m running just at 200MHz!?13:29
kejiahucan anybody help me with the error please? . also, markupsafe can be built if I checkout it from upstream and manually run build command13:54
pedroalvarezssam2: that might be it, but, wasn't it failing in stage1-binutils?13:54
SotKkejiahu: the stratum containing markupsafe needs to build-depend on python-core13:54
*** a1exhughe5 has quit IRC13:55
kejiahuSotK, shouldn't that been defined in morph file already?13:56
SotKwe recently moved setuptools and some strata didn't get updated13:56
SotKI think this is fixed in master of definitions though13:57
kejiahuSotK, thanks13:57
straycatI kind of feel that python-setuptools might have been better left in core13:58
straycatperhaps it's a bit late to be saying that, I missed the patch that moved it13:58
* straycat disappears13:59
franredstraycat, as far as we fix it, it is not more pain14:06
*** CTtpollard has joined #baserock14:07
tiagogomes_hey richard_maw, as you were the one to write the pxeboot extension, would have you time to review it?14:09
*** a1exhughe5 has joined #baserock14:10
richard_mawtiagogomes_: review it for what?14:11
* richard_maw lacks context14:11
tiagogomes_richard_maw deploying baserock to a moonshot node14:12
tiagogomes_sorry, I meant reviewing the patches that I've sent to the mailing list yesterday for adding support to deploy baserock into a HP Moonshot node14:13
richard_mawsure, I'll take a look14:14
SotKHow do we want the OSTree artifact cache to work with cross-boostrap?14:17
Zara(is there any way of searching branches on g.b.o for files, by name, without cloning the repo? the search bar in the top right doesn't seem able to do this, so I'm guessing not)14:30
ssam2pedroalvarez: no, it was failing in stage1-gcc, due to being unable to execute programs from stage1-binutils14:33
ssam2i left a build running over lunch having emptied tmp/chunks and it's got up to stage 2114:33
ssam2so I think I was right14:34
tiagogomes_stage 21 :)14:34
radiofreessam2: working for me as well14:35
ssam2right. I shall send an email to the list :(14:37
persiaZara: I believe gitano has an ls-files command, but I don't know how it works.  You might ask on #gitano.  That said, if we move to a different git server, you'll have to learn a new rune :)14:44
*** jonathanmaw has quit IRC14:46
*** jonathanmaw has joined #baserock14:46
KinnisonI'm not sure quite what you're suggesting there but I dn't think it has what you think it has14:47
pedroalvarezI think the morph-cache-server of a trove has inproved with this patch:
radiofreeafter moving my /src/cache/gits directory back in place, i'm getting15:10
radiofreeERROR: Git directory /src/cache/gits/git___git_baserock_org_baserock_baserock_morph has no commit at ref bc3e042560424b433a408f01725f36cc2d47859c^{tree}.15:10
radiofreewhen trying to build a system from master of definitions15:10
radiofreerm /src/cache/*.pickle seemed to fix it15:12
franredwhat Im missing in my cluster to see this error
ssam2radiofree: if your Morph is older than f5163dd418e342fe6e5fb18625828076130a5e57 it's a known bug15:13
ssam2upgrade to hopefully fix it15:13
ssam2franred: second item under 'deploy:15:14
radiofreeah ok, i should switch back to master morph (i'm using your petrify branch...)15:14
straycatZara, git ls-remote15:14
ssam2radiofree: rebase the petrify branch on top of master if you want15:14
ssam2freanred: for whatever reason, you have to put: 'deploy:\n    name-of-system:\n        type: ...'15:15
ssam2if that makes sense15:15
ssam2name-of-system can actually be anything, but putting the name of the system is conventional I think15:15
franredssam2, I forgot to add "relase:"15:16
pedroalvarezfranred: as Sam said, it can be whatever :)15:16
richard_mawfranred: biff15:17
richard_mawsorry, not franred15:17
richard_mawtiagogomes_: biff15:17
Zarastraycat, persia: ah, sorry, I meant if it was possible to do it through the web interface. I should have been clearer.15:17
franredssam2, oohh, I think I got it15:18
persiaZara: Sorry.  No idea.15:18
tiagogomes_thanks richard_maw!15:18
*** pacon has quit IRC15:26
straycatfranred, yeah probably not15:30
pedroalvarezpdar: I don't think you need to resubmit the patch :) It has already a +2 and the commit message can be fixed during the merge15:31
pedroalvarezpdar: do you have push access to g.b.o?15:32
pedroalvarezQ: I've heard that we should move from urllib2 to requests, why? Does it work better?15:33
pedroalvarezi think it was ssam2 ^^15:33
straycatrequests does seem nicer15:34
* SotK prefers requests15:34
pdarpedroalvarez: Oh great! I dont have push access to g.b.o no15:35
ssam2pedroalvarez: it is better15:35
*** zoli__ has quit IRC15:36
pedroalvarezI may take a look at it15:43
*** gfinney has quit IRC16:00
*** gfinney has joined #baserock16:02
tiagogomes_not again ERROR: Git directory /src/cache/gits/git___git_baserock_org_baserock_baserock_morph has no commit at ref bc3e042560424b433a408f01725f36cc2d47859c^{tree}.16:04
radiofreethat sounds like the bug i had before tiagogomes_, apparently it's been fixed in morph master16:04
jjardonssam2:  "BAD morphlib/ Copyright (C) 2012-2014  Codethink Limited" :)16:12
ssam2oh, is that ./check in master of morph ?16:19
ssam2i'll push a fix16:20
*** CTtpollard has quit IRC16:23
franredwhy the devel systems for arm and ppc have openstack-clients on them? do we have any openstack running in these machines?16:53
franredpedroalvarez, ^^16:56
ssam2you might still want to interact with an openstack cloud16:56
ssam2I think we should aim for our systems to have the same components in them on all architectures16:56
pedroalvarezI agree16:56
franredfair enough16:57
*** petefoth_ has joined #baserock16:57
*** petefoth has quit IRC16:58
*** petefoth_ is now known as petefoth16:58
radiofreedoes openstack need kvm?16:58
pedroalvarezfranred: also, we don't need to have  an openstack running on those architectures. These clients will allow us to deploy to an x86 openstack from arm, for example16:59
pedroalvarezradiofree: it can work with qemu16:59
radiofreeqemu-kvm surely16:59
radiofreei wouldn't like to use something not using kvm....16:59
pedroalvarezWe have kvm in baserock :)16:59
radiofreepedroalvarez: yes, kvm is on the kernel side17:00
radiofreeyou'll be using qemu-kvm for the actual 'meat'17:00
pedroalvareznote that kvm and qemu are the same thing now17:00
radiofreeanyway, this stuff from siemens might be interesting, kvm on jetson-tvm17:00
radiofreeor at least, adds support to u-boot to allow you to use kvm on a jetson17:01
persiaThere are openstack environments for ARM available.17:01
persiaFor POWER, they don't use KVM, but rather the POWER hypervisors.  I have the impression that there's work happening upstream to add KVM to POWER for the next generation, but I can't confirm this.17:02
persiaAlso, there are lots of other hypervisors supported, Xen is popular, for example.17:02
*** mdizzle has quit IRC17:10
radiofreepersia: sure, but we have a lot of jetsons, baserock works there17:12
radiofreeat least needs PSCI support adding to u-boot for hypervisor to work17:12
persiaDo you mean for remote console to work?17:12
radiofreeno i mean for KVM to work at all17:14
radiofreecpu needs to be booted in hyp mode17:14
persiaOh, right.17:17
persiaI'm happy with folk using Jetsons: I just wanted to point out there *were* environments where having the openstack clients available for ARM/POWER were useful.17:18
persiaAnother example that comes to mind is the Servergy hardware, which presented an OpenStack API for logical partitioning of the server.17:19
persiaThat was PPC, not POWER, but close, and I'd presume anyone doing PPC would start from the POWER systems.17:19
radiofree[    4.321202] kvm [1]: Hyp mode initialized successfully :D17:41
paulsherwoodradiofree: what device?17:44
* paulsherwood has missed backscroll17:45
tiagogomes_knowing James, I would be on jetson17:45
tiagogomes_bet on17:45
jjardonmmm, running morph test fill up my /tmp directory, known issue?17:46
paulsherwoodjjardon: do you have a /src partition?17:48
paulsherwood(and is it mounted?)17:48
ssam2jjardon: how big is your /tmp ?17:49
paulsherwood(and is morph.conf containing tempdir = /src/tmp)17:49
ssam2there are some checks in the Yarns that go 'if there's not enough space available, skip this test' but they aren't perfect17:49
ssam2try setting TMPDIR=/src/tmp17:49
ssam2./check doesn't honour morph.conf's tempdir setting17:50
* paulsherwood adds that to his list of gripes about morph's test code17:50
jjardonto be clear: Im running the tests in a chroot, but its my host /tmp the one who gets filled17:50
jjardonpaulsherwood: all seem correctly configured in my chroot17:51
tiagogomes_ jjardon did you mount bind /tmp?17:51
tiagogomes_jjardon, is your host /tmp a tmpfs17:52
jjardontiagogomes_: it is a tmpfs, yes17:52
jjardontiagogomes_: not sure, I do what enter-baserock does ;) will check later17:53
*** tiagogomes_ has quit IRC18:00
*** ssam2 has quit IRC18:08
*** bashrc has quit IRC18:10
*** jonathanmaw has quit IRC18:15
jjardonthe TMPDIR=/src/tmp hint fixed the issue; paulsherwood: Did you add this to storyboard/wiki or should I?18:17
paulsherwoodjjardon: please do18:18
*** rdale has quit IRC18:19
pedroalvarezyay, requests tries 3 times to fetch an artifact:
jjardonpaulsherwood: btw, probably you already know but python3 is already merged, so I closed the task assigned to you18:27
paulsherwoodjjardon: i saw that. sorry, i've been 'on the road'18:28
jjardonsure, dont worry18:28
*** sambishop has quit IRC18:39
*** a1exhughe5 has quit IRC18:55
paulsherwoodjjardon: i'm trying to review your versioning code... why precisely do you need all those checks in it at this point? it seems to me that any failcase, or valid version 0 should just assume version 0? any valid version 1 is treated as 1, 2 as 2, etc... ?19:05
paulsherwood(for example, suppose a user has created a VERSION file themselves, containing jrandom something else... morph will now fail for them???)19:07
jjardonpaulsherwood: with the current implementation, yes19:08
jjardonVERSION can only be "version: 0" after the patches19:08
paulsherwoodthat's not useful for users with existing definitions - you break compatibility? i don't see why that is necessary19:11
paulsherwoodfor example if VERSION is a non-yaml file. morph should just ignore it, not fail19:12
jjardonpaulsherwood: not really, definitions without VERSION file is treated as if they have a VERSION file with version: 0 on it19:13
jjardonpaulsherwood: yeah, I was not sure about that (VERSION be a yaml file) we can rework that19:13
paulsherwoodjjardon: i'm not explaining myself properly, maybe....19:14
paulsherwoodpossible cases:19:14
paulsherwoodno VERSION file: morph should assume version: 019:14
paulsherwoodVERSION file contains non-yaml: assume version: 019:15
paulsherwoodVERSION contains yaml, but version: is undefined or missing: assume version 019:15
straycatwhat's version 0?19:15
paulsherwoodVERSION contains yaml, version: is 0 - assume version 019:15
paulsherwoodVERSION contains yaml, version: is 'something', not 0, then reject unless morph understands the value of 'something'19:16
paulsherwoodso there's no need for any testing except can i find a value of version: which is non-zero19:17
straycatoh ok19:18
* straycat disappears19:18
paulsherwoodjjardon: we could be even simpler and say VERSION contains yaml and version: is a version we understand, handle it. else assume version 019:24
jjardonpaulsherwood: you are assuming we are going to carry the code to handle version 0 forever then19:26
jjardonI though the idea is to have a migration path for definitions instead19:26
paulsherwoodthe idea is to support multiple versions, iiuc. it would be trivial to drop the 'else' at some point.19:29
paulsherwoodi am not assuming we carry code to handle version 0 forever - but i do expect we need to support a *set* of versions, not just one at a time19:32
*** gary_perkins has quit IRC19:35
*** CTtpollard has joined #baserock19:37
*** CTtpollard has quit IRC19:39
jjardon"VERSION contains yaml and version: is a version we understand, handle it. else assume version 0"; if we only understand version: 0, we will assume version: 0 if VERSION contains version: 4, for example. Are we sure we want to do that?19:39
* jjardon notes that is late on friday so maybe his brain is not working correctly anymore19:40
*** gfinney has quit IRC20:05
*** zoli__ has joined #baserock20:31
paulsherwoodjjardon: if we don't understand 'version: 4' yet, the only options are break, or treat it as version: 0. i prefer not to break by default20:35
persiaPerhaps if we can parse the VERSION file, but do not comprehend the version, we can report "This is version X, which this morph doesn't understand.  Shall it be treated as version 0?"20:38
persiaWhereas if we can't parse it, we declare it version 0.20:38
persiaSadly, lots of people use the "use-latest-morph" instructions to end up with an old morph because they forget to update morph when updating their devel system.20:39
*** zoli__ has quit IRC22:40
*** zoli__ has joined #baserock22:41
*** zoli__ has quit IRC22:53
*** pacon has joined #baserock23:21

Generated by 2.14.0 by Marius Gedminas - find it at!