IRC logs for #baserock for Wednesday, 2015-11-25

*** toscalix has joined #baserock08:34
*** paulwaters_ has joined #baserock08:42
*** CTtpollard has quit IRC08:52
*** rdale has joined #baserock08:52
*** toscalix_ has joined #baserock08:53
*** toscalix has quit IRC08:53
*** CTtpollard has joined #baserock08:57
*** ctbruce has joined #baserock08:58
*** bashrc_ has joined #baserock09:00
VLetrmx[gerrit2@salo:~]$ java -jar gerrit-2.2.2.war init --batch -d ~/gerrit_testsite09:36
VLetrmxExecuting /home/gerrit2/gerrit_testsite/bin/ start09:36
VLetrmxStarting Gerrit Code Review: OK09:36
VLetrmxsometimes i am a bit of a moron09:36
*** franred has joined #baserock09:49
pedroalvarezVLetrmx: no!09:51
pedroalvarezdoes that mean that you gave up and you are doing it outside of baserock?09:51
*** gtristan has joined #baserock09:52
VLetrmxit could've worked in baserock really10:04
VLetrmxi build a devel system + java10:04
VLetrmxbut then this simple gerrit setup still failed because we have busybox start-stop-daemon10:04
VLetrmxso i'm running it on my host system10:05
pedroalvarezmuch easier10:05
*** edcragg has joined #baserock10:05
VLetrmxwell yes but only because we have busybox start-stop-daemon10:05
*** ssam2 has joined #baserock10:06
*** ChanServ sets mode: +v ssam210:06
pedroalvarezI don't exactly understand why that makes it fail. And if I should be worried about that as a possible error in the future in our current deployment for infra10:07
franredVLetrmx, I made gerrit worked in baserock, but it got removed, but maybe you can have a look how we did it in the pass10:10
pedroalvareznah, go for what you are doing, and stop wasting time :)10:11
VLetrmxhey, my "time-wasting" gave us baserock 15.47 :p10:13
pedroalvarezand I really thank you for that  :)10:14
persiaHappiness is a hairless yak.10:15
rdalei have some definitions checked out in my baserock vm, and i want to build a system from them. when i use 'morph build ...' it wants to clone a temporary copy of the original definitions  repo, but i don't have permission to do that from within the vm. is it possible to tell morph to clone the local copy of defintions?10:20
VLetrmxif you don't have permission to clone, how did you clone the original repo?10:21
*** Lachlan1975 has joined #baserock10:22
rdalei mounted /src with sshfs and cloned it10:23
VLetrmxokay if you cloned over ssh, then your vm needs your public key so possibly ssh in with -A, else do a new clone over git:// or add another remote with git:// ?10:24
VLetrmxactually adding a new remote like that probably won't work10:24
pedroalvarezI agree it's confusing that `morph` needs to pull before building?10:24
VLetrmxyou could possible set origin though10:24
*** toscalix_ has quit IRC10:25
SotKI think doing `--local-changes=include` on your morph command forces it to clone the local checkout10:25
* SotK could be wrong10:25
rdaleok, i'll try that10:26
tiagogomesmorph clones definitions to do a checkout10:26
tiagogomesbut it should clone from the local repo10:26
SotKalso if you don't have all the other repos required for the build cached already you'll need to be able to clone them from the git server you're using10:26
*** toscalix_ has joined #baserock10:27
rdaleusing '--local-changes=include' doesn't fix it10:28
*** toscalix_ has quit IRC10:44
*** gtristan has quit IRC11:19
*** gtristan has joined #baserock11:26
*** jonathanmaw has joined #baserock11:33
*** CTtpollard has quit IRC11:34
*** CTtpollard has joined #baserock11:35
*** toscalix_ has joined #baserock11:38
rdaleis there a simple way to migrate a defintions repo from version 1 to the current version for 15.47.1?12:26
tiagogomesrdale have you tried to run ./migrations/run-all ? That script is on the latest definitions12:28
pedroalvarezmy approach when  doing it was:12:28
pedroalvarez- Merge tag12:28
pedroalvarez- Given that merging the tag modifies the file VERSION, modifiy it again to set it to "1"12:28
pedroalvarez- run ./migrations/run-all12:28
pedroalvarezif your definitions have only a few patches on top, might be easier to rebase?12:29
rdaleit isn't the baserock definitions repo12:29
*** toscalix_ has quit IRC12:30
pedroalvarezthat's why the first step of my approach is to merge latest tag12:30
ssam2i'd suggest: manually copy the migrations/ dir from into your repo; run migrations/run-all; then merge latest tag of baserock definitions (if you want)12:31
rdaleyes, that sounds a good idea12:31
rdalethe code to migrate shouldn't belong to a specific definitions repo really - nor should it belong to a build tool if we have multiple build tools12:33
tiagogomesthat looks the sensible approach. Maybe we should document it somewhere in definitions?12:33
ssam2it should be documented in README already12:37
ssam2it kind of is, but could do with tweaking if anyone has time12:38
ssam2I agree that it's not ideal having the code living in our definitions repo -- just seemed the simplest means to the end at the time12:38
rdalein the forked repo i'm working on we've been using the master branch for changes, which seems a bad idea wrt respect to keeping it in line with the baserock master12:42
ssam2yeah, i wouldn't recommend that12:42
ssam2easy to fix though12:43
ssam2the README should probably mention that too12:43
rdaleah ok12:43
ssam2the same situation exists with Buildroot, but there doesn't seem to be any text in the buildroot manual that we could nick12:44
richard_mawnote to self: Expand all in gerrit doesn't provide a unidiff of all the changes12:52
tiagogomesI think that if the migration scripts were in a different repo, it actually would be simpler to update some definitions12:53
jjardonkudos for the one that implemented a progress bar in morph!12:58
tiagogomesthat was Mr Ipsum12:59
ssam2tiagogomes: yes... i think so too13:00
ssam2on reflection13:00
*** locallycompact has joined #baserock13:00
* tiagogomes tries gerrit edit mode to replace "have been" by "has been"14:26
tiagogomesToday is a remarkable day in Morph's history. Branch-and-merge is dead14:30
tiagogomesthanks richard_maw and VLetrmx for the reviews14:30
franredtiagogomes, good!!14:31
ssam2i think that was the first task I was given to do when joining the Baserock project :-)14:31
ssam2(writing it, not killing it)14:31
persiatiagogomes: Thank you.14:31
ssam2seemed quite a good idea at the time :-)14:31
ssam2although it was a total nightmare even then, at that time you could still have an infinite number of definitions repos all pointing to different refs of each other14:32
tiagogomeshehe :)14:32
tiagogomesyeah, when the strata and systems had repo and ref14:32
tiagogomeswell, you couldn't say that it lacked flexibility at that time :)14:33
VLetrmx"+[[change.submitWholeTopic]]change.submitWholeTopic (*Experimental*)::"14:50
rdalewhen i try to build a system in my forked definitions repo, i get an error about cloning a lorry repo:
richard_mawVLetrmx: wassat mean?14:53
VLetrmxfor those interested, we're now down to 23,000 lines from 25,000 according to sloccount14:53
rdaledo i need to specify a local lorry in morph.conf or some other config file?14:53
VLetrmxrichard_maw, seems gerrit has an experimental feature for submitting entire topics14:54
ssam2rdale: seems that the definitions are referring to a ref that doesn't exist in your Trove's version of baserock/baserock/lorry.git14:54
ssam2could it be that your mirror is out of date?14:54
ssam2fwiw that commit is the latest one on
rdalei don't know how the mirrored lorry repo is kept in step with the baserock one, or how often14:55
rdalei rebased the local definitions repo on the upstream baserock definitions repo. but i see that sha1 in the definitions repo: ./strata/lorry.morph:  ref: d64da0cb16679f0791077ff5e1f9d03fa16a745f14:57
pedroalvarezwhich is:
pedroalvarezand exists14:58
VLetrmx( )15:01
pedroalvarez'New change submission workflows, "Submit Whole Topic" and "Submitted Together"'15:04
tiagogomesVLetrmx 23,000 is still a lot :|15:11
pedroalvarezI will need to tweak some scripts that use morph-edit15:13
rdalethe last commit in my local version of baserock:baserock/lorry with a missing sha1, was 10th november - has something changed since then15:14
pedroalvarezrdale: yes, I linked the commit  before15:15
pedroalvarezI looked at the trove you are using and looks like it's lorrying slowly things15:15
VLetrmxtiagogomes, i guess we can improve15:16
rdaleah ok, it's a couple of weeks behind15:16
VLetrmxspeaking of which is there any reason we still have the exts in morphlib?15:16
pedroalvarezrdale: that commit was done yesterday I believe15:16
SotKVLetrmx: I left them there for backwards compatibility ages ago, and never had time to get rid of them15:17
pedroalvareznuke them!!!!!!15:17
rdalepedroalvarez: i assume there were lots of other commits between 10-24th november, which i'm not getting15:17
pedroalvarezrdale: oh, i thought you were referring to commits in lorry.git15:18
VLetrmxSotK, okay, it should be safe to get rid of them now, given we only support versions 6 and 715:18
ssam2VLetrmx, SotK: it might be good to do a migration for the exts.. although that boat has kind of sailed already15:18
ssam2so never mind15:18
SotKI thought we did make it need a new version?15:19
ssam2anyone who hasn't updated the 'configure-extensions' and deployment 'type' fields to prepend 'extensions/' to all of them will have already got weird errors15:19
ssam2SoTK: i don't think so15:19
rdalepedroalvarez: yes, it is a commit in lorry.git, but my cloned cached version doesn't have any commits after 10th november15:19
SotKperhaps we hijacked a different version number and didn't put it in the migration though15:19
VLetrmxthat's a good point15:19
ssam2yeah, see:
ssam2version 5 *allows* new-style extensions, but we never officially *disallowed* the old style15:20
ssam2even though the old style is actually broken15:20
ssam2so, whatever, things already broke15:20
jjardonHi, Im getting this error when loggin in baserock wiki through openid: "Error: OpenID failure: naive_verify_failed_return: Direct contact invalidated ID provider response. " known bug?15:24
pedroalvarezwhich openid?15:25
pedroalvarezhm.. idk15:27
pedroalvarezthat should work15:27
pedroalvarezjjardon: that just workedfor me15:33
* tiagogomes never tried logging in to the wiki with OpenID15:35
jjardonpedroalvarez: with a OpenID launchpad provider?15:36
pedroalvarez( )15:37
jjardonpedroalvarez: it doesnt here: Imactually getting another error now: "Error: OpenID failure: time_bad_sig: Return_to signature is not valid."15:40
jjardon:(( I will try the mail login15:41
pedroalvarezor baserock openid!15:43
*** franred has quit IRC15:49
tiagogomesIt worked for me15:53
pedroalvarezjjardon: oh, thats because we banned you from editing the wiki!15:55
tiagogomesmorph help tar.write is giving me a stacktrace16:08
tiagogomessighs, when I try to fix one thing, I find out more things to fix16:09
gtristanSo I think this is a bug but I'm not sure... I have pulseaudio's system integration hooks running twice16:15
gtristangnome.morph pulls in audio-bluetooth.morph and also pulls in multimedia-gstreamer.morph, which also pulls in audio-bluetooth.morph16:15
gtristanthen I get errors in my new system integration hooks saying the pulse user already exists16:16
pedroalvarezhm... this might be a bug in ybd16:16
gtristanso what is the bug ? is gnome.morph at fault for redundantly pulling in audio-bluetooth ?16:16
pedroalvarezI actually don't know how that is done in ybd16:16
gtristanmaybe that16:17
pedroalvarezI implemented that in morph16:17
perryli've pushed a quick change to my cgit filter patches...apologies for not being too visible in here, i've been bogged down trying to get the URL redirect working! i'm away for the next two weeks so won't be able to implement any changes until mid December probably...16:19
*** toscalix has joined #baserock16:19
gtristannow, will removing the audio-bluetooth.morph reference in gnome.morph cause a webkit rebuild...16:19
jjardonI think there is not a bug in the definitions: gnome depens on those 2 stratum because the components in gnome depends on stuff present on those stratums; I prefer to be explicit than implicit in the dependencies16:19
jjardongtristan: yes16:19
*** gtristan has quit IRC16:21
*** gtristan has joined #baserock16:22
ssam2yeah, seems like ybd is wrong to me16:24
gtristanholy crap16:25
gtristanremoving audio-bluetooth.morph from the dependency list of gnome.morph causes a rebuild16:25
* gtristan facerock16:25
gtristanthat is certainly a bug16:25
SotKwhy is that a bug?16:26
pedroalvarezI can explain how it works in morph16:26
* tiagogomes heard that morph is getting in shape16:26
gtristanSotK, because multimedia-gstreamer.morph also depends on audio-bluetooth16:26
pedroalvarezoh yes indeed, this shouldn't trigger a rebuild16:27
gtristanso gnome.morph builds exactly the same thing with or without explicitly mentioning that dependency16:27
SotKah, sounds like ybd is confused a bit there then16:27
pedroalvarezbut but16:27
pedroalvarezI believe that the cache key is calculated with the contents of the yaml file16:27
pedroalvarezmorph file16:27
gtristanpedroalvarez, that cannot be exactly true16:27
pedroalvarezidk to be honest :S16:27
gtristanI think it takes into account the content of build & install commands16:28
gtristanbut conveniently does not change for system-integration hooks16:28
gtristansince those do not require rebuilds16:28
pedroalvarezthey do require rebuilds in morph16:28
pedroalvarez(but they might be different implementations)16:28
* gtristan found that to be quite convenient ;-)16:28
* richard_maw double checks whether system-integrations are included in the cache keys16:29
pedroalvarezrichard_maw: they are part of the chunk artifact, so they should be in the cache key16:29
* richard_maw thought we stopped doing the implementation that dropped files in /baserock somewhere16:30
richard_mawat which point they have no effect on the chunk itself16:30
pedroalvarezrichard_maw: it seems that was done in ybd16:30
gtristanpedroalvarez, morph runs system integration hooks on each artifact ?16:30
gtristanpedroalvarez, not the case in ybd, only runs all system integration at the end16:30
pedroalvarezgtristan: no, morph generates scripts in /baserock/system-integration, for every chunk that has some of them. And then at the end, morph executes all of them16:31
richard_mawpedroalvarez: it used to at least, can't remember if it still does that16:31
richard_mawah, I changed it to not use run-parts, I didn't change it to not run the scripts16:32
* richard_maw got confused as to whether these were changes he wanted to make or changes he had made16:32
pedroalvarezright so it's almost the same16:32
*** toscalix_ has joined #baserock16:35
*** toscalix has quit IRC16:35
*** gtristan has quit IRC16:42
*** gtristan has joined #baserock17:01
gtristanhmmm, scratch that it's actually not a bug in ybd17:02
gtristanfor "some reason" the pulse user/group exists already, I suspect that pulse make install did something17:02
gtristannot sure what17:02
gtristanbut system-integration does not run twice, anyway17:03
richard_mawIMO it's a deficiency in morph, since it doesn't *have* to make system-integration commands part of the key, as you can implement it without putting data in the artifact.17:03
gtristanrichard_maw, right we're talking about two separate things though17:03
richard_mawso we are!17:04
* richard_maw got confused17:04
gtristanthe bug I suspected existed in ybd, was that if you indirectly depended on the same strata twice, system integration would run twice17:04
pedroalvarezgtristan: right, look at this:
*** ssam2 has quit IRC17:14
* gtristan looks17:18
gtristanpedroalvarez, probably best to factor that out eventually ?17:19
*** ctbruce has quit IRC17:19
jjardonthis is exactly the reason why I suggested this
pedroalvarezgtristan: note that changes there will trigger a full rebuild :)17:20
gtristanpedroalvarez, bah, rebuilds are ok in the winter time17:20
pedroalvarezI think that "creating users" is a topic that we need to discuss17:20
pedroalvarezusers, and ownership of files, etc17:20
gtristanI think clearly its more desirable if everything is cleanly split into their own components/chunks17:21
gtristanprobably we want to abolish install-files and the like as well17:22
gtristaneven in those odd cases where the files one needs is not provided by an upstream repo, that would be more cleanly handled by a chunk that does not build a repo17:23
gtristanjjardon, I still have a browser tab open on that systemd thingamajig17:24
gtristanI dont particularly like it mostly because I dont like the idea of tying down definitions to be 'systemd only'17:25
VLetrmxwhich is where richard_maw's multiple sources could come in17:25
VLetrmxsince you'd just have a chunk that doesn't define a source17:26
gtristanin any case, it's a little orthogonal, because whether a user is created in a system-integration hook with useradd, or by installing some systemd-foo and expect systemd to create it... it can be done in the respective chunk17:26
gtristanVLetrmx, yes that is a good thing, and elegantly handles the git submodules problem as well17:27
pedroalvarezThe thing is that in the sandbox environment  we can't set ownership on file, and thus, creating users might be impossible17:28
pedroalvarezat least with linux-user-chroot in Morph17:28
gtristanpedroalvarez, then gnome is broken in morph ?17:28
pedroalvarezcreating a file for systemd to make it create the user later "solves" this limitation17:29
gtristanpedroalvarez, it heavily relies on the useradd/groupadd statements in the existing system-integration hooks17:29
pedroalvarezgtristan: I don't think so given that Mason has been building it17:29
pedroalvarezgtristan: I guess that if  you are not creating home folders for them, then it doesn't fail17:29
gtristanone thing we cannot do reliably it seems though, is file capabilities17:29
VLetrmxrjek suggested fakeroot for that17:29
gtristansetcap in a system-integration hook doesnt work, but would be nice to have that work17:29
pedroalvarezsee, all these things we need to discuss :)17:30
richard_mawgtristan: unless tar records file capabilities, I'm not sure there's any value in doing so17:31
gtristanI dont believe tar does that, I'm just noting that it's something one typically needs to do when installing some software17:32
gtristansomething we have no present solution for17:32
richard_mawI think it's something not supported in package based systems either generally, I heard debs would strip all that out anyway and require a post-install script to sort out permissions17:33
richard_maw(for us that would have to be a first-boot thing)17:33
gtristanrichard_maw, right, it's supported by debian as post scripts - which is something we dont really have17:34
gtristanunless I'm unaware of something that does exist ;-)17:34
gtristanbut I think we stop at the image, once the image is created there is nothing that runs on the system for integration in a first-boot scenario or such17:35
richard_mawhm, it'd have to be a disk image creation hook or first-boot17:35
*** locallycompact has quit IRC17:36
richard_mawhmm, doing it on first boot would be incompatible with having an immutable /usr mount too…17:38
*** tiagogomes has quit IRC17:38
gtristanrichard_maw, well, it's not a fire "right now" until it becomes one... I think this system-integration in general probably "should" be run in a live system, and that is probably something to consider when thinking of the distribution/upgrade problem17:39
* richard_maw doesn't want deployers to have to run code from the system they are deploying17:41
gtristansheesh, lots of problems to solve huh... if we actually had to upgrade live systems, we would then need system-unintegration hooks !17:41
persiaDebian's "post-install" maintainer script content is usually Baserock's "system-integration" script.17:41
persiafirst-boot stuff is different (and exists for both Debian and Baserock, so that a single image can have multiple instantiations safely)17:41
persiagtristan: Why system-unintegration?17:41
gtristanpersia, for when we upgrade a system to a version which say, no longer includes chunk "foo", but chunk "foo" has left some debris via it's integration hook17:42
gtristanor, just for a reliable upgrade path for the same chunk "foo", to cleanly uninstall the older version before integrating the new one17:43
persiagtristan: Ah, right.17:43
richard_mawatomic updates17:43
* richard_maw hides17:43
persia documents the set of scripts Debian supports17:43
pedroalvarezyeah,  I think our upgrades handle that case, we don't need system-unintegration things17:45
gtristanhmmm, 20 odd GB of free space is not enough to deploy a 6GB gnome image :-/17:45
persiaI thought we fixed that: it used to take 3+ copies to deploy, but isn't it down to 2 now?17:46
gtristannot sure... ybd here...17:46
* gtristan just hit 2 'no space left on device' errors in a row, made some more space now I have 30GB17:47
*** toscalix_ has quit IRC17:49
*** gtristan has quit IRC17:49
*** jonathanmaw has quit IRC18:00
*** bashrc_ has quit IRC18:07
*** rdale has quit IRC18:51
*** Lachlan1975 has quit IRC19:35
*** edcragg has quit IRC19:40
*** edcragg has joined #baserock22:57
*** edcragg has quit IRC23:29

Generated by 2.14.0 by Marius Gedminas - find it at!