IRC logs for #baserock for Tuesday, 2014-09-02

*** flatmush [] has quit [Ping timeout: 245 seconds]00:04
*** flatmush [] has joined #baserock00:05
*** flatmush [] has quit [Ping timeout: 255 seconds]00:43
*** flatmush [] has joined #baserock00:46
*** flatmush [] has quit [Ping timeout: 260 seconds]00:58
*** flatmush [] has joined #baserock01:14
*** flatmush [] has quit [Ping timeout: 245 seconds]01:26
*** flatmush [] has joined #baserock01:32
*** flatmush [] has quit [Ping timeout: 255 seconds]01:36
*** flatmush [] has joined #baserock01:41
*** flatmush [] has quit [Ping timeout: 255 seconds]02:02
*** flatmush [] has joined #baserock02:04
*** flatmush [] has quit [Ping timeout: 260 seconds]02:20
*** flatmush [] has joined #baserock02:21
*** flatmush [] has quit [Ping timeout: 245 seconds]02:30
*** flatmush [] has joined #baserock02:33
*** flatmush [] has quit [Ping timeout: 260 seconds]02:45
*** flatmush [] has joined #baserock02:46
*** flatmush [] has quit [Ping timeout: 245 seconds]02:52
*** flatmush [] has joined #baserock02:54
*** flatmush [] has quit [Ping timeout: 255 seconds]03:04
*** flatmush [] has joined #baserock03:05
*** flatmush [] has quit [Ping timeout: 245 seconds]03:12
*** flatmush [] has joined #baserock03:13
*** flatmush [] has quit [Ping timeout: 260 seconds]03:24
*** flatmush [] has joined #baserock03:26
*** flatmush [] has quit [Ping timeout: 245 seconds]03:34
*** flatmush [] has joined #baserock03:38
*** flatmush [] has quit [Ping timeout: 245 seconds]04:10
*** flatmush [] has joined #baserock04:11
*** flatmush [] has quit [Ping timeout: 255 seconds]04:33
*** flatmush [] has joined #baserock04:33
*** flatmush [] has quit [Ping timeout: 245 seconds]04:47
*** flatmush [] has joined #baserock04:58
*** flatmush [] has quit [Ping timeout: 245 seconds]05:04
*** flatmush [] has joined #baserock05:06
*** flatmush [] has quit [Ping timeout: 245 seconds]05:10
*** flatmush [] has joined #baserock05:11
*** flatmush [] has quit [Ping timeout: 260 seconds]05:20
*** flatmush [] has joined #baserock05:26
*** flatmush [] has quit [Ping timeout: 260 seconds]05:40
*** flatmush [] has joined #baserock05:44
*** flatmush [] has quit [Ping timeout: 260 seconds]05:49
*** flatmush [] has joined #baserock05:52
*** flatmush [] has quit [Ping timeout: 255 seconds]05:56
*** flatmush [] has joined #baserock05:58
*** flatmush [] has quit [Ping timeout: 260 seconds]06:03
*** flatmush [] has joined #baserock06:09
*** flatmush [] has quit [Ping timeout: 260 seconds]06:15
*** flatmush [] has joined #baserock06:20
*** flatmush [] has quit [Ping timeout: 260 seconds]06:30
*** flatmush [] has joined #baserock06:42
*** fay_ [] has joined #baserock06:47
*** fay_ [] has quit [Ping timeout: 245 seconds]06:59
*** fay_ [] has joined #baserock07:00
*** flatmush [] has quit [Ping timeout: 245 seconds]07:06
*** fay_ [] has quit [Ping timeout: 260 seconds]07:10
*** flatmush [] has joined #baserock07:13
*** flatmush [] has quit [Ping timeout: 255 seconds]07:17
*** fay_ [] has joined #baserock07:20
*** flatmush [] has joined #baserock07:27
*** tiagogomes [~tiagogome@] has joined #baserock07:29
*** flatmush [] has quit [Ping timeout: 245 seconds]07:38
*** flatmush [] has joined #baserock07:39
*** fay_ [] has quit [Read error: Connection reset by peer]07:42
*** fay__ [] has joined #baserock07:42
*** franred [] has joined #baserock08:06
pedroalvarezpaulsherwood: I'm interested about what did you try to do with
pedroalvarezI want to make it work08:14
*** ssam2 [] has joined #baserock08:24
pedroalvarezI see, `morph checkout` doesn't work08:29
pedroalvarezseems to be working now08:31
pedroalvarezmorph checkout tries to `git clone` using  git:// protocol, but never falls back to http://08:32
*** jonathanmaw [] has joined #baserock08:42
richard_mawbah, I'll go restart the git-daemon then08:48
pedroalvarezrichard_maw: you mean that g.b.o should be working?08:55
pedroalvarezoh yeah!08:55
pedroalvarezI'm lookin forward to add ca-certificates to baserock if that solves the problems I'm having to use https:// from baserock09:02
richard_mawI think the plan is that today we're going to sort out the issues with the VM09:02
pedroalvarezthat's the plan, yes09:03
franredpedroalvarez, I can add them as part of the gerrit works09:03
persiafranred: I thought java ca-certificates was different from regular ca-certificates09:04
franredpersia, that's what I think too...and because of that Im using its ca-certificates, maybe worth try regular ca-certificates once again in case that I did something wrong09:06
pedroalvarezI can't say anything about ca-certificates. I was raising the point because I know we did some research.09:09
pedroalvarezAnd we never integrate them.09:11
*** franred [] has quit [Remote host closed the connection]09:21
*** franred [] has joined #baserock09:24
paulsherwoodi integrated them. but maybe not in an acceptable way...09:50
pedroalvarezpaulsherwood: did you use the ca-certificates repo that we have in g.b.o?09:51
pedroalvarezahhh! ok09:51
paulsherwoodthis was enough for the docker integration i did09:52
paulsherwoodnote i didn't propose the docker work for mainline because i'm assuming my certs approach is not ok09:57
ssam2I'm not sure what would be OK09:58
bjdooks_ is now known as bjdooks09:58
ssam2maybe lorry should be responsible for updating a Git repo containing the certificates themselves09:58
ssam2the alternative is this sort of approach:
ssam2where the 'build' of the ca-certificates chunk downloads them09:59
ssam2but that seems wrong for Baserock09:59
ssam2effectively we're going to mirror Debian's or Mozilla's certificates, so why not use our mirroring tool, Lorry ?09:59
paulsherwoodok by me10:00
ssam2(disclaimer: I'm dumb, liw-orc and Kinnison no doubt no more :)10:00
ssam2and persia10:00
liw-orcssam2, how dare you say you're dumb?!10:00
richard_mawwe can't download certs at build-time10:01
persiaI believe I already had an approved lorry for ca-certificates, which ought be on g.b.o10:02
persiaThe only blocker I had was getting the system integration script to be robust enough.10:03
persiaThat would be it.10:03
persiaEssentially, one needs to capture the set of certificates on a system, stick that in an enablement configuration file, and then run `update-ca-certificates`.10:03
liw-orclorrying from Debian, building in chunk, integrating in the suitable place -- sounds good to me10:04
persiaI seem to remember there being a missing part in the ca-certificates upstream Makefile that mattered for Baserock but not for Debian, but I don't remember the details at the moment (I'll dig it out of my vm in a bit)10:11
liw-orcI would like to increase the disk space on, but that requires a short service break due to a reboot: is now a good time to do that? I expect it'll take less than half an hour, but will enable people to push again10:29
tlsafine by me10:29
fay__ is now known as fay_10:30
franredliw-orc, works for me10:30
liw-orcpaulsherwood, richard_maw, ssam2, SotK_, franred, pedroalvarez, persia: ^ (I hope I didn't forget anyone)10:30
SotK_go for it10:30
SotK_ is now known as SotK10:30
pedroalvarezliw-orc: go ahead10:31
ssam2please do!10:31
petefoth_liw-orc: I don't mind either :)10:31
liw-orcpetefoth_, oops, I forgot you10:32
pedroalvarezpersia: wrt ca-certificates. Running make and make install. Then populate /etc/ca-certificates.conf with a list of all the files installed in /usr/share/ca-certificates/ and run update-ca-certificates , made the glance client work against an https:/ openstack server10:35 going down in a moment10:35
pedroalvarezpersia: but Ican't clone a repo using a https/ url, so yeah, there is a missing part10:39
persiapedroalvarez: Right.  My memory is that we needed git to have working ssl at build time to get ssl support, but I ran into issues with stratum layering, and never got back to it.10:50
paulsherwoodliw-orc: fine by me too, i assume you've done it by now :)11:16
ssam2$ morph branch baserock:baserock/definitions rubytest --no-git-update11:17
ssam22014-09-02 11:17:18 Updating git repository baserock:baserock/definitions in cache11:17
ssam2each code path has to handle settings['no-git-update'] itself I guess11:17
liw-orcpaulsherwood, it's happening :) the data copying takes longer than I expected, though11:18
persiaIs it possible to consolidate those?  Otherwise we end up with an extending list of things that any new feature needs to support, making changes hard.11:18
ssam2sure, and it partially is11:18
ssam2on closer investigation the code in question should be handling no-git-update11:19
richard_mawit depends on how it's used currently, if you're creating a source pool, e.g. for building, then it already reacts to no-git-update11:19
ssam2seems what it's trying to do in this case is actually clone the repo, because I told it not to update11:19
ssam2pdb in Python 2.7 deals with generators in a confusing way11:30
richard_mawhow so?11:30
ssam2I was trying to work out why a function that did almost nothing was blocking, but it turned out that the code was actually in __exit__() way up somewhere else11:31
ssam2oh, I mean context managers, not generators11:31
richard_mawso the function had exited, and jumped to the __exit__, but since it wasn't part of the function you were inspecting, pdb paused execution?11:32
ssam2or something like that11:32
ssam2i'd moan at pdb, but we are several versions behind11:33
paulsherwoodssam2: i that is correct behaviour - but --no-git-upfdate is a confusing name11:44
ssam2it turns out that the actual problem was somewhere else: a call to `git remote update --prune` that ran unconditionally11:45
paulsherwood--no-git-update refers to checking for uptdatesd to chunks etc on building11:45
liw-orcstatus update: my gbo disk copy is still running and the break may take a good while more, since file copying turns out to be slow due to a very large number of small files11:45
ssam2yes, the name is confusing, I'd sometimes like to have a '--no-network' option11:45
richard_mawor --local11:46
paulsherwoodfor pulling, maybe one of pedroalvarez' new troves would be usable while awaiting the return of gbo11:46
pedroalvarezindeed, I believe this one is usable:
ssam2wow we have an army of imitators11:48
richard_mawssam2: pardon?11:48
ssam2well one imitator11:48
ssam2and by "we" I mean git.baserock.org11:48
ssam2that trove clone is helping, thanks!11:57
paulsherwoodi confess, i want my own.11:58
paulsherwoodand i'd prefer to set my own lorrying policies :)11:58
* persia wants lorry on devel machines : "troves" use too much bandwidth uselessly for single developers12:01
paulsherwoodi second that. i actually did that, and wrote it up :)12:02
persiaI missed that.  Where is the writeup?12:03
paulsherwoodit was last year on the ml, i believe12:03
petefoth_And the possibility to do at least CI on devel machines too? So a devel machine could have the functionality of Trove and Mason12:04
persiapetefoth_: I'm not sure what "CI on devel machines" means, and folk have their devel in all sorts of environments, from memory-limited bare-metal to managed clouds, so it's hard to know where to deploy the test machines.12:05
persiaI'd be delighted with a means to trivially run a test suite against a remote machine that I had just deployed.12:06
* ssam2 would like it if Trove and Mason were lightweight enough that they could be run on a developer machine12:06
ssam2but either approach is an improvement12:06
petefoth_persia: "automated build and running (some sub-set) of system tests"12:07
persiaI actively don't want automated builds on devel machines.12:07
persiaAs a developer, I know better than any automation when to perform a build.12:07
petefoth_persia: really?12:08
persiaYes.  Any automation just means I have to know the magic to trigger it, and it's conceptually easier to just type `morph build`12:09
petefoth_persia: so you build the system you are wrking on. Might you not want to build other systems to check your changes haven't broken them?12:10
petefoth_Why should you have to have another machine to do that?12:11
persiaOnly in the rare case that I happened to be building a development system, and in such a case, I'd want to do that by deploying it, and running a test suite against it (which test suite might involve building clusters, etc.)12:11
persiaBut I never want to wonder whether something is going to try to magically deploy something (probably in the wrong place) or consume resources testing it (probably at the wrong time).12:12
petefoth_persia: I'm not suggesting that the automated build and test happen without you triggering it. But I am arguiant that you shoyuld be able to trigger it without having to have another machine to do it on12:13
persiaI don't need another machine to do that: I need a way to host an environment.  I can use a VM on baserock, a VM hosted on the same machine or in the same cloud as a devel VM, a second piece of target hardware (if I only have 1G RAM, I *really* don7t want to run a VM in there and try to build stuff inside), etc.12:13
persiaYes, but that flexibility belongs in `morph deploy`.12:13
persiaSo my workflow is something like `morph build ${SYSTEM}`, morph deploy ${CLUSTER};, morph test ${INSTANCE}`12:14
paulsherwoodpersia: turns out i published my work, but not the discussion12:14
persiaThere's no need for automation to achieve this.12:14
persiapaulsherwood: Ah, which may be why I missed it :)  Is it as simple as just adding a lorry chunk to a devel environment?12:14
paulsherwoodi also added a morrph lorry command
petefoth_MY use case is: I am modifying a component that is used in many BAserock systems. I want to be able to check that all affected systems can a: still build andd b: pass some subset of tests12:15
persiapetefoth_: Can you not use the workflow I just described to do that, where your test suite happens to build systems and test them?12:16
petefoth_persia: yes, but isn't that Mason functionality - or a useful sub-set thereof?12:17
persiaI don't have sufficient semantics attached to "Mason" to consider things in relation to it.12:17
persiaBut I don't want any automation or overhead on my devel machines.12:18
* petefoth_ gives up12:18
persiaFrom what I understand about Mason, I suspect there's a lot of parallels between what I want on a developer machine and what Mason does, but the architecture may differ significantly.12:18
persiapaulsherwood: Looking at that: do we need to wrap lorry in morph?  Seems quite a lot of changes to morph, as opposed to my naive believe we might be able to just run some `lorry` command.12:21
*** gallit [] has joined #baserock12:24
*** thecorconian [] has joined #baserock12:37
paulsherwoodyou may be right.12:58
*** thecorconian [] has quit []12:58
persiaThat it needs to be in morph, or that it doesn't need to be in morph?12:59
* persia hadn't intended to express either as a declarative statement12:59
paulsherwood'we might be able to just run some lorry command'13:00
persiaDoes such a command exist already?13:00
persiae.g. `lorry ${FILE}` to pull something into some known location?13:01
paulsherwoodyes. just not in  devel. there was discussion on our internal list when i did this is there a 'normal' way to introduce historical private discussion to a public list (assuming participants agree to the publication)13:02
persiaI don't know of any good ones (that preserve threading, etc.).  I generally see people summarise things, inviting futher comment.13:03
paulsherwoodi'll just restart as a new topic, then13:08
paulsherwoodlots has changed13:08
persiaProbably makes more sense13:11
*** franred [] has quit [Quit: Leaving]13:38
*** franred [] has joined #baserock13:39
*** fay_ [] has quit [Quit: Leaving]13:49
*** fay_ [] has joined #baserock13:50
straycatThough we tend to lorry with lorry controller because lorrying manually is too dangerous with the current tools.13:55
paulsherwoodstraycat: dangerous?13:55
persiaHow so?13:55
paulsherwoodall i'm suggesting is user can use it to efficiently get a git repo of non-git upstream13:55
persiaThe alternative being to install git-svn, git-hg, git-cvs, etc.13:56
straycatlorry, run as it stands on a trove is destructive13:57
paulsherwoodthis wold be on a devel machine13:57
paulsherwoodfor occasional use by integrators needing jrandom nongit upstream13:57
persiaFor both integration and for testing lorry files before submission to a lorry-controller instance.13:59
persiaBut that lorry run on a trove is destructive worries me: how do we ensure that refs we care about aren't clobbered?14:00
liw-orcstraycat, what do you mean by destructive?14:00
straycatliw-orc, iirc refs from the new repo overwrite any matching refs in the current repo.14:01
persiaYou mean in case of collision?14:03
liw-orchow is this a problem when a developer runs lorry on the devel system to create git repos that are _not_ pushed by lorry to a trove?14:03
straycatIt's not14:04
straycatI didn't realise you were planning to run lorry on a devel machine, sorry14:05
liw-orcno worries, then :)14:05
ssam2Nothing makes my day more than a library which fails to build in Morph, but succeeds to build when I try to build it manually14:55
ssam2hmm, it definitely seems to be running the tests before it's finished building the library14:55
ssam2but it also definitely seems to be setting 'MAKEFLAGS=-j 1', as I told it to14:56
ssam2and I just can't make it break outside morph, even running the identical commands14:57
ssam2maybe identical environment will make it break14:57
persiaHave you tried to build it in the chroot in the failed/ directory?14:58
paulsherwoodssam2: maybe it needs network?15:00
ssam2i'm trying in the chroot and using linux-user-chroot15:00
paulsherwood(go runs tests which need network, which fail  in morph)15:01
richard_mawyou don't even get loopback network in linux-user-chroot15:01
paulsherwood(because we turn of network)15:01
ssam2I get test failures interspersed with the building status messages when Morph builds it15:02
ssam2but manually, I get all the building messages, followed by all the tests succeeding15:02
richard_mawit's concievable that we could enable loop-back networking in builds, which would probably be sufficient for go's tests15:02
ssam2ah! if I do `MAKEFLAGS='-j 6' make` then I can reproduce the failure.15:03
richard_mawhooray for non-deterministic build failures(!)15:03
paulsherwoodrichard_maw: i opted for building witohut the tests15:05
ssam2i realise now that when I added a chunk morph, I didn't add the 'morph: strata/chef/yajl.morph' entry necessary to make Morph actually look at my chunk morphology15:06
richard_mawssam2: out of interest: do you think you would have made that mistake if you could have in-lined the max-jobs into the stratum morphology?15:06
ssam2no, i'd definitely have inlined it :)15:07
paulsherwoodi see the way this discussion is going :/15:13
persiaDoes morph output advise whether it's using a local morphology or discovering one?15:14
richard_mawpersia: not directly, it may output information related to the fact that it is updating git repositories so it can find the morphology15:15 should be back online15:15
richard_mawweb UI is15:16
richard_mawand snappy15:16
liw-orccould someone try to push something?15:16
persialiw-orc: Thanks!15:16
persiarichard_maw: Hrm.  I think we probably want to know more directly, so that one can see the output and understand why the build fails miserably more quickly than ssam2 took today.15:17
richard_mawliw-orc: push ok15:17
liw-orci will clean up the ghost jobs15:19
richard_mawpersia: unless we have a big warning message saying that our stratum hasn't included a path to a chunk morphology, I don't think it will have helped, as we produce so much output that any normal message would easily get lost15:22
richard_mawand we'd get far too many false warnings with that scheme anyway, since we auto-detect a lot of morphologies15:23
persiaI think I agree with you.  I still think being able to differentiate autogenerated from provided morphologies is important.15:23
persiaPerhaps we need to trim other things, but I'm not convinced we should stop adding useful things just because we're generating so much useless stuff that it's better ignored.15:23
richard_mawI'd prefer to make the "morph" field mandatory, and move the build-systems into the definitions repository, though that will require us removing the "name" field from the chunk morphology15:25
ssam2given an interface of text files containing yaml, I will find a way to confuse myself no matter what the exact structure :)15:26
richard_mawssam2: you want a GUI?15:27
paulsherwoodliw-orc: new jobs are not allowed, i assume that means it's not lorrying now?15:27
liw-orcpaulsherwood, jobs are disallowed until I've cleaned out the ghost jobs15:28
persiarichard_maw: I can see that being attractive, but 1) I still don't want my screen flooded with useless things, 2) I still want morph to tell me when it decides to do magic things, rather than being mysterious about it, and 3) the more we move to definitions, the less one can use morph without definitions.15:29
persiaWhile some of this stuff may not belong in morph, I'd much rather not have definitions hugely cluttered (I'm already unhappy that I can't easily understand all the content).15:30
persiaIdeally I'd like to be able to start a definitions repo from scratch, and expect things to mostly work.15:31
*** gallit [] has quit [Quit: Leaving]15:31
paulsherwoodpersia: me too. what's stopping us?15:32
richard_mawnothing, as I understand the problem15:33
persiapaulsherwood: I'm not sure.  There was the idea of moving all the configuration extensions there, to make them easier to discover (but increase the footprint of a "bare" definitions).  Moving build systems would be more.  There's currently some scripts, some of which may be useful or important.15:34
ssam2richard_maw: i can confuse myself using complex GUIs too15:34
ssam2it's quite a talent I have15:34
persiaIt's the dataset that is confusing, not the interface.15:34
liw-orcthere, jobs are allowed again15:38
paulsherwoodliw-orc: tvm. i assume they'll kick in when the schedule catches up?15:41
paulsherwood112GB :)15:41
liw-orcpaulsherwood, yep, they'll kick when it's time15:42
*** tiagogomes [~tiagogome@] has quit [Ping timeout: 260 seconds]15:55
*** fay_ [] has quit [Remote host closed the connection]16:01
paulsherwoodis php in baserock?16:22
paulsherwooddo we want it?16:22
paulsherwoodor do we hate it so bad we'll never allow it? :)16:22
richard_mawonly as long as we have a paying customer, otherwise I think we prefer non-php solutions16:23
paulsherwoodthere was discusion on another channel about how to setup an openid provider, and this came up... http://simpleid.koinic.net16:24
richard_mawthere's also myriad openid server applications listed on
persiarichard_maw: Do you have a preferred one from that list?16:25
richard_maw since it's python16:26
*** franred [] has quit [Quit: Leaving]16:27
persiaLooks like it hasn't been touched in 8 years16:27
* paulsherwood 'did' django in python recently, in case anyone needs that16:28
*** mwilliams_ct [] has joined #baserock16:28
richard_mawpersia: if it works with Gerrit, I'm not sure we care16:29
persiaThe only other standalone app in python from that page is 5 years untouched and described as "needing cleanup".16:30
paulsherwoodand storyboard, richard_maw :)16:31
persiaSo I suppose DjangoID is reasonable, although my preference is to have an active upstream for anything I use (if only to have someone to chat with when I run into issues).16:31
paulsherwoodhas some more recent activity, at least16:45
persiaIsn't that just a consumer?16:46
paulsherwoodmaybe. i skipped lunch, my brain is not present16:47
persiaLooks like a consumer in that example (and moreso reading the exposed API)16:48
persia (near the bottom)16:48
ssam2richard_maw: do you want me to merge your 'Use git's remotes to determine unpushed branches' patch ?16:49
ssam2I've reviewed and it looks good16:49
richard_mawI'm currently running tests on it to ensure merging didn't introduce a regression16:50
ssam2oh, me too16:50
ssam2i'll race you16:50
richard_mawI just finished16:50
persiaIs the winner determined only by time, or by number of tests performed as well?16:50
pedroalvarezonly by time assuming they were running the same test suite16:51
richard_mawthe commit sha1 is aea1029044b7e0d4578f3896bf85898f33791c89 if you want to do the definitions.git update16:51
richard_mawssam2: ^16:54
ssam2i'm happy for you to :)17:00
*** ssam2 [] has quit [Quit: Leaving]17:01
*** jonathanmaw [] has quit [Quit: Leaving]17:01
pedroalvarezmorph controllerdaemon --config /etc/morph-controller.conf17:03
pedroalvarezERROR: [Errno -2} Name or service not known17:03
*** flatmush [] has quit [Quit: Leaving.]17:31

Generated by 2.15.3 by Marius Gedminas - find it at!