IRC logs for #baserock for Monday, 2014-09-15

*** jjardon_ [sid723@gateway/web/] has joined #baserock00:00
*** persia [quassel@2400:8900::f03c:91ff:feae:3452] has joined #baserock00:01
*** persia [quassel@2400:8900::f03c:91ff:feae:3452] has quit [Changing host]00:01
*** persia [quassel@ubuntu/member/persia] has joined #baserock00:01
*** cyndis_ [] has joined #baserock00:03
jjardon_ is now known as jjardon00:14
*** jjardon [sid723@gateway/web/] has quit [Ping timeout: 271 seconds]00:29
*** jjardon_ [sid723@gateway/web/] has joined #baserock00:30
*** petefotheringham [] has quit [Ping timeout: 272 seconds]00:40
*** petefotheringham [] has joined #baserock00:57
*** fay [] has joined #baserock07:35
fay is now known as fay_07:35
*** SotK [] has joined #baserock07:41
jjardon_ is now known as jjardon07:54
*** jjardon [sid723@gateway/web/] has quit [Changing host]08:04
*** jjardon [sid723@gateway/web/] has joined #baserock08:04
*** flatmush [] has quit [Read error: Connection reset by peer]08:10
*** flatmush [] has joined #baserock08:12
*** ssam2 [] has joined #baserock08:14
Kinnisonpaulsher1ood: I'll make a point of asking around today to see if anyone fixed it without telling the channel08:18
*** franred [] has joined #baserock08:23
* straycat restarted thte git service08:28
straycatIt was using 5GB of memory, I don't know why08:29
KinnisonCould you please, should you do that again, drop a mail to -dev explaining what you did.  It might be of use to any other Trove owners who have issues08:29
*** tiagogomes [~tiagogome@] has joined #baserock08:33
*** jonathanmaw [] has joined #baserock08:37
*** benbrown_ [] has joined #baserock08:44
*** violeta [] has joined #baserock08:48
paulsher1oodbut thanks, straycat for fixing it :-)08:58
KinnisonAye, thanks straycat09:01
KinnisonToday I am annoyed with the Linux OOM killer, if anyone has good references for how to tune it, I'd appreciate links.09:01
ssam2 may help?09:04
rjekSimply turn off over-commit! :)09:04
rjekAnd watch programs totally fail to cope when malloc() returns NULL09:04
ssam2a while back I disabled overcommit and swap on my laptop, cus I was sick of it dying swap-deaths all the time09:04
ssam2it actually works pretty well now, as long as I remember to exit Firefox before I start a big VM09:05
KinnisonThanks for the lwn link ssam209:05
ssam2another one:
* Kinnison shall look it over09:05
rjekI remember Mozilla being a big pro for over-commit because it used to be that it only actually touched half the RAM it allocated.09:06
ssam2I note that I've not tried that solution on my personal laptop which only has 1GB of RAM09:06
straycatKinnison, so the oom killer was killing the process, shouldn't there usually be something logging that in dmesg?09:06
Kinnisonstraycat: it did log in dmesg, but I originally only saw the OOM killer killing a cc1plus process09:07
Kinnisonstraycat: the killing of morph in addition was harder to spot09:07
*** ridgerunner [] has joined #baserock09:08
straycatAre we still talking about gbo?09:08
KinnisonNo we're talking about something else09:09
Kinnisona distbuild worker09:09
Kinnisonbuilding QtDeclarative09:09
straycatOkay :)09:09
*** dutch [] has joined #baserock10:42
pedroalvarezhm... I just had a nice systemd loop when rebooting a fresh trove :(11:01
pedroalvarezbut I know why :)11:02
paulsher1oodssam2, richard_maw - my issue with raising the btrfs problem is i don't understand the tech details well enough, let alone the ml protocol :)11:04
pedroalvarezThis commit should fix the trove problem I'm having:
ssam2paulsher1ood: me neither :) I'm happy to send the problem report, though11:08
straycatWhy do troves deployed to openstack need an initramfs?11:21
pedroalvarezstraycat: they dont need it.11:29
pedroalvarezopenstack uses virtio disks, so they use /dev/vdX instead of /dev/sdX11:30
pedroalvarezthe openstack write extension replaces that in extlinux.conf11:31
pedroalvarezbut if you want to deploy to a rawdisk image, then you have to add initramfs to the system11:31
straycatOh I see okay11:34
straycatSo if I deploy my trove with the openstack extension I can skip adding the initramfs subsystem?11:47
pedroalvarezstraycat: yes11:47
straycatOkay cool11:48
ssam2idle thought: it might make 'what Baserock is' a bit clearer if we were to clearly separate the reference systems into 'examples' and 'infrastructure'12:25
ssam2so Trove, Mason and Devel are infrastructure12:25
ssam2and everything else is an example12:25
*** dutch [] has quit [Remote host closed the connection]12:27
*** CT-Dutch [] has joined #baserock12:31
*** violeta [] has quit [Remote host closed the connection]12:55
franredI need a short review updating morph in definitions:
pedroalvarezfranred: +112:56
ssam2franred: +113:05
franredcheers, guys :)13:05
paulsher1oodssam2: +1 on your idea13:25
*** ctgriffiths [] has joined #baserock13:26
ctgriffithsHello, I heard there was some trouble lorrying MariaDB.  Is it a definite no go?13:29
rjekIt seems to be in bzr, which I thought was supported.13:30
rjek(The stuff in BR that I'm upstream for is in bzr.  So I imagine it has to be.)13:30
paulsher1oodit was big, that was all. when i tried to push it through, there was not enough free space on gbo13:32
paulsher1oodshould be doable now, i believe13:32
KinnisonMariaDB? Oh dear13:56
KinnisonDebian's having a lot of trouble with MariaDB right now13:56
KinnisonIt's an unmergeable fork of MySQL13:59
rjekIt's the fork that happened when Oracle bought MySQL.14:00
KinnisonWhy do we want mariadb ?  We already have postgresql available14:01
rjek(Because nobody wanted to deal with Oracle.  Also see: OpenOffice)14:01
rjekKinnison: Some software is written by the unenlightened.14:01
Kinnisonrjek: aye, but do we want unenlightened software? </purist>14:02
ctgriffithsKinnison: storyboard uses doesn't support postgres currently, it uses MySQL so we want to use MariaDB as a drop in replacement14:02
Kinnisonctgriffiths: storyboard? Oh.  urgh.  I see.  Can't we fix Storyboard?14:02
KinnisonIt's surely not *that* DB-heavy14:02
Kinnisonand IIRC it's written in Python14:02
persiaUpstream used to support both Postgres and MySQL, and migrated to MySQL only some time ago14:03
KinnisonHow strange14:03
persiaSomething about table locking semantics, iirc14:04
Kinnisonpsql is ACID and MySQL is not?14:04
KinnisonFor something like storyboard I fail to see how the performance impact of full ACID could be a detriment though14:05
rjekDoes MySQL support cell-based locking yet?  Last time I looked into this it only supported table-based (not even column-based)14:05
persiaBug me on a day when my brain is working, and I'll dig up the relevant meeting logs.14:05
Kinnisonrjek: iIRC it depends on the engine you use14:06
rjekAlthough the thing to remember about MySQL is that it's actually a dozen different databases with different feature sets that share an average-quality SQL interpreter.14:06
straycathey persia :)14:06
Kinnisonrjek: myisam is fast but lacks features.  innodb is much more featureful but certainly used to be slower14:07
rjekSo you often end up with "Yes, MySQL has these features!  But not at the same time!"14:07
straycatHrm, so at the moment there's a MASON_ID, which gets used by a gitano post receive hook to send notifications to a mason, except that at the moment mason isn't using that trigger it just runs periodically, so at the moment MASON_ID doesn't need to be set? If that's true then we probably shouldn't mention it in the docs.14:09
Kinnisonstraycat: the mason stuff in trove was from when trove triggered the ci controller, we'd like to get back to that at some point, but probably by having the ci controller subscribe at runtime, so we are likely to be able to drop the configuration variables entirely if they're optional now14:10
straycatOkay, for now I'm content to drop mention of MASON_ID from the docs, but if you're sure we won't want it in the future I could also remove it from the code?14:13
KinnisonLeave it in the code for now14:13
KinnisonI've not even started to write the server-side support for subscriptions, and if we move to Gerrit, who knows what we'll do14:14
SotKhas anyone seen "/usr/include/glib-2.0/glib/gtypes.h:32:24: fatal error: glibconfig.h: No such file or directory" when building something which depends on glib in baserock before?14:14
Kinnisonctgriffiths: You said you'd heard there was an issue with mariadb lorrying -- do you have any more data than that?14:14
ssam2SotK: not seen it in Baserock, but have seen it14:15
radiofreeSotK: i've seen it before outside baserock14:15
ssam2is the build system using `pkg-config --cflags glib-2.0` correctly ?14:15
radiofree/usr/lib/glib-2.0/include isn't in your include path14:15
paulsher1oodKinnison: i tried to sneak in the mariadb lorry - but gbo was short of resources at the time14:16
* Kinnison tsks paulsher1ood 14:16
* Kinnison shall take a peek14:16
SotKssam2: I doubt it14:16
paulsher1oodKinnison: this was months ago.14:16
paulsher1oodi haven't tried since14:16
paulsher1oodi was tskd at the time14:17
* Kinnison tsks you more14:17
paulsher1oodthere is the point that maybe all downstream troves don't want everything from gbo. can they disable repos easily?14:18
paulsher1ood(we're in danger of adding java and its friends, for example)14:18
SotKssam2, radiofree: thanks :)14:18
richard_mawpaulsher1ood: downstream troves can add an 'ignore' list to the lorry-controller config14:21
richard_mawpaulsher1ood: unfortunately, it's opt-out, rather than opt-in, so they need to set that up before we add them14:21
paulsher1oodrichard_maw: can they scru repos they don't need?14:22
paulsher1oodotherwise they need to be prescient?14:22
richard_mawpaulsher1ood: if we're adding bug stuff we could notify downstreams and ask if they want $LONG_LIST_OF_REPOS, rather than being prescient14:23
richard_mawpaulsher1ood: there's no lorry-controller interface for removing lorried repos, so you'd need root on the box14:24
richard_mawpaulsher1ood: but it is possible to scrub repos you don't need14:24
paulsher1oodok. maybe we need a better default14:24
richard_mawwe might be able to add java as another prefix (e.g. java/ instead of delta/)14:33
richard_mawwhich the default trove config won't include14:33
richard_mawbut we'd need to fiddle with the gitano config to make lorry able to push there14:33
Kinnisonrichard_maw: no fiddling required14:34
Kinnisonrichard_maw: lorry-controller would need a new stanza for that subsection, but otherwise it'd just work14:35
Kinnisonrichard_maw: and that's just an entry in lorry-controller.conf in the local-config repo14:35
richard_mawI thought the gitano rule-sets had entries saying "only lorry may push to delta/" and "lorry may not push to $TROVE_PREFIX"14:36
Kinnisonpaulsher1ood: FYI, the mariadb import is currently pondering october 200314:36
Kinnisonrichard_maw: Nope, the rulesets are "may push to ¬ prefix"14:37
Kinnisonrichard_maw: and may not touch refs starting with prefix14:37
Kinnisonrichard_maw: otherwise lorry wouldn't be able to work for downstream troves14:39
Kinnisonpaulsher1ood: and now it's pondering october 200414:43
Kinnisonpaulsher1ood: at this rate, (roughly 6 minutes per year) we can expect it to complete the export from bzr in around an hour14:44
paulsher1oodthat's not completely awful14:45
* paulsher1ood would prefer to see storyboard with postgres, though...14:45
* Kinnison concurs14:45
paulsher1oodbut that's probably orthogonal :)14:45
Kinnisonanything mysqlish makes me antsy because it's so much less nice to admin IMO14:45
Kinnisonbut I am biassed14:45
* persia will dig up the reason why it was made to not support postgres anymore tomorrow14:45
Kinnisonpersia: thanks, concentrate on getting fully well first :-)14:46
persiaIt uses sqlalchemy for code-admin14:46
* richard_maw finds concentrating is the last thing he wants to do while trying to get well14:46
Kinnisonrichard_maw: You mean you don't lie in bed thinking "Be well, citizen" to yourself over and over and over?14:47
richard_mawKinnison: no, it's generally stranger things like "hm, I didn't send the impulse to move that muscle. Ah, there must be someone else in here too now. Bah, they'll want equal access to the eyes, but I want to keep them closed"14:49
Kinnisonoh my14:49
richard_mawwhich then leads on to weird spirals like "Do I have the right to deprive the other intelligence inhabiting my body access to all its functions"14:51
SotKanyone here built glibmm?14:52
richard_mawis that the C++ binding?14:52
radiofreeabout 5 years ago SotK14:52
radiofreeso i doubt i'll be much help14:53
* paulsher1ood is comforted to see persia back on irc, even if not at full power14:54
KinnisonHmm it has slowed down, it's still on august 200514:55
paulsher1oodmaybe they did more work in 2005?14:55
* straycat is struggling a little with the docs14:58
straycatWhat is $FLAVOUR here?
Kinnisonpresumably it's the kind of VM14:59
straycatI have no idea14:59
Kinnisonflavour is an openstack term14:59
straycatThat's nice14:59
straycatThe doc says nothing14:59
KinnisonOur deployment doc cannot tell you what flavours are available on your cloud15:00
KinnisonWe should perhaps discuss what shape of flavour would be best though15:00
Kinnisone.g. in terms of vCPUs etc15:00
straycatIt could tell me what a flavour is though?15:00
KinnisonI think there's an assumption that if you're doing an Openstack deployment, you know what nova, horizon, flavour, etc all mean15:00
Kinnisonotherwise we end up duplicating openstack's glossary15:00
paulsher1oodstraycat: this is a bit like ridgerunner's q the other day. we can't document all the stuff baserock interacts with. i think we have to assume if folsk are deploying to openstack, they have some understanding of openstack15:01
straycatI'm sorry I disagree.15:01
straycatYou've got random magic in these docs that isn't even slightly explained.15:01
straycatI'm not expecting a duplicate of the glossary.15:01
KinnisonI'd be up for ensuring that we link to the openstack glossary15:01
Kinnisone.g. when we say "flavour" we should link that to the definition in Openstack's site15:02
straycatI'm sorry that I failed to phrase my criticism in a more positive light.15:02
Kinnisons'okay, it can be very frustrating15:02
KinnisonI still don't know what all the terms mean15:03
Kinnisonbut I've learned to identify openstack terms so I know where to look15:03
straycatI know how to use google, but I don't expect to have to when I'm following a guide.15:03
KinnisonIf you want to, you could annotate the doc with links out to appropriate sites as you go.  I appreciate this doesn't help you now, but it would help someone else in the future.15:04
persiaIn case anyone is interested in storyboard, there's an upstream meeting in #openstack-meeting just starting now15:05
paulsher1ood+1 - i fix the wiki when i find things broken or unclear. so far i've not had to deploy to OpenStack15:05
straycatpaulsher1ood, *nod* I do small fix ups, but as someone who's never touched openstack this guide is helpful, but could be a lot clearer, I know it's difficult to write clear docs when you already know how everything works.15:08
straycatI remember that the original trove deployment guide I wrote suffered from similar problems.15:09
*** genii [~quassel@ubuntu/member/genii] has joined #baserock15:09
straycatAlso, the cloud-init script seems to generate my admin key, is the expectation that I copy the private key off the system to use it?15:10
KinnisonTrove-admin key should be supplied to the system, surely?15:11
KinnisonTrove will create the gitano-admin key on-board and that shouldn't really be used outside of the unit15:11
straycatIt doesn't seem to be in this case, but I could be mistaken.15:11
Kinnisonpedroalvarez: Can you weigh in on this, you wrote the cloud-init stuff15:12
pedroalvarezThat cloud-init script is an example to configure a trove. That script generates all the keys needed (admin, lorry, worker...).15:14
pedroalvarezso if you want to inject your key for the admin user, you can easily modify the cloud-init script15:15
Kinnisonpedroalvarez: I wonder if it would be good to provide an example where you supply the trove-admin key15:16
Kinnisonpedroalvarez: otherwise people might get the wrong idea15:16
*** genii [~quassel@ubuntu/member/genii] has quit [Remote host closed the connection]15:19
pedroalvarezshould be like this:
pedroalvarezKinnison: a lot of different things can be done in OpenStack, and in many different ways15:20
KinnisonBut making sure we put Good examples in, is important15:20
straycatpedroalvarez, cheers I'll put that example in15:27
Kinnisonstraycat, pedroalvarez: awesomesauce15:29
Kinnisonpaulsher1ood: up to 2009 now15:30
pedroalvarezstraycat: np! :)15:31
*** thecorconian [~thecorcon@] has joined #baserock15:44
Kinnisonpaulsher1ood: july 2011 and nearly 50G of temporary space consumed15:56
Kinnisonpaulsher1ood: no wonder mariadb import failed before15:56
straycatSo what would we recommend for a trove, m1.medium? Plus some separate disk allocation for gits and artifacts?15:56
Kinnisonpedroalvarez: ^^ ?15:57
straycat is all we need to discuss it15:58
pedroalvarezstraycat: depending on your openstack provider you may  have more flavor available15:59
straycatSure, but those are the defaults are they not?15:59
franredstraycat, why medium? Small should be enough if you attach a volume16:00
pedroalvarezfranred: but small only have 1 VCPU16:00
straycatplus, you need enough memory to be cloning git and a bunch of other things and running a bunch of web and git services.16:01
KinnisonRAMs are important16:01
rjekThe inflexibility of "flavours" is something I really don't like about Openstack.16:01
*** ssam2 [] has quit [Ping timeout: 246 seconds]16:01
rjekI had this problem with our internal one; none of them were ideal for my need.16:01
franredhow much RAM is consuming a Trove instance?16:01
pedroalvarezrjek: did you ask your sysadmin to add one more suitable for you?16:02
rjekpedroalvarez: It seems, to me, silly to need to have an admin make a whole new "flavour" for one thing.16:02
rjekI just want to be able to say "I want a machine with X MB of RAM and Y GB of disc space and Z CPUs.  Go."16:03
*** CT-Dutch [] has quit [Quit: Quit]16:03
pedroalvarezstraycat: I think the medium is not enough16:06
KinnisonWe're almost there.16:07
*** jmacs [] has joined #baserock16:08
pedroalvarezbut just because of the disk space. If you plan to attach a volume, then I think that the medium flavor is enough16:09
Kinnisonpaulsher1ood: it's packing....16:10
Kinnisonpaulsher1ood: 58G of tempspace in use right now16:10
paulsher1oodKinnison: please approve anand's email?16:13
Kinnisonpaulsher1ood: already done, slowpoke :-)16:13
*** ssam2 [] has joined #baserock16:14
KinnisonJust over 10% of the import sequence done now16:17
KinnisonI fear this repo is going to be hyoooge16:17
paulsher1oodKinnison: email the list notifying we may delete it, so don't rely on it yet?16:18
KinnisonOnce it's in, I don't think we should stop lorrying it.  Other stuff might need it in the future16:19
Kinnisonthe pain is, after all, one off16:19
Kinnisonca. 20% imported16:20
*** franred [] has quit [Quit: Leaving]16:20
straycatActually I take back what I said, the whole thing assumes you've got nova sorted so you'd probably also assume the reader knows about flavours.16:23
straycatIt's just confusing for me because I've never used openstack.16:23
*** tiagogomes [~tiagogome@] has quit [Quit: Leaving]16:32
Kinnisonsuch slowness16:35
Kinnisonvery big repo16:35
*** jonathanmaw [] has quit [Quit: Leaving]16:35
richard_mawhi pedroalvarez, I have more quoting questions to your UPSTREAM_TROVE patch16:40
pedroalvarezrichard_maw: one sec, I'm reading your email first16:41
pedroalvarezrichard_maw: go ahead16:42
pedroalvarezanswering your question: No, I haven't tested the JSON parsing with weird input :/16:43
richard_mawat that point I think it's YAML parsing16:43
richard_mawwhich just dropping the |quote would fix16:43
richard_mawsince AIUI the interpolations happen after the yaml is turned into objects16:43
pedroalvarezI'll do a quick test dropping 'quote'16:44
pedroalvarezrichard_maw: suggestions for weird input?16:46
*** genii [~quassel@ubuntu/member/genii] has joined #baserock16:46
richard_mawstick a " in it16:47
pedroalvarezrichard_maw: I guess I  have to define that var like:16:49
pedroalvarezUPSTREAM_TROVE: "something with weird chars like \" on it"16:49
paulsher1oodssam2: is it me, or does anand's patch not apply?16:50
pedroalvarezthe result in that case is:
pedroalvarezrichard_maw: so yeah, quote is not needed in this case, thanks!16:52
* paulsher1ood fudges anand's patch, kicks off a build16:52
richard_mawpedroalvarez: no problem16:54
richard_mawit's not a valid hostname, but I'd much prefer the error be noticed there than cause weird bugs earlier when generating or loading the config16:55
paulsher1oodssam2: anand's patch (fudged) works.maybe if i submit the working diff, i can get credit for the patch? :-)17:03
KinnisonStiiiil converting17:05
richard_mawpaulsher1ood: I don't think you can sneak that past the maintainer, when he pointed you to Anand Jain in the first place.17:06
ssam2paulsherwood: good that it works, can you reply and tell him? I need to head off17:10
ssam2actually I can, it takes 1 minute17:10
*** ssam2 [] has quit [Quit: Leaving]17:15
KinnisonOkay, i think the mariadb conversion completed but you might have to wait for it to be transferred17:26
* Kinnison heads off17:26
*** genii [~quassel@ubuntu/member/genii] has quit [Remote host closed the connection]21:37
*** ridgerun1er [] has joined #baserock23:46
*** jjardon_ [sid723@gateway/web/] has joined #baserock23:53

Generated by 2.15.3 by Marius Gedminas - find it at!