IRC logs for #baserock for Wednesday, 2015-07-22

*** zoli__ has joined #baserock04:57
*** paulw has joined #baserock06:09
*** mike has joined #baserock07:18
*** mike is now known as Guest5677007:18
*** mariaderidder has joined #baserock07:46
*** mdunford has joined #baserock08:04
*** franred has joined #baserock08:08
*** edcragg has joined #baserock08:32
*** jonathanmaw has joined #baserock08:57
*** franred has quit IRC09:03
*** franred has joined #baserock09:09
*** lachlanmackenzie has joined #baserock09:20
pedroalvarezIs there a way to disable persistent device names in baserock?11:29
tiagogomes_why would you want to do that?11:33
pedroalvarezbosh, when creating a stemcell, tries to do that11:39
pedroalvarez# Remove persistent device names so that eth0 comes up as eth011:39
pedroalvarezrm -fr $chroot/etc/udev/rules.d/70-persistent-net.rules11:39
persiaWhat an ugly hack11:40
pedroalvarezthis is full of ugly hacks :)11:40
persiaIt would be better to track down what consumes the "eth0" string, and cause that to parse the devices more sensibly.11:40
persiaI seem to recall that for the reference systems, there was some initial effort to cling to "eth0", and then everything got changed so that it wasn't required anymore.11:41
persiaAnd I suspect that every other project needs to make the same transition, for which patches are mostly a matter of parsing the network devices, rather than blindly assuming "eth0" works.11:41
persia(even without persistent device names, there are a number of circumstances in which one has working networking and no working "eth0")11:42
franredpedroalvarez, wasn't systemd the one which names the network devices? (latest version)11:42
pedroalvarezpersia: fixing this to remove that hack makes sense, if I can figure out why was this needed11:43
pedroalvarezfranred: yes11:43
franredalso have you check if they create the net rules in a different file? - maybe this is why they remove the default persistent rules11:44
persiapedroalvarez: For most cases of this problem, grep seems to find the offending code.  Essentially, there should be no code anywhere that expects "eth0" to be a meaningful strong, so any appearance outside a comment is likely a bug (and most appearances inside comments are likely to also be pointing at bugs)11:44
persias/strong/string/11:44
* franred though you could also overwrite udev rules depending on the precedence of them without having to remove the default ones11:45
persiafranred: You can, but that doesn't fix buggy code that assumes the primary network adaptor is named "eth0"11:46
franredpersia, true, I wasn't thinking in buggy code... just in the udev configuration files11:47
persiaUnless you are stuck dealing with closed-source code, it is often better to fix the bug than work around it :)11:48
pedroalvarezexpect(ssh(public_ip, 'vcap', '/sbin/ifconfig eth0', @our_ssh_options)).to match /#{static_ip}/11:49
persiaUgh.  I don't know if the tools are typically there, but I'd expect something parsing `ip -o addr` provides the desired data.11:51
radiofreeyikes11:52
radiofreewhat is this pedroalvarez?11:52
radiofreecloudfoundry?11:52
pedroalvarezradiofree: indeed11:52
persiaAmusingly, on the system I used to try to find a useful command, there is no eth0 nor any en* interface, and this doesn't even use the new systemd11:53
radiofreeon fedora my ethernet is "em1"11:53
pedroalvarezbaserock systems on openstack have eth0 :)11:55
persiapedroalvarez: Yes, but is this a good thing?11:59
pedroalvarezis a bad if I want to test a change of that code12:00
pedroalvareznot exactly good, but somehow good if I only want to add support to create baserock stemcells for openstack12:02
persiaOh, heh, because you can ignore the issue entirely.12:03
franredat the moment at least, it can beat us in the future12:04
pedroalvarezor  even in a different openstack environment.. who knows12:06
tiagogomes_mmm, the patch that I'm about to send to bosh only makes sense using persistent device names12:14
pedroalvareztiagogomes_: you mean "non persistent"?12:16
pedroalvarezeth0, eth1, etc?12:17
tiagogomes_persistent, in the sense that NIC 1 will always be mapped to the same interface name across reboots (eth0, em1)12:20
tiagogomes_it doesn't need to necessarily use the ethX terminology12:20
pedroalvarezthen, using the exsisting stemcells may be dangerous :)12:21
nowsterirda0, for instance? :)12:22
paulsherwoodQtWebkit build time vs cores... https://docs.google.com/spreadsheets/d/149jS3VVJ7jA14dSJybeOr4otlEFrzCVJjbxMSppiMcg/pubchart?oid=1071803514&format=interactive13:20
radiofreewow that's fast13:22
rjek7 minutes with 20 cores?  Hmm13:23
edcraggi'm a little confused about what 'merge conflict' means on gerrit. i rebased a branch onto the gerrit master without conflict, resubmitted, and gerrit still reports a path merge conflict. this change in particular: https://gerrit.baserock.org/#/c/794/313:26
radiofreeis it part of series of patches?13:27
paulsherwoodpoint is that going beyond 20 cores seems not worth it, rjek ... unless i'm missing something fundamewntal13:28
franrededcragg, I think what it means is that you need to merge the previous commit first13:28
edcraggradiofree: yes13:28
radiofreei think gerrit treats everything as an individual patch, so if a patch relies on changes in a previous path, it'll have a merge conflict13:28
radiofreei'd say just ignore "merge conflicts"13:28
edcraggthat can't be the case, as surely then all patches in a series would have a merge conflict13:28
radiofreedoes the first one have a merge conflict?13:28
robtaylorpaulsherwood: shame they dont show how much ram they've allocated13:29
edcraggradiofree: no, it seems to be 'random'13:29
edcraggas in, probably related to some actual conflict with something13:29
radiofreejust ignore it and let whoever is responsible for merging sort it out13:29
franrededcragg, it is not random, it is waiting until you merge the previous commit13:29
robtayloroh, actiay its 160GB13:29
edcraggfranred: are you saying anything which relies on changes in any other patch that's not also merged will cause this?13:32
franredIt could be, yes13:33
franredbut in this case I think it is waiting for https://gerrit.baserock.org/#/c/738/413:34
franredthat is what I meant ^^13:34
edcraggok, in which case it isn't an issue13:35
edcraggwhen the series gets merged, the preceding patch will be merged before it tries to merge the one with the conflict13:36
pedroalvarezyou could squash the the first and the second commit in this case13:36
franrededcragg, yes, that is what it would happen13:36
edcraggpedroalvarez: ok, yes, but that may be forcing a squash where it's only required for gerrit, where separate commits may otherwise be more appropriate or clear13:38
franrededcragg, I suggest you to leave it as it is13:39
pedroalvarezin this case , the extra commit doesn't make it easier to review13:39
pedroalvarezbut yes, this is not a problem to get it merged if it has been +2ed13:39
edcraggcool, so are we saying we can generally ignore merge conflicts where patch series are concerned? is that something that could be added to the contributing wiki page?13:40
pedroalvarezthis is something that should be fixed in gerrit '-.-13:44
edcraggwell, patch series yes, whoever uses them anyway!13:45
edcraggthanks for the help. Is there any chance anyone would be able to merge https://gerrit.baserock.org/#/c/940/ ?13:50
nowsterlibogg and libvorbis lorries are both fixed tarball names at the moment. Xiph has set up git repos for all its products. How would be the best way to change these lorrys? New ones and deprecate the old names or adapt the old ones?13:52
franrednowster, create a new lorries with -git at the end?13:53
nowsterSounds like a good idea.13:54
franred;-)13:56
paulsherwoodrobtaylor: is there some trick ybd could do to report that? one thing i'm worried about is oom for the gang/herd approach... would like some way to police it14:18
nowsterfranred: would it be a good idea to name gnome components (used outside desktop gnome) with gnome/component-name ?14:44
nowstereg. gupnp14:44
nowsteror libmediaart14:44
persiapaulsherwood: On your graph showing the lack of improvement at 20 cores: do you know whether that is the limit of parallelism in the workload or some limitation of memory and/or storage bandwidth?14:46
robtaylorpaulsherwood: checking memory limits per worker?14:48
paulsherwoodrobtaylor: yes. i don't know how to do that14:49
persiarobtaylor: Tricky, as each worker then spawns any number of processes to actually do the work14:49
paulsherwoodpersia: i don't know. if others have suggestions on how to find out, i'm very interested :)14:49
robtaylorpaulsherwood: probably use memory cgroups so you can have per worker limits, and also possibly use memory threshold events to monitor it14:50
robtaylorpaulsherwood: all in http://lxr.free-electrons.com/source/Documentation/cgroups/memory.txt14:50
robtaylorpaulsherwood: and can be setup by systemd or libcgroup14:50
robtaylorpaulsherwood: i believe radiofree has been working with this recently so probebly knows the details14:50
persiapaulsherwood: My method to find out would be to measure maximum memory and/or storage bandwidth by running iostat/memstat with a large synthetic load, then comparing to the numbers while running the builds.14:50
persiaIt is a crude way to detect potential saturation, but may give some guidance.14:51
paulsherwoodrobtaylor: erk... i'm just a hobbyist python n00b... that sounds like a job for someone expert :)14:51
*** petefoth has quit IRC14:53
*** franred has quit IRC15:00
*** franred has joined #baserock15:01
nowsterfranred: would it be a good idea to name gnome components (used outside desktop gnome) with gnome/component-name ? (eg. gupnp or libmediaart)15:01
franrednowster, Im not sure if jjardon has started to do that or not, so maybe worth to talk with him15:02
nowsterI notice that a lot of the gnome components are already gnome/something.15:02
nowstereg:     "gnome/libgee" and "gnome/libnotify"15:03
pedroalvarezoh, for new lorries? we dont have a policy for that15:05
nowsterWould it be a good thing to use that convention for libraries that are derived from parts of gnome?15:06
*** franred has quit IRC15:07
pedroalvarezmakes sense to me to use the gnome/ namespace for them15:08
jjardonrobtaylor: AFAIK, in latest systemd the idea is to use high level systemd configuration (CPUShares, MemoryLimit...), instead deal with cgroups directly: http://0pointer.de/blog/projects/resources.html15:09
radiofreepaulsherwood: it's really easy to launch things in cgroups and get the stats15:14
radiofreejjardon: that's a little bit much when you just want to do something simple like "run this task in a cgroup and collect data"15:14
radiofreecgexec -g memory:/foo/ myprocess15:15
radiofreecat /sys/fs/cgroup/memory/foo/memory....15:15
*** franred has joined #baserock15:16
paulsherwoodGCC graph https://docs.google.com/spreadsheets/d/149jS3VVJ7jA14dSJybeOr4otlEFrzCVJjbxMSppiMcg/pubchart?oid=1593295908&format=interactive15:18
franrednowster, I would say, if you have seen components in that namespace please go ahead with that15:18
franredI meant, use the gnome namespace if it makes sense15:19
nowsterok15:19
*** paulw has quit IRC15:19
*** Guest56770 has quit IRC15:41
jjardonpaulsherwood: how many cores that machine has? normally what it worked best for me is (number of cores + 1), but I only have 415:52
flatmush(number of cores * 5) / 415:54
radiofreeqtbase from qt4 might be interesting15:56
paulsherwoodjjardon: 4016:12
jjardon40??? O_o you lucky :16:18
radiofreevery lucky... it built qtwebkit in 8 minutes!16:19
persiaIt is more money than luck.  Anyone can schedule a 40-core AWS instance, if they want to pay for it.16:21
paulsherwoodyou can be so lucky too - AWS, $2.50 per hour :)16:21
persiaIt is very interesting to know what they are good for, because for many workloads it is actually cheaper than scheduling a 1-core for a longer period of time.16:21
jjardoninteresting, didnt now that16:21
*** mariaderidder has quit IRC16:25
pedroalvarezhm.. IIRC that means that building ci.morph would be less than $416:27
nowsterWe're almost back to the days of timesharing mainframes, effectively.16:29
* rjek submits a batch job on paper tape16:29
* nowster tries to forget JCL.16:30
*** zoli__ has quit IRC16:31
*** jonathanmaw has quit IRC16:32
*** zoli__ has joined #baserock16:32
paulsherwoodpedroalvarez: yup. and much faster than the current mason :)16:33
paulsherwoodeven better if we can sort out shared cache...16:35
pedroalvarezhm.. mason could spawn instances on demand, and use clean environments all the time16:51
paulsherwoodyup. still needs cached repos and artifacts, though16:53
paulsherwoodtook over 30 mins to clone the repos for base-system16:54
persiaThat would be an ideal model, because then mason can use the morph from HEAD, which avoids the versioning issues we've seen in the past.16:54
paulsherwoodmorph? ii thought you meant ybd :)16:54
persiaI think I mean "the integration tool"16:54
* paulsherwood decides to go and troll at the pub, instead of here16:54
*** bashrc has quit IRC17:04
*** mdunford has quit IRC17:05
*** edcragg has quit IRC17:25
*** franred has quit IRC18:02
*** lachlanmackenzie has quit IRC18:53
*** zoli__ has quit IRC19:32
*** petefoth has joined #baserock19:39
*** zoli__ has joined #baserock19:52
*** zoli__ has quit IRC20:10
*** zoli__ has joined #baserock20:18

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!