IRC logs for #baserock for Tuesday, 2016-03-15

*** edcragg has quit IRC00:11
*** rdale_ct_ has quit IRC03:33
*** gtristan has quit IRC06:17
*** gtristan has joined #baserock06:25
*** CTtpollard has joined #baserock07:56
*** locallycompact has joined #baserock08:04
*** ctbruce has joined #baserock08:27
*** bashrc has joined #baserock09:02
*** toscalix has joined #baserock09:22
*** tiagogomes has joined #baserock09:26
SotKpaulsherwood, pedroalvarez: IIRC the deployment stuff I wrote did support subsystems, but it was a long time ago and I haven't checked09:36
paulsherwoodack09:36
pedroalvarezif it's supported, then my guess is that the subsystem is deployed after the system is deployed09:43
pedroalvarezmaking the check of that extension fail09:44
* pedroalvarez yawns09:44
pedroalvarezgood morning baserock!09:44
*** jonathanmaw has joined #baserock09:45
*** franred has joined #baserock09:54
*** edcragg has joined #baserock10:01
*** franred has quit IRC10:05
*** ssam2 has joined #baserock10:07
*** ChanServ sets mode: +v ssam210:07
*** franred has joined #baserock10:21
*** jonathanmaw_ has joined #baserock11:10
*** jonathanmaw has quit IRC11:13
*** franred has quit IRC11:14
*** gtristan has quit IRC11:16
*** jonathanmaw_ has quit IRC11:24
*** jonathanmaw has joined #baserock11:24
*** franred has joined #baserock11:26
*** ctbruce has quit IRC11:49
*** gtristan has joined #baserock12:00
jjardonbenbrown_: Hi, about https://gerrit.baserock.org/#/c/1990/; what the reason to add that stratum the to the build- and devel- systems?12:10
pedroalvarezjjardon: it's a ybd dependency12:11
benbrown_jjardon: primarly due to ybd depending on it12:11
benbrown_^12:11
jjardonis ybd in the build or devel systems?12:12
jjardonor do you plan to add it?12:12
pedroalvarezthat's actually a good point, although I take they are planning using it from git12:13
benbrown_I have no current plans to add ybd to Baserock, if someone else feels like that should be done then they can go ahead and do that.12:13
benbrown_I just found it surprising that I couldn't clone ybd into the build image from wiki.b.o and run it.12:14
jjardonbenbrown_: sure, can you put the reason in the commit message? I will merge after that12:15
benbrown_jjardon: will do12:15
*** N-a-N has quit IRC12:16
benbrown_jjardon: done12:18
jjardonbenbrown_: merged, thanks!12:19
benbrown_jjardon: Thanks!12:19
benbrown_And thanks for the reviews all! ( ssam2 pedroalvarez franred )12:20
pedroalvarezkeep baserocking! :D12:20
jjardonpedroalvarez: Im curious, how did you detect https://gerrit.baserock.org/#/c/1992/ ? Ive never had a build problem with libical12:21
pedroalvarezjjardon: I'm kind of building every commit of definitions for 3 architectures and one of them in 2 times12:22
pedroalvarez(all the mason instances we maintain :)12:23
pedroalvarezlet me find the log12:23
jjardonpedroalvarez: don't worry, I trust you :)12:24
pedroalvarezjjardon: here https://mason-x86-64.baserock.org/log/78e75e303a8d599907563f54f17bd10cad073aeb--2016-03-12%2016:46:54.log12:24
pedroalvarezwarning, huge log. Libical builds twice (because a distbuild bug) one of them it succeeds, and the other fails12:25
pedroalvarezs/it//12:25
pedroalvarezEverytime I see red in Mason, I try to figure out why12:26
pedroalvarezI'm also trying to fix wayland-ivi-extension race condition, already contacted upstream12:26
pedroalvarezalthough if anybody fancies taking a look at their CMake, I think it's small enough to be easy to fix12:27
* CTtpollard would be happy if that got fixed upstream 12:31
pedroalvarezjjardon: heh, I see now the upgrade to libical 2.0.0 :) I didn't have time to test the upgrade to fix the problem this way12:33
jjardonpedroalvarez: :)12:34
pedroalvarezjjardon: from commit log history "libical 1.0.1 is required to build gnome-calendar"12:36
jjardonpedroalvarez: thats the minimum version yes12:37
pedroalvarezcool, just double checking12:37
jjardonpedroalvarez: I have gnome calendar with libical 2 instaled here12:37
pedroalvarezreviewed, (note added, just in case :P)12:38
jjardonpedroalvarez: sure, thanks!12:38
jjardonis there a way to use chunk to build and then completely remove it from the system? Im thinking on busybox here12:43
pedroalvarezyes12:44
pedroalvarezbut involves splitting rules, so I'm not going to be able to give you useful input :(12:45
paulsherwoodjjardon: in ybd, it should be possible to have it as a build-depend only12:45
paulsherwood(i haven't tried that though...)12:45
pedroalvarezwarning: busybox is in build-essentials12:46
paulsherwoodbuild-essential would need to change ...12:46
paulsherwoodeg create a new stratum for-build-only, make that a build depend of systems, not in contents12:46
pedroalvarezwhich also has "runtime-essentials" :)12:46
paulsherwood:)12:47
pedroalvarezit should be possible to do this also with artifact-splitting12:49
paulsherwoodyup12:49
* pedroalvarez runs from build-essential discussions12:50
gtristansystemd takes soooo loooong to compile :'(12:51
paulsherwoodreally?12:51
paulsherwood< 5 mins on most of the things i use12:53
gtristanvery long...12:55
gtristanand its distributing over distccd, just verified12:55
pedroalvarez> 20 minutes in the things I use12:56
paulsherwoodci.scaleway.armv7lhf.log:15-09-06 02:47:58 [84/698/698] [systemd] Elapsed time for build of systemd.aa4aabb8e707287a0324aff9c09b5374f583b4a47d32bca89f5ae508f7df6f42 00:41:4512:56
gtristanyeah, 41min looks probably right ... I think bandwidth is the main issue with this setup though12:57
paulsherwoodci.15.44.log:3 15-10-28 00:42:05 [23/792/792] [systemd] Elapsed time for build of systemd.c6e3e772136afd70d070cefaa349a64150ed0d2d8c582841c08d4d88cc1e24dc 00:01:5712:58
paulsherwoodit does seem to vary widely though12:58
gtristanfor some reason I'm seeing the host 3 files ahead of what distccd is compiling12:58
paulsherwooddo you have a new video ? :)12:58
gtristanwell, armv7lhf is gonna be slower isnt it ?12:58
gtristannot worth a new video, have to run a recompile everything again12:59
gtristanfwiw, it took ~16hrs to get up to the beginnings of foundation last night...13:00
gtristanbut I havent started optimizing yet, really need to get the virtio networking13:00
gtristaneven though the bottleneck is *mostly* configure, it's not really for larger code bases13:00
gtristangcc & glibc for instance take hours, and they could be improved greatly with a faster network13:01
* gtristan discovered interesting oddities with gcc today13:01
gtristanfixed a couple things, first... ccwrap.c in aboriginal needs to recognize the -pthread option...13:05
gtristan-pthread implies -lpthread, but does not when -nostdlib is there13:05
gtristanit also implies preprocessor frobnication13:05
gtristanwhich happens to be -D_REENTRANT, but we forward -pthread along and let gcc decide that part, while forcing the extra -lpthread in the case -pthread was asked for13:06
gtristanbut also I noticed that gcc -dM -E - < /dev/null... behaves _differently_ when -nostdinc is passed13:06
gtristanwhich is gross13:07
gtristanbasically... gcc loves to know what libc it's compiling for, and hardcodes stuff in itself for the libc it's compiling for (if a libc is present)13:07
gtristanso it has these things in gcc/config/... if an eccentric corner case libc happens to be there that it knows about... like glibc for instance... then it has this function which makes it include "stdc-predefs.h"13:08
gtristanwhich is a file that comes with glibc, that gcc presumes to know itself that it should include13:09
gtristanyuck13:09
gtristanso, I've added an extra env var to control ccwrap, CCWRAP_PREDEF... if it's set then we can tell the compiler to include that13:09
gtristaneverything seems to work without the extra things from stdc-predef.h, but for correctness, I will rebuild everything13:10
gtristanoh and also the lto stuff... when you configure gcc with --disable-lto, it *still* produces the binutils wrappers gcc-nm, gcc-ar and gcc-ranlib, which wont run without an lto plugin13:11
robtaylorgtristan: an option to speed up configure is to use cache files https://www.gnu.org/software/autoconf/manual/autoconf-2.65/html_node/Cache-Files.html13:11
gtristansystemd enjoys the lto plugin, prefers the gcc-${prog} binutils wrapers, and breaks13:11
*** bruce_ has joined #baserock13:12
robtaylorah yeah, lto hurts13:13
* paulsherwood wishes he understood even 5% of what tristan is doing13:13
gtristanrobtaylor, thats interesting, havent gotten to optimizing yet but basically... that means that configure results from one package can be reused by configure scripts in another package ?13:13
robtaylorgtristan: one other potetnial next step is to use cross-linking13:13
gtristanrobtaylor, I did some reading on that, and not sure it's possible or will be a significant optimization13:14
robtaylorgtristan: which you can do thanks to virtfs.13:14
robtaylorit'll be big for LTO13:14
gtristanoh well, with virtfs indeed, it would be interesting13:14
gtristanbecause then you dont jam up the network13:14
gtristanbut still doesnt work with our virtfs model13:14
gtristanrobtaylor, remember on the host, all files are -rw-------- user:user13:15
gtristanit's sneaky/dangerous to start man-handling them from the host side13:15
gtristanpossibly doable though, if you have a qemu-kvm dedicated to cross-linking with it's own mount of the same share hehe13:16
robtaylorgtristan: yep, i was just about to say that ;)13:16
gtristantbh, I have --disable-lto just because the cross compiler builds fail without it, and I ran out of weeks to try and get it to build with lto :)13:17
gtristancross-linking is a very interesting thing all of a sudden... but it's not up there at the top of the list of bottlenecks, though13:18
robtaylorgtristan: yeah, lto can actually result in running out of memeory on 32bit builds..13:18
gtristanfirst configure and virtio networking13:18
* gtristan has 1GB swap for each sandbox vm, probably overkill but better to be safe13:19
robtaylorgtristan: i hear firefox linking takes > 4G..13:19
gtristanreallllly ? ouch13:20
gtristanwebkit might take a lot too13:20
robtayloryep13:20
gtristanwell, we have 256MB of ram limit, so it will have to be swap once we get that high in the stack13:20
robtaylorah, you can probably get linkinhg wins just by upping that13:21
gtristan(at least for armv5l, probably the armv7 / aarch64 have higher limits)13:21
robtaylorthats too small for most LTO linking13:21
gtristanI'm not doing LTO though13:21
gtristananyway, all that aside, linking, really isnt the bottleneck, even if it takes 15min to link webkit it will not be high on the list13:22
*** jonathanmaw has quit IRC13:24
robtaylorgtristan: ack, on that front, any plans for instrumenting the builds?13:27
gtristanummm, use an accordian or a harmonica ? ... sorry bad joke... how do you mean instrumenting them exactly ?13:30
gtristanmeasure performance ?13:30
gtristannot yet at this time, I think the top priority is to actually make a deployment13:30
robtaylorgtristan: *nod*13:31
paulsherwoodybd can output to riemann, i believe13:31
gtristanThem of course we already have ybd's build timers, that doesnt tell us where the bottlenecks are but is a general measure13:31
pedroalvarezI'd be interested on qtwebkit build times for armv7lhf :P13:31
gtristan*then13:31
CTtpollardpedroalvarez: are you feeling ok?13:32
pedroalvarezhah, why13:32
robtayloryeah its the individual parts of a build - which parts dominate the time, is parallism being used effectivly, etc13:32
gtristanfrom what I'm seeing, the lack of virtio, due to my incompetence in actually calling qemu-system-arm -net nic,model=virtio -net user and it actually working... is a big bottleneck13:33
gtristannot for the lack of trying it, of course...13:33
robtaylorhmm, does it just fail silently?13:34
gtristanifconfig eth0 10.0.2.15 fails with an ioctl no such device error13:34
gtristanthat part of the setup is done in the aboriginal /init script13:35
robtaylorgtristan: it may be that virtio nic doesnt work with system-arm13:36
gtristanhere: https://github.com/gtristan/aboriginal/blob/master/sources/root-filesystem/sbin/init.sh#L3113:36
robtaylorgtristan: have you tried the qemu dev list?13:36
gtristannot yet, mostly poking around in #qemu irc so far13:36
*** jonathanmaw has joined #baserock13:36
robtaylorits notable that in https://gmplib.org/~tege/qemu.html virtio is used in the aarch64 lines, but not in the arm lines13:37
gtristanhavent taken the time yet... but from googling about, I've not seen an example of virtio networking in conjunction with the simple -net user setup13:37
jjardonpedroalvarez: probably similar to webkitgtk: ~5h I think13:37
* gtristan keeps a tab open on tege's page13:38
*** jonathanmaw has quit IRC13:43
*** jonathanmaw has joined #baserock13:43
gtristanbefore I run off... I should also mention regarding bottlenecks13:47
gtristanlandley mentioned that he was able to get a 20% performance gain in configure with statically linked busybox13:48
gtristandue to how qemu handles memory paging13:48
gtristanI think it's possible to also compile coreutils as one fat static binary with symlinks, not sure but I think I saw that13:49
gtristanof course you wouldnt get the shell & coreutils into the same blob13:49
gtristanbut could help13:50
robtaylorinteresting13:51
robtaylorgtristan: have a good evening!13:51
gtristangood afternoon :)13:53
*** gtristan has quit IRC14:00
*** jonathanmaw_ has joined #baserock14:24
*** gtristan has joined #baserock14:25
*** bashrc has quit IRC14:25
*** edcragg has quit IRC14:26
*** jonathanmaw has quit IRC14:26
*** franred has quit IRC14:26
*** CTtpollard has quit IRC14:27
*** ssam2 has quit IRC14:27
*** CTtpollard has joined #baserock14:30
*** ssam2 has joined #baserock14:31
*** ChanServ sets mode: +v ssam214:31
*** franred has joined #baserock14:40
*** ssam2 has quit IRC14:40
*** bashrc has joined #baserock14:42
*** edcragg has joined #baserock14:42
*** ssam2 has joined #baserock14:47
*** ChanServ sets mode: +v ssam214:47
*** tiagogomes has quit IRC14:53
robtaylorgtristan: something for tomorrow - have you checked you have the virtio nic kernel drivers built-in? (CONFIG_VIRTIO_NET i think)15:03
robtaylorgtristan: also try out -net user -device virtio-net-device,vlan=015:05
robtaylorgtristan: if you've got telnet running in there, you could also try -net user,hostfwd=tcp::2023-:23 -device virtio-net-device,vlan=015:06
robtaylorgtristan: which should allow you to telnet to localhost:2023 to connect to telnet in the guest15:06
*** tiagogomes has joined #baserock15:11
*** CTtpollard has quit IRC16:44
*** edcragg has quit IRC17:16
*** edcragg has joined #baserock17:16
*** franred has quit IRC17:24
gtristanrobtaylor, thanks for the tips I'll try that out... yeah I'm sure I have VIRTIO_NET17:28
gtristaninterestingly toybox has an experimental/unfinished telnet, so I might be able to have that too17:29
paulsherwood:)17:31
paulsherwooddo you *never sleep* gtristan ?17:31
*** franred has joined #baserock17:34
gtristanheh, I am shifted to a strange schedule, I usually go to gym from 11pm - 1am, then eat afterwards, sleep by 5, start the day in the afternoon17:34
gtristanso yeah I sleep regularly... but would probably be a good idea to get more sunlight17:34
*** franred has quit IRC17:59
*** fay_ has quit IRC18:00
*** bashrc has quit IRC18:03
*** ssam2 has quit IRC18:03
*** jonathanmaw_ has quit IRC18:04
*** radiofree has quit IRC18:16
*** radiofree has joined #baserock18:17
*** toscalix has quit IRC19:05
*** edcragg has quit IRC19:16
*** bruce_ has quit IRC19:51
*** locallycompact has quit IRC23:34

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!