IRC logs for #baserock for Monday, 2015-08-17

*** paulw has joined #baserock06:23
*** mike has joined #baserock06:54
*** mike is now known as Guest1006606:54
*** mdunford has joined #baserock07:00
*** richard_1aw is now known as richard_maw07:07
*** ChanServ sets mode: +v richard_maw07:08
*** fay_ has joined #baserock07:35
*** jonathanmaw has joined #baserock08:02
*** bashrc_ has joined #baserock08:07
*** petefoth has joined #baserock08:14
*** gary_perkins has joined #baserock08:36
*** CTtpollard has joined #baserock08:53
pedroalvarezhehe, one of the latest build times of mason: 123459s09:14
pedroalvarezalmost!09:14
rjekhaha09:14
rjekMake a very small optimisation.09:15
pedroalvarez:)09:15
*** petefoth has quit IRC09:17
*** petefoth has joined #baserock09:20
*** rdale has joined #baserock09:21
*** petefoth has quit IRC09:52
*** petefoth has joined #baserock09:55
*** petefoth has quit IRC09:56
*** petefoth has joined #baserock10:26
paulsherwoodpedroalvarez: ybd :-)10:44
pedroalvareztbf this was from scratch, 4 systems including the openstack one, and in a not-powerful VM10:47
pedroalvarezbut yes, too slow10:47
pedroalvarezNot sure that ybd is more optimal than morph, altough I have to confess that I've never used it10:48
pedroalvarezmeh, the partitioning change makes the flashing script for the jetson board fail :)11:05
paulsherwoodpedroalvarez: does the release run *from scratch* or re-use existing cache artifacts if available?11:10
pedroalvarezwell, that depends on where and how you do the release, but ideally it re-uses existing cache artifacts11:17
radiofreepedroalvarez: send a patch for the flashing script!11:33
pedroalvarezradiofree: yes! I was figuiring out how to do it11:34
pedroalvarezI was pondering using losetup, and then try to mount partition 1, and fall back to mount the whole disk11:35
radiofreepedroalvarez: it's a bit weird, you'd have to find out the where boot and the rootfs start (e.g parted -> u -> b -> p and note the Start bytes)11:36
radiofreethen you could mount the img with mount -o loop,offset=STARTBYTES yourimage tmp/bootpartition11:37
radiofreecopy everything in there... probably11:37
pedroalvarez:)11:37
radiofreealthough actually maybe if you just format the emmc and dd the whole lot over... that might work11:37
pedroalvarezbut iiuc, the script extracts the u-boot file to flash it11:39
pedroalvarezI should read the flashing script before continue thinking about this11:40
radiofreeyou could rewrite the whole thing in something sensible like python if you want :D11:41
pedroalvarezhaha11:41
pedroalvarezI wasn't plannig to spend a lot of time on this tbh :)11:42
pedroalvarezbut it's a good idea11:42
*** mdunford has quit IRC12:29
*** CTtpollard has quit IRC12:39
*** mdunford has joined #baserock12:44
*** rdale has quit IRC12:44
*** CTtpollard has joined #baserock12:54
*** mdunford has quit IRC12:58
*** mdunford has joined #baserock13:03
pedroalvarezjust "fixing" that wasn't enough. I've dd'ed the whole disk image (with 2 partitions) to mmcblk0p213:03
pedroalvarezIt's probably easier to create a partitioning layout and then dd the whole thing13:04
pedroalvarezs/partitioning layout/partitioning layout for rawdisk deployments/13:05
*** petefoth has quit IRC13:18
*** petefoth has joined #baserock13:20
radiofreepedroalvarez: if you have an image with a /boot / partition layout, you should be able to just dd it to mmcblk013:24
radiofreeu-boot will look in mmcblk0p1:/extlinux/extlinux.conf13:25
radiofreeso assuming your boot partition is sane it might work13:25
*** petefoth has quit IRC13:41
*** zoli___ has joined #baserock14:08
pdarI am aiming to do some patch reviews, but until I soe[B[B[B[B[B14:16
pdarapologies,14:17
pdarMy terminal was frozen and then it suddenly wasnt14:18
* pedroalvarez is waiting for something else after "but until I ..."14:26
pdarOh, it wasnt meant even for this channel, sorry pedro!14:28
pedroalvarezboo14:28
* pedroalvarez continues reviewing on its own14:29
pdarI'll do some reviews later if I have time, I'm promised to another project for now though14:31
pedroalvarez:)14:37
paulsherwoodpedroalvarez: mason ci took 34.5 hours, is that correct?14:49
paulsherwoodhow (much) does it parallelise?14:50
pedroalvarezpaulsherwood: is only one worker, so nothing14:51
paulsherwoodlast time i ran ybd to build all the x86 components on my mac a single instance took 07:13:3714:51
paulsherwoodpedroalvarez: so is that only x86?14:51
pedroalvarezall the x86_64 systems on clusters/ci.morph14:52
rjekpaulsherwood: ISTR that your Mac has *significantly* superior IO than our cloud host14:52
paulsherwoodi daresay.14:52
pedroalvarezindeed14:52
paulsherwoodrjek: is there an amazon machine equivalent to 'our cloud host'?14:53
rjekpaulsherwood: How do you mean?14:53
paulsherwoodsomething of similar performance - otherwise i have no way to compare14:54
rjekI am not very familiar with AWS's offering, sorry14:54
paulsherwoodwhat is the mason running on, then?14:54
rjekOr how their internal infrastructure works14:54
rjekpaulsherwood: I had assumed OpenStack at DC14:55
* rjek 's information may be out of date.14:55
paulsherwoodah, so i could try the same14:55
rjekCeph has some very nice properties, but performance and low latency are not among them14:55
richard_mawand it's not quite distributed btrfs, which I was disappointed with14:56
pedroalvarezIt might be better/cheaper to have a super powerful VM that we only power of to do a build, and only take 1 hour to build everything, than having a slow one and taking 34 hours14:58
rjekYeah14:58
rjekAlso use a massive ram disc as a bcache :)14:58
rjekI know Collabora do than when building LibreOffice.14:58
tlsadoesn't mason build the chunks that are common between all four systems four times?  If so, an obvious optimisation is to deduplicate that15:05
*** jonathanmaw has quit IRC15:08
*** jonathanmaw has joined #baserock15:08
*** mdunford has quit IRC15:10
*** zoli____ has joined #baserock15:11
pedroalvareztlsa: nope, Sam fixed that a while ago :)15:11
*** zoli_____ has joined #baserock15:12
tlsaI don't think it's fixed15:13
pedroalvareztlsa: hm.. why15:13
tlsahttp://paste.baserock.org/nazeyayasu15:14
*** zoli___ has quit IRC15:14
pedroalvarezoh!15:14
*** zoli____ has quit IRC15:14
SotK:(15:14
pedroalvarezgrrrrrrrrrr15:14
tlsaI picked glibc at random; its that same for everything I looked at15:14
pedroalvarezmaybe it's fixed but the version used in mason doesn't have the full fix15:16
pedroalvarezI tested the fix, and it seemed to work :/15:17
tlsais it still the release-build script in definitions?15:17
pedroalvareztlsa: thanks for pointing it out15:17
pedroalvareztlsa: yes15:17
tlsaif so, it doesn't look like that can do any de duplication15:17
tlsait just runs a separate morph build command for each system15:17
pedroalvareztlsa: indeed, the fixes were for morph distbuild, so that it doesn't build twice something15:18
tlsayou probably need to do it in morph, so `morph build [list of systems]`15:18
SotKtlsa: I thought that script uses distbuild-morphology?15:18
SotKI hope it does anyway15:19
tlsamaybe, I didn't look deep15:19
*** zoli_____ has quit IRC15:21
*** zoli___ has joined #baserock15:21
*** mdunford has joined #baserock15:25
* pedroalvarez tries to reproduce the error15:30
*** zoli___ has quit IRC15:40
*** zoli___ has joined #baserock15:42
pedroalvarezreproduced \o/15:42
*** paulw has quit IRC15:45
*** zoli___ has quit IRC15:45
*** zoli___ has joined #baserock15:45
pedroalvarezand still present in master /o\15:46
SotKoh dear :(15:50
pedroalvarezlooks like my testing-case wasn't good enough15:57
*** zoli___ has quit IRC15:58
*** zoli___ has joined #baserock15:58
pedroalvarezI'm not able to compete against ybd anymore16:06
*** mdunford has quit IRC16:06
paulsherwoodlol16:08
paulsherwoodybd doesn't do stripping, and its cache algorithm is different, so i can't actually propose it as a 'solution' yet16:09
pedroalvarezIMO we should stop comparing and merge them16:10
pedroalvarezor create a third one :)16:11
rjekyabit; yet another build/integration tool16:13
pedroalvarezI knew this was going to happen :)16:13
pedroalvarezbaserock-builder, or bb16:13
rjekhaha16:13
pedroalvarezwait, no16:13
CTtpollardnot bb16:13
pedroalvarezhehe16:13
rjekbakebits16:13
paulsherwoodpedroalvarez: on aws, i believe ybd can do the whole x86 ci in under 2 hours16:35
paulsherwood(from scratch)16:35
*** bashrc_ has quit IRC16:48
paulsherwoodthere are some issues, though... for example http://paste.baserock.org/igedayasix16:51
paulsherwoodafter this is logged, my terminal goes blue text/underline :/16:51
paulsherwoodrunning on a mason-equivalent box now, 15-08-17 16:56:17 [CACHE-KEYS] Elapsed time for cache-key calculations 00:42:2517:03
paulsherwood(that's the time to clone all the gits)17:03
paulsherwoodwill see how it does overnight17:04
*** Guest10066 has quit IRC17:25
*** paulw has joined #baserock19:38
*** paulw has quit IRC19:42

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!