IRC logs for #baserock for Thursday, 2016-05-05

*** gtristan has quit IRC01:34
*** gtristan has joined #baserock01:41
*** gtristan has quit IRC02:14
*** gtristan has joined #baserock03:28
*** franred has joined #baserock07:05
*** CTtpollard has quit IRC07:07
*** CTtpollard has joined #baserock07:13
*** ssam2 has joined #baserock07:23
*** ChanServ sets mode: +v ssam207:23
*** tiagogomes has joined #baserock07:27
*** rdale has joined #baserock07:45
*** toscalix has joined #baserock07:46
*** paulwaters_ has joined #baserock07:51
*** jonathanmaw has joined #baserock07:59
*** anahuelamo has joined #baserock08:08
*** franred has quit IRC08:33
*** franred has joined #baserock08:48
*** locallycompact has joined #baserock09:15
*** Lachlan1975 has joined #baserock09:26
*** CTtpollard has quit IRC09:27
*** CTtpollard has joined #baserock09:30
*** Lachlan1975 has quit IRC09:48
*** franred_ has joined #baserock10:23
*** franred has quit IRC10:27
*** fay_ has joined #baserock12:04
paulsherwoodrdale: are you around?12:21
paulsherwoodi'm on a voyage of discovery, trying to retrofit tests for splitting...12:22
paulsherwoodand while exploring reproducibility i've noticed that i can end up with a different set of files in devel-system between one run and another12:25
paulsherwoodthe diff is http://paste.baserock.org/erozadiqom12:25
paulsherwoodi'm wondering how host python might leak into a built artifact12:25
richard_mawhow are you determining those to be from the host system? we do build all those for the openstack systems12:30
paulsherwoodoh, maybe they're not from host, then12:30
paulsherwoodon aws instance, these files are not added to the artifact ... on my laptop, they are12:31
paulsherwoodsame version of ybd, same checkout of definitions12:31
richard_mawif it turns out they are, assuming it still operates similarly enough to morph, the only way that could happen would be if something in the bootstrap process copied them in12:31
ssam2it seems odd that only the setup_requires.txt files would appear12:34
ssam2a python egg directory contains more than just setup_requires.txt12:34
ssam2is there other stuff in usr/lib/python2.7/site-packages/cliff-1.10.1-py2.7.egg-info/ for example ?12:34
paulsherwoodyup... http://paste.baserock.org/opowirazin12:36
paulsherwoodthat first paste was just the diff of `find . -type f` in the artifact on both machines12:37
paulsherwood26 extra files in a set of 77thousand12:38
* paulsherwood was *really* expecting to get the same list both times12:40
ssam2weird12:40
paulsherwoodyup12:41
paulsherwoodhttp://paste.baserock.org/giyuzipozi12:54
*** franred has joined #baserock13:02
richard_mawhttps://github.com/projectatomic/bubblewrap may be of interest here, enhanced descendant of linux-user-chroot13:04
richard_maw(which for some reason has a picture of a cat in bubblewrap in the Readme… which was taken from the flickr account of someone I share a house with)13:05
*** franred_ has quit IRC13:05
paulsherwoodrichard_maw: yup... jjardon has also suggested that13:06
paulsherwoodbut i remain confused about this case... i don't think it can be the sandboxing per-se13:06
richard_mawit's either that, or something badly broken in your artifact cache13:07
ssam2richard_maw: what a perfect coincidence13:08
ssam2(about the cat)13:08
richard_maw13:08
ssam2the main advantage of bubblewrap over linux-user-chroot would be if it gets more widely packaged, so we can just use it everywhere13:08
ssam2instead of having sandboxlib13:08
rdalepaulsherwood: i'm around - i was going to send you what i'd done with splitting tests13:28
paulsherwoodrdale: that would be lovely, thanks13:29
rdaleok, i'll go and find it13:29
paulsherwoodrdale: did you ever test on anything bigger than minimal-system? eg devel?13:29
rdaleno - a minimal system was best because in order to be minimal it needed a lot of splitting13:30
*** tiagogomes has quit IRC13:33
paulsherwoodrdale: fine... however 77000 files in devel suggests that maybe it's not splitting anything?13:36
rdalethat's quite possible if it is using entire strata to assemble a system, assuming the ybd bug about including build depends in a final system has remained fixed13:37
rdaleis there anyway to revert a 'git add' command?13:40
paulsherwoodgit reset HEAD13:41
rdalewill that lose local changes too?13:41
paulsherwoodno13:41
paulsherwoodreset --hard if you want to scrap the changes13:41
paulsherwoodpls could you clarify what you mean by 'the ybd bug about including build depends in a final system has remained fixed13:42
rdalethere was a bug where if you needed a stratum such as python 3.0 to build syslinux, then python 3.0 ended up in the final system even if it wasn't specified in the strata list for the system13:44
paulsherwoodok... and 'remained fixed' ?13:48
*** ssam2 has quit IRC13:48
rdaleyes, ie didn't regress13:48
paulsherwoodwell... master can generate exactly the same files for minimal-system as your rdale/150-master-splitting branch, i think13:49
paulsherwoodwould the bug occur for that system?13:50
rdaleok sounds good - but i wouldn't expect a devel system to use any splitting rules13:50
paulsherwoodaren't there default splitting rules?13:50
rdaleyes, foo-runtime and foo-devel13:51
rdalebut if you just specify the stratum you get both13:51
rdaleand the splitting code isn't invoked13:51
paulsherwoodack13:52
*** ssam2 has joined #baserock13:52
*** ChanServ sets mode: +v ssam213:52
*** gtristan has quit IRC14:00
*** tiagogomes has joined #baserock14:02
paulsherwoodis it possible to run morph outside of baserock now? if so, are there instructions anywhere?14:08
ssam2no instructions that I know of14:18
SotKpaulsherwood: http://git.baserock.org/cgit/baserock/baserock/morph.git/tree/README#n5714:19
SotKdisclaimer: I have no idea if it works14:19
rdalepaulsherwood: just sent you a giant email, sorry14:23
rjekBig endian?14:25
*** franred has quit IRC14:36
*** gtristan has joined #baserock14:47
*** franred has joined #baserock14:51
paulsherwoodrdale: rather than a tarball, don't you have a branch somewhere?15:15
paulsherwood(but thanks for the explanations, that's helpful)15:15
paulsherwoodSotK: thanks! i confirm it appears to be working on AWS...15:16
SotK\o/15:16
* paulsherwood wonders why morph starts by cloning definitions, when it's being run in a definitions checkout15:16
SotKit is too long since I worked on morph for me to give a reliable answer to that question15:17
paulsherwood:)15:17
SotKmaybe to avoid including uncommitted changes by default?15:17
*** jonathanmaw has quit IRC15:18
ssam2legacy really15:19
rdalepaulsherwood: i did 'git add' and found it was adding bazillions of files, hence the question about how to revert a 'git add' command15:27
rdaleso didn't push a branch in the end - it needs a bit more thought15:27
rdalealso needs rebasing on ybd master and so15:27
rdaleon15:27
paulsherwoodweird :)15:31
paulsherwoodnever mind, then15:31
paulsherwoodi can take it from there15:31
rdalethere a loads of files that are generated by the test i think and i don't want to commit those, but i can't remeber exactly which ones yet15:33
paulsherwoodrdale: it's fine. i think i understand your approach, and the 'mode' stuff i've added to ybd should make it doable now without the monkey-patching etc15:35
rdaleok15:36
paulsherwoodhttps://github.com/devcurmudgeon/ybd/blob/master/docs/testing.md#check-splits15:37
rdaleok, that would check that splitting hasn't regressed - i'm trying to think what my test would do if you added some new splitting functionality and wanted to test it15:42
paulsherwoodrdale: if we change splitting functionality, artifact-version needs to bump15:43
rdalei suppose i was trying to test without building everything for speed optimization15:44
* paulsherwood is still working to verify that current splitting behaviour is deterministic, or at least close enough to be relied on15:44
paulsherwoodack15:44
rdaleas far as i know it should be - the build order shouldn't affect it15:45
paulsherwoodrdale: hence my questions about the strange appearance of extra files above15:45
paulsherwoodhttps://irclogs.baserock.org/%23baserock.2016-05-05.log.html#t2016-05-05T12:22:3015:45
rdaleare the .meta file different?15:46
*** CTtpollard has quit IRC15:46
rdalehow are those two .txt files generated?15:48
paulsherwoodyes, to the extent that they feature the extra setup_requires.txt files15:48
paulsherwoodfind . -type f | sort15:48
rdaleok - i think i would like to see how those files differ in the .meta files then15:49
*** franred has quit IRC15:50
rdalecan they be build twice from two different chunks, like awk in busybox and gnu awk15:50
paulsherwoodhttp://paste.baserock.org/erebovobay15:51
paulsherwoodthose lines are the total diff of all the .meta files (ie on aws, the .meta files don't contain any setup_requires.txt lines)15:52
rdalehmm egg meta data15:52
*** ssam2 has quit IRC16:00
*** astrophys has quit IRC16:01
*** toscalix has quit IRC16:05
*** tiagogomes has quit IRC16:06
*** toscalix has joined #baserock16:08
*** toscalix has quit IRC16:10
*** gtristan has quit IRC16:44
*** gtristan has joined #baserock16:50
*** anahuelamo has quit IRC16:58
*** mwilliams_ct has quit IRC17:01
*** rdale has quit IRC17:44
*** rdale has joined #baserock17:56
*** locallycompact has quit IRC17:59
*** rdale_ct has joined #baserock18:15
*** rdale_ct_ has joined #baserock18:16
*** rdale_ct has quit IRC18:16
*** rdale has quit IRC18:18
*** rdale_ct_ has quit IRC18:33
*** locallycompact has joined #baserock18:47
*** gtristan has quit IRC19:50
*** fay_ has quit IRC21:00
*** tiagogomes has joined #baserock21:37
*** astrophys has joined #baserock21:40
paulsherwoodso... turns out in the sandbox on my mac, setuptools==7.0.dev0 - whereas on aws it ends up at setuptools==15.221:48
*** gtristan has joined #baserock21:50
paulsherwoodin the former case, the extra file is being created by http://git.baserock.org/cgit/delta/python-setuptools-bitbucket.git/commit/?h=baserock/master&id=0aa6a4de5931d02876428388678802db2371fd3721:51
*** tiagogomes has quit IRC21:52
paulsherwoodin the latter, i have no idea how the sandbox ends up with that version of setuptools22:06
*** astrophys has quit IRC22:51
*** astrophys has joined #baserock22:51
*** toscalix has joined #baserock22:53
*** toscalix has quit IRC23:03
*** toscalix has joined #baserock23:04
*** toscalix has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!