IRC logs for #baserock for Tuesday, 2016-02-23

*** persia has quit IRC02:38
*** persia has joined #baserock02:42
*** gtristan has quit IRC04:27
*** gtristan has joined #baserock04:48
*** ctbruce has joined #baserock07:54
*** toscalix has joined #baserock08:16
*** bashrc has joined #baserock08:54
pedroalvarezpaulsherwood: that's nice09:10
pedroalvarezis this the full list of overlaps of a system? I was expecting more09:14
rdalei think we should create two split components for busybox  - busybox-shell for the binaries, and busu09:14
rdalebusybox-init for just the init executable09:14
paulsherwoodthat's everything for trove, iiuc09:19
* pedroalvarez thinks we should drop the word "trove" from this conversation09:20
pedroalvarezI misunderstood09:20
pedroalvarezgood moringn09:20
paulsherwoodactually, there's a bug in my regex (not surprising)... so some are false positives09:21
pedroalvarezthat's good, less overlaps!09:22
*** CTtpollard has quit IRC09:22
pedroalvarezI'm really happy with that list tbh :)09:23
*** CTtpollard has joined #baserock09:26
gtristanpython 2 & 3 look worrying, shouldnt they parallel install nicely ?09:28
gtristanindeed the shortness of the list is satisfying though :)09:29
paulsherwoodgtristan: maybe those overlapping files are identical?09:29
* paulsherwood will have to chck09:29
pedroalvarezthe overlaps of python2 vs python 3? are symlinks to the specific versions09:30
pedroalvarezjjardon is trying to fix them in, but I found some issues testing that patch09:31
*** CTtpollard has quit IRC09:49
*** jonathanmaw has joined #baserock09:53
*** CTtpollard has joined #baserock09:55
*** gtristan has quit IRC10:07
*** edcragg has joined #baserock10:08
*** Lachlan1975 has joined #baserock10:26
*** gtristan has joined #baserock10:34
*** gtristan_ has joined #baserock10:35
*** jonathanmaw_ has joined #baserock10:37
*** Lachlan1975 has quit IRC10:40
*** jonathanmaw has quit IRC10:41
*** jonathanmaw_ has quit IRC10:42
*** jonathanmaw has joined #baserock10:42
*** locallycompact has joined #baserock10:44
*** gtristan_ is now known as gtristan10:51
*** Lachlan1975 has joined #baserock10:55
paulsherwoodis there anything special about procps-ng ?11:10
pedroalvarezI don't understand the question, did i miss some context?11:18
paulsherwoodsorry, yes. it's probably just me. the overlaps check fails to report that procps-ng.meta contains bin/watch and bin/pidof, because that meta file does not exist at the point of detection... yet the detection works, which is weird11:21
paulsherwoods/weird/spooky software magic/11:22
rdaleit would be nice to retrieve the meta files from the ybd cache, without needing to retrieve the complete artifacts but maybe deriving the cache keys needs the git repos for chunks to be retrieved locally11:23
* paulsherwood turns his brain inside out attempting to parse that sentence11:24
rdalei mean it would be nice to get meta data to see if chunks overlap, without needing to build anything, just retrieve the meta data from the cache11:26
gtristannot sure you need the gits... as all the sha1s are in the definitions11:27
paulsherwoodrdale: ok, but it's only possible to create metadata after build :)11:27
paulsherwoodie files are the *results* of build11:27
rdaleyes, but maybe i want to make queries about the contents of the cache that other people have built11:28
pedroalvarezthis usecase sounds weird to me :)11:29
rdalei don't want to have to build every single artifact that has ever been built locally, to make reports about chunk overlap11:29
paulsherwoodrdale: you don't need to, if they're already in the artifact server?11:29
rdaleyes, but is there a way of just retrieving the meta data, and not the complete artifacts as the artifacts will take up a lot of disk space and take a long time to download11:30
paulsherwoodnot currently, no. i'll think about it - would be easy enough for kbas to offer that info11:30
paulsherwoodbut i agree with pedroalvarez - not sure what the usecase is in general11:31
pedroalvarezpaulsherwood: regarding the procps-ng, could it be that procps-ng.meta hasn't been unpacked when you are unpacking bin/watch or bin/pidof?11:32
pedroalvarez(unpacked or moved to the final rootfs)11:32
pedroalvarez(or copied)11:33
paulsherwoodah, ok. yes. thanks!11:33
paulsherwoodlots of patches stuck on +1, help wanted? :)11:36
pedroalvarezreviews are always welcome :)11:39
* richard_maw will review things if highlighted and given a link11:40
* richard_maw doesn't have time to browse pending patches11:40
paulsherwoodi've done what i can - i don't consider myself competent for morph reviews11:40
pedroalvarezI would like to get an opinion on this one:
* locallycompact can also review non-morph things11:44
richard_mawpedroalvarez: ah, another attempt in the long line of trying to handle unicode in python11:53
pedroalvarezyes.. another11:54
locallycompactpaulsherwood, can I bring in pyfilesystem into ybd/ to make it more legible12:16
paulsherwoodlocallycompact: by all means12:22
* paulsherwood doesn't know what pyfilesystem is for, though, and how it will help exactly12:36
locallycompactpaulsherwood, where you have with app.chdir(directory), for dirname, dirnames, filenames in os.walk('.'): //... ten lines of path.join frabbing12:48
locallycompactin pyfilesystem that can be12:48
locallycompact    defs_fs=OSFS(directory)12:48
locallycompact    all_morphs = list(defs_fs.walkfiles('/','*.morph'))12:48
paulsherwoodok cool!12:49
paulsherwoodthere are *five* examples of os.walk in ybd, while you're on :)12:50
paulsherwoodwell, if you're adding a dependency, might as well make full use of it :)12:56
*** ssam2 has joined #baserock13:06
*** ChanServ sets mode: +v ssam213:06
*** franred has joined #baserock13:08
jmacsHas anyone here attempted to do dependency analysis by stracing the build process?13:09
* paulsherwood has not13:10
persiajmacs: You're looking to validate the set of runtime utilities required during build?13:11
pedroalvarezI think we gave it a thought only, as a useful way of extracting the dependencies needed for a chunk13:11
jmacspersia: Roughly, yes, although I'd say packages rather than utilities13:14
rjekjmacs: I know a guy once did automatic package creation from "make install" using an LD_PRELOAD, not sure if that also tracked things like what libraries were opened etc.13:15
persiajmacs: I've looked at build straces trying to find calls to utilities: I seem to recall having a fancy grep that told me what was being run.  I've not tried to reassociate those utilities into file collections (e.g. packages) automatically.13:15
jmacsIt can be done if you have a package manager that knows what files a package provides13:16
rjekActually, that was for Slackware, which I don't think has a meaningful concept of dependencies.13:16
jmacsBut I've only done a few quick experiments so far13:17
paulsherwoodmay be relevant - has a product for tracing dependencies https://www.ellexus.com13:17
ssam2rjek: `checkinstall` right?13:17
persiajmacs: In other efforts (not the same time I did the stracing), I've played with package manages that know what provides the file.  The tricky bit is when there are choices or local collaborations (e.g. Debian's alternatives mechanism)13:18
ssam2I used to use that many years ago and found it to be good13:18
* jmacs adds Ellexus to the list of things to research13:18
paulsherwoodjmacs: i think breeze is the name of their tool13:18
rjekssam2: Yep, looks very similar to what I recall13:21
*** faybrocklebank has quit IRC13:33
*** faybrocklebank has joined #baserock13:37
*** fay_ has joined #baserock13:37
*** fay_ is now known as faybrocklebank13:37
*** tiagogomes_ has quit IRC13:50
*** tiagogomes_ has joined #baserock13:53
*** gtristan has quit IRC13:53
*** gtristan has joined #baserock14:06
*** Lachlan1975 has quit IRC14:36
*** tom__ has joined #baserock15:07
*** CTtpollard has quit IRC15:07
*** paulw has joined #baserock15:23
*** jonathanmaw has quit IRC15:53
*** jonathanmaw has joined #baserock15:57
*** tom__ is now known as CTtpollard15:58
*** franred has quit IRC16:50
*** faybrocklebank has quit IRC17:08
jjardonIs it possible to build mesa without python2, but still depend on llvm which has a dependency on python2?17:11
persiaYou'd probably have to fuss with the ./configure script or pass options.17:11
persiaSuch a thing is theorectically possible, but in practice, most configuration scripts will scour a system to see what is installed, so if you are using python2 as a runtime dependency of llvm, mesa will likely try to compile against this.17:12
pedroalvarezi don't think llvm depends on python2, but it's expecting to find python and not python2 nor python317:13
paulsherwoodyou could write an anti-python2 chunk that obliterates it, for example :)17:13
persiaIf llvm is compatible with any python, one could simply have a /usr/bin/python symlink that pointed at python317:16
pedroalvarezhahaha, overlaping the files?17:16
benbrown_name: anti-python17:16
benbrown_description: summons locallycompact17:16
pedroalvarezpersia: yes, but we were looking at making that symlink only for python217:16
mwilliams_ctbenbrown_: I used to have a summoning ritual for rjek which involved claiming the superiority of the windows kernel17:17
persiapedroalvarez: Hrm?  Why?17:17
pedroalvarezpersia: well, I said "we", but in this case, I don't have a clue about why17:17
pedroalvarezthere was some discussion about this17:18
pedroalvarezjjardon has all the context I believe17:18
pedroalvarezjjardon: maybe the solution is to patch upstream configure script to make it look for python3 aswell if present?17:19
persiaI have no idea what problem was being solved by only having /usr/bin/python only point at python2, but I wouldn't trust that assertion, as there are plenty of systems for which it could not be true, making it fragile.17:20
pedroalvarezwe have been discussing python2 and python3 for months now :(17:24
jjardonNo, maybe my example was not good; I want to compile mesa against python2 but I do not want to have python2 present at all when I compile GNOME, for example17:24
pedroalvarezand that's possible if the python -> python3 symlink exists17:26
pedroalvarezhang on, you are talking about mesa, I was talking about llvm17:27
pedroalvarezbut in your previous comment you said that the only problem was llvm, so I think we are still on the same topic17:27
pedroalvarezjjardon: build llvm with "--with-python"17:29
jjardonpedroalvarez: you are talking about specific strata, when my question is generic :)17:31
pedroalvarezI see17:33
pedroalvarezthere was also discussion about transitive dependencies a while back17:33
jjardonHow can I compile a stratum without the dependencies of other dependent strata; I guess that's not possible at all?17:34
pedroalvarezit is not17:34
*** ctbruce has quit IRC17:35
paulsherwoodjjardon: i could make it possible, as an option in ybd?17:36
paulsherwoodbut lots of things would break17:37
persiaWell, it depends.17:37
paulsherwoodno, i mean, lots of current definitiosn would not build with that option17:38
rdaleit wouldn't be very difficult to write a migration to flatten the stratum build dependencies - that's what ybd does anyway17:39
persiaWhen you compile something, you must have all the runtime dependencies of that thing's build-dependencies installed.17:39
persia*but* the build dependencies of the dependencies can be not installed, if you like.17:39
persiaIn the case of building tools that build python libraries (e.g. with C bindings), one solution is to build *all* the python bindings, then use artifact splitting to have different names for the different results of the build, only putting the ones one wants as build dependencies of the next layer.17:39
* paulsherwood is glad that this channel has brigher folks than him in it17:41
*** cosm has quit IRC17:41
rdaleyou can only use artifact splitting for the final system, it isn't used when assembling build dependencies for strata at present17:43
persiaI consider that a bug.  We should fix it.17:43
persiaOtherwise it becomes impossible to build all sorts of interesting stuff.17:43
paulsherwoodfine - but does the splitting syntax actually support it? i thought its currently *stratum* splitting, for systems?17:44
* persia headdesks17:45
paulsherwoodrdale: maybe you implemented this more generically?17:46
rdaleno it implemented as it was intended to work for morph, but it wouldn't be too hard to change the build-depends of a stratum to have split stratum artifacts in the list, as well as lists of stratum paths. i don't like the syntax like this much:  morph: strata/build-essential.morph17:48
rdalewhat is the 'morph:' bit before the strata path supposed to mean exactly?17:49
paulsherwoodit's relative path17:49
paulsherwood(it is relative path)17:49
rdaleit doesn't say 'path: ' thought, it says 'morph:'17:49
ssam2it says 'morph' due to legacy17:50
ssam2used to be a 'repo', 'ref', 'morph' triplet, back when strata could live anywhere17:50
rdaleright, so it would be nice to fix that if we were allowing split stratum components17:50
persiaIn my delusional perception, I'd like to not have to consider "stratum" when splitting.  It is when I am composing a chunk|component that I know the sensible split semantics, and everything else should be built from the language so defined.17:50
paulsherwoodthere's lots of stuff to fix17:50
paulsherwoodssam2: how do we advance your spec idea?17:50
ssam2keep pestering me to do it :-)17:51
paulsherwoodi don't think locallycompact's spanner is enough to stop the wheel turning17:51
paulsherwoodssam2: consider yourself pestered17:51
ssam2no, although i do mean to respond when I have time17:51
ssam2feel free to contribute to or fork the github repo i set up, in the mean time17:52
paulsherwoodssam2: pls also review my fixes to definitions/schema :)17:52
paulsherwoodssam2: i sent a pr17:52
paulsherwoodadding the lost history :)17:52
ssam2I saw. I might have to rebase on top of it rather than merge it though, for it to make sense17:53
ssam2while we're here, I prefer the name 'spec' to 'schema'17:53
ssam2possibly because 'schema' seems like it already has a more specific meaning, in computer science at least17:53
paulsherwoodok by me17:54
rdaleis 'spec' different from 'specification' ?17:54
ssam2I think what we have compares more with, for example17:54
paulsherwoodit's shorter17:54
ssam2rdale: same thing but shorter, in my opinion17:54
*** cosm has joined #baserock17:54
rdalewell i suppose i'm boring but i find specification more readable17:54
*** tiagogomes_ has quit IRC17:55
paulsherwoodi'm +1 on either17:55
* paulsherwood wishes, incidentally, that there was a video of JPohlmann's 'Don't fear the rebase' talk17:56
locallycompactI'm failing to understand the implementation of splitting in ybd17:57
locallycompactWhat is the function of mark_used_path17:57
paulsherwoodrdale: ^^17:57
rdaleso that items don't appear twice in the file list i think - that was from nowsters original code17:58
locallycompactIt seems very roundabout17:59
*** edcragg has quit IRC18:00
*** jonathanmaw has quit IRC18:02
*** bashrc has quit IRC18:02
paulsherwoodamazing... docker won't send me password reset emails, but *does* send me invites to their new  Docker Datacenter (DDC) gubbins18:03
paulsherwoodi'd tweet it, but what's the use :)18:16
*** locallycompact has quit IRC18:24
*** toscalix has quit IRC18:30
*** ssam2 has quit IRC19:04
*** cosm has quit IRC20:23
*** gtristan has quit IRC20:26
*** cosm has joined #baserock20:37
jjardonpaulsherwood that would be very helpful for me :)22:40
jjardonI see persia concern but I know python is only a build dependency of mesa, so I do not need it at all when building the gnome stratum, for example22:42
*** persia has quit IRC22:58
*** cosm has quit IRC22:58
*** paulw has quit IRC22:58
*** juergbi has quit IRC22:58
*** JPohlmann has quit IRC22:58
*** richard_maw has quit IRC23:00
*** benbrown_ has quit IRC23:00
*** perryl has quit IRC23:00
*** pedroalvarez has quit IRC23:00
*** jmacs has quit IRC23:00
*** SotK has quit IRC23:00
*** paulsherwood has quit IRC23:00
*** _longines has quit IRC23:00
*** wdutch has quit IRC23:00
*** dabukalam has quit IRC23:00
*** persia_ has quit IRC23:00
*** radiofree has quit IRC23:00
*** lachlanmackenzie has quit IRC23:00
*** Zara has quit IRC23:00
*** rjek has quit IRC23:00
*** bjdooks has quit IRC23:00
*** gary_perkins has quit IRC23:00
*** mwilliams_ct has quit IRC23:00
*** inara has quit IRC23:00
*** thrace has quit IRC23:00
*** doffm has quit IRC23:00
*** CTtpollard has quit IRC23:00
*** rdale has quit IRC23:00
*** bfletcher has quit IRC23:00
*** cyndis has quit IRC23:00
*** bwh has quit IRC23:00
*** ratmice has quit IRC23:00
*** trn has quit IRC23:00
*** Stanto has quit IRC23:00
*** jjardon has quit IRC23:00
*** ChanServ has quit IRC23:00
*** Zara has joined #baserock23:02
*** lachlanmackenzie has joined #baserock23:02
*** radiofree has joined #baserock23:02
*** persia_ has joined #baserock23:02
*** dabukalam has joined #baserock23:02
*** wdutch has joined #baserock23:02
*** _longines has joined #baserock23:02
*** gary_perkins has joined #baserock23:02
*** bjdooks has joined #baserock23:02
*** mwilliams_ct has joined #baserock23:02
*** rjek has joined #baserock23:02
*** jjardon has joined #baserock23:02
*** Stanto has joined #baserock23:02
*** trn has joined #baserock23:02
*** perryl has joined #baserock23:02
*** benbrown_ has joined #baserock23:02
*** richard_maw has joined #baserock23:02
*** JPohlmann has joined #baserock23:02
*** paulsherwood has joined #baserock23:02
*** SotK has joined #baserock23:02
*** jmacs has joined #baserock23:02
*** pedroalvarez has joined #baserock23:02
*** thrace has joined #baserock23:02
*** inara has joined #baserock23:02
*** sets mode: +v pedroalvarez23:02
*** persia has joined #baserock23:03
*** cosm has joined #baserock23:03
*** paulw has joined #baserock23:03
*** juergbi has joined #baserock23:03
*** CTtpollard has joined #baserock23:04
*** rdale has joined #baserock23:04
*** bfletcher has joined #baserock23:04
*** cyndis has joined #baserock23:04
*** bwh has joined #baserock23:04
*** ratmice has joined #baserock23:04
*** doffm has joined #baserock23:05
*** ChanServ has joined #baserock23:06
*** sets mode: +o ChanServ23:06
*** edcragg has joined #baserock23:38

Generated by 2.15.3 by Marius Gedminas - find it at!