IRC logs for #baserock for Monday, 2015-07-20

*** zoli__ has joined #baserock02:20
*** drnic has quit IRC05:35
*** drnic has joined #baserock05:40
*** paulsherwood has quit IRC05:40
*** paulsherwood has joined #baserock05:41
*** paulw has joined #baserock06:18
*** mike has joined #baserock06:35
*** mike is now known as Guest1234306:35
*** mariaderidder has joined #baserock07:25
*** drnic has quit IRC07:31
*** jjardon has quit IRC07:31
*** drnic has joined #baserock07:43
*** jjardon has joined #baserock07:53
*** zoli__ has quit IRC07:57
*** zoli___ has joined #baserock07:57
*** Guest12343 has quit IRC08:02
*** bashrc has joined #baserock08:04
*** jonathanmaw has joined #baserock08:09
*** jjardon has quit IRC08:10
*** mdunford has joined #baserock08:15
*** Guest12343 has joined #baserock08:17
*** edcragg has joined #baserock08:22
*** CTtpollard has joined #baserock08:46
*** mike has joined #baserock09:06
*** mike is now known as Guest3496909:06
*** Guest12343 has quit IRC09:07
*** Guest34969 has quit IRC09:37
*** Guest34969 has joined #baserock09:52
*** paulw has quit IRC10:51
*** paulw has joined #baserock10:52
*** De|ta has quit IRC11:58
*** tiagogomes_ has quit IRC12:22
*** 18VAAA24X has joined #baserock12:23
*** sambishop has quit IRC12:35
* radiofree wonders if he could build a functional baserock system with systemd 4412:36
rjekhahaha12:37
rjekhahahaha12:37
rjekhahahahaha12:37
paulsherwood?12:38
rjekradiofree: You probably could, but I doubt much in the reference definitions would play along.12:38
radiofreehow much actually depends on systemd though?12:38
radiofreeto build that is, i don't care if everything breaks, as long as i can get in and compile12:39
pedroalvarezwe disabled some things from busybox when moving to a newer systemd12:39
pedroalvarez(networking services, etc)12:39
radiofreeany service files we added might need some changing (if they're using new features)12:39
pedroalvarezit should be possible I guess12:40
radiofreedid there ever exist a baserock system with systemd v44?14:11
radiofreemaybe i can see just how reproducible baserock is :)14:12
*** 18VAAA24X is now known as tiagogomes14:12
radiofreeDate:   Fri Mar 16 01:57:47 2012 +010014:13
* radiofree can't remember if baserock is that old14:13
rjekIsh14:13
rjekVery early days14:13
radiofreeooh it is14:14
radiofreeinitial commit was Oct 8 201114:14
* radiofree can't find the release tag for anything before 1414:16
*** jjardon has joined #baserock14:16
radiofreeis there a document of release->sha anywhere?14:17
persiaI think just the tags.14:18
persiaAlthough I seem to remember mentions of releases in the commit messages for earlier releases.  Maybe try inspecting the set of commits that have the systemd you want in something like tig or gitk to see if you find any hints?14:19
radiofreethis is hard14:20
radiofreehttp://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/commit/?id=ac1462d117e4734c0f961c7c44126ae33350283c14:21
radiofreerm/forwardpatch doesn't exist anymore14:21
pedroalvarezradiofree: ugh.. that definitions is too old14:27
pedroalvarezwouldn't be easier to try a more recent one downgrading systemd?14:27
pedroalvarezI was thinking from just before ea286f89fbc7eda7efc8245be0fcc45475ddd67c happened14:29
*** zoli__ has joined #baserock14:35
*** jjardon_ has joined #baserock14:38
radiofreepedroalvarez: i tried downgrading in a recentish system, but as you can imagine it's quite a severe jump14:38
radiofreei don't really want to spend any time on this, i'll see if i can find an ancient fedora or other system14:38
radiofrees/jump/fall14:38
*** SotK_ has joined #baserock14:40
*** perryl_ has joined #baserock14:40
rjekI know Mithrandir has been maintaining Debian packages for systemd for a long time; perhaps you could use that dgit thing of Ian Jackson's to see if it goes back that far >:)14:41
*** inara has quit IRC14:42
*** jjardon has quit IRC14:42
*** zoli___ has quit IRC14:42
*** lachlanmackenzie has quit IRC14:42
*** persia has quit IRC14:42
*** perryl has quit IRC14:42
*** SotK has quit IRC14:42
*** inara has joined #baserock14:42
*** persia has joined #baserock14:42
*** persia has quit IRC14:42
*** persia has joined #baserock14:42
*** lachlanmackenzie has joined #baserock14:43
*** De|ta has joined #baserock14:47
*** jjardon_ is now known as jjardon14:48
*** perryl_ is now known as perryl14:49
pedroalvarezugh, I tried to build that sha1 of definitions, and stage1-gcc failed :/14:53
* pedroalvarez stops playing14:56
*** gary_perkins has joined #baserock15:04
*** tiagogomes has quit IRC15:06
*** tiagogomes_ has joined #baserock15:06
*** mariaderidder has quit IRC16:12
*** paulw has quit IRC16:15
*** gary_perkins has quit IRC16:27
*** jonathanmaw has quit IRC16:35
*** persia_ has joined #baserock16:35
*** Guest34969 has quit IRC16:46
*** edcragg has quit IRC17:06
*** mdunford has quit IRC17:18
*** paulw has joined #baserock17:38
paulsherwoodfolks.. i've +2 and merged https://gerrit.baserock.org/#/c/991/ unilaterally since i heard on another channel that this was blocking someone. in general i believe i should have waited for another reviewer, but i think it's too late today for that17:51
*** lachlanmackenzie has quit IRC17:51
paulsherwood(and it's a one-line change, by a frequent contributor)17:52
paulsherwoodplease let me know if folks think this kind of shortcut should never be taken, rather than just discouraged17:54
nowsterpaulsherwood: agreed with that change17:54
paulsherwoodnowster: tvm17:54
nowsterownership shouldn't go into tarballs, but it happens17:54
nowstercertainly not for source distribution tarballs17:56
persiaWe've a history of doing that sort of thing, but generally we've discussed it on this channel *before* doing the merge.17:57
persiaI'm in favour of simply discouraging the shortcut, rather than prohibiting it, but do think it best to communicate before, rather than after, just in case anyone has concerns, especially if you hear things "in another channel"17:58
paulsherwoodpersia: fair. that occurred to me after i'd hit the button. i'm still impulsive at times18:00
paulsherwoodpersia: incidentally, what did you think of the ybd gang numbers?18:05
persiaI liked "herd" better, but in the abscence of comparisons over differemt environments, the numbers were somewhat meaningless to me.18:06
persiaThat running 10 jobs in a single instance, rather than one, improves things, tells me that the tool is not taking advantage of available parallelism very well.18:08
persiaIt would be more interesting to compare running 10 instances on 40 cores to running 10 independent images, each with 4 cores, with one instance each.18:08
persiaThat helps understand the balance between storage contention and cpu/memory contention18:09
*** paulw has quit IRC18:09
paulsherwood'not taking advantage of available parallelism very well'... lots of parts of the work are not paralellisable... eg creating artifact, configure-commands, install-commands18:18
paulsherwoodat least, there were reliability problems when ybd parallelised configure... morph doesn't parallelise that either iiuc18:18
paulsherwood(so i think this is not the tool, but the intrinsic nature of the workload)18:20
paulsherwoodi don't understand what you mean by 19:08 < persia> It would be more interesting to compare running 10 instances on 40 cores to running 10 independent images, each with 4 cores, with one instance each.18:21
paulsherwoodwhat is an 'image' in this context?18:21
* paulsherwood did compare different environments... macbookcpro vs AWS... but could try others if persia has something specific in mind18:22
persiaBorrowed from SSI vs. MSI terminology: a large computer system can be "single system image", which runs one OS, or "multuple system image", running several OSes.  The first is the classic minicomputer architecture, and the second more of an MPI architecture.18:23
persiaThat was for "image"18:23
persiaAnyway, it would be interesting to me to see if there were differences running many processes in the same system vs. running many processes in many systems, for the same total core count/memory, as this would tell if there was an IO issue.18:24
paulsherwoodbut that introduces the extra variable of connectivity between the systems?18:24
persiaThe macbook vs. AWS umbers aren't interseting because 1) not everyone has a macbook, and this isn't likely to be shared build infrastructure for a team, and 2) AWS infrastructure is unreliable: aside from the limited metrics involved in sizing, it is almost impossible to understand the topology of system interconnect within a system, or the nature of the available storage bandwidth (and even if one can determine these, they tend to change on the18:25
persianext reservation, due to heterogeneity of Amazon's infrastructure)18:25
persiaBuild times on unconnected systems are only interesting within a system18:26
persiaAnd for that matter, unless we comprehend the limiting factor for a given system, optimisation may not help the general case.18:26
persiaYes, I' telling you why this is hard, unhelpfully.  Sorry about that.18:26
paulsherwoodheh. i disagree with your reasoning18:26
persiaMore helpfully, I think that 1) it makes sense to understand the nature of the parallelism exposed by running a herd.18:26
persiaAnd 2) I think it makes sense to try to determine a common sort of environment that would be a build server, for optimisation against.18:27
persiaYou disagree with my reasoning about why I find the numbers uninteresting?  I'd be delighted if you could convince me they were interesting.18:28
paulsherwoodno, you can choose what you find interesting, of course. i disagree with some of the conclusions you're drawing18:29
persiaHelp me understand the distinction you're making18:30
paulsherwoodi believe that the AWS infrastructure i chose is a reasonable example of a 'cloud system' and that the improvement by the gang approach is signficant. i would expect to see similar results on other systems with many cores, modulo their relative io vs cores vs memory capabilities18:31
persiaIgnoring the topologies, yes, running a herd is better than running an instance.18:32
persiaThis suggests there is parallelism in the workload that isn't being exploited except by herding.18:32
persiaI submit that it would be interesting to understand this parallelism and exploit it, rather than just throwing lots of hardware at it and depending on a random number generator to guess about parallelism.18:33
paulsherwoodyes, agreed. i hoped that the scenarios i chose (and the logs) gave enough info to begin to understand it18:34
persiaAnd while I agree that AWS is a reasonable example of a "cloud system", I don't believe that 1) it makes sense to schedule large computation loads on "cloud infrastructure" (better to schedule many small ones in an elastic manner), or 2) that it is a sensible optimisation target simply because the variables won't be constant.18:34
persiaAs I understand it, it's mostly a matter of running more of the builds in parallel.18:35
persiaThe current model seems to have build dependencies on a strata level, which is confusing.18:35
persiaI don't like packages, because it means the deployed systems are unreliable, but I do rather like the notation by which packages are defined, as I think it provides better guidance to build automation of what to build when, and how.18:35
persia(to be clear, the specific bit I don't like about packages is that there is post-install integration logic that is executed on a per-install basis, opening the possibility for two installs to differ slightly, even when performed from the same sequence for an intended identical system)18:37
paulsherwoodbut anyway, so far i have nothing to compare this with... the only  published build-times i can find for for baserock prior to this are single machines running morph18:37
paulsherwoodregarding the model, i agree with you and would like to improve on it. i'm hoping that the work sssssssssssam is doing will lead to a mechanism which allows us to transform the model much more easily than currently18:38
paulsherwood(as a result i'm holding back on suggesting improvements to definitions format for the moment)18:41
persiaIndeed.  To a certain degree, there is an elegance in strata vs. control files or spec files, but there is also some inflexibility (as one would expect from a name based on stone)18:41
paulsherwoodlol18:41
*** paulw has joined #baserock20:12
*** zoli__ has quit IRC23:26
*** zoli__ has joined #baserock23:27
*** zoli__ has quit IRC23:29
*** zoli__ has joined #baserock23:44
*** zoli__ has quit IRC23:46
*** zoli__ has joined #baserock23:54
*** zoli__ has quit IRC23:56
*** zoli__ has joined #baserock23:57
*** zoli__ has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!