IRC logs for #baserock for Thursday, 2014-07-31

*** sambishop [~sambishop@] has quit [Quit: Leaving]05:49
*** fay_ [] has joined #baserock07:25
*** sambishop [] has joined #baserock07:39
*** franred [] has joined #baserock07:53
jjardonpersia: then seems I hit a bug because I couldn't build the same system I was building when I have internet connection07:54
jjardonSam suggested that maybe the problem is that in the previous build morph was taken the cache from the server07:54
persiaPulling cache from the server should have updated the local cache.08:05
persiaDid you change the version of morph?08:05
*** mwilliams_ct [] has quit []08:24
*** jonathanmaw [] has joined #baserock08:32
*** ssam2 [] has joined #baserock08:41
*** tiagogomes [] has joined #baserock08:41
*** tiagogomes [] has quit [Ping timeout: 245 seconds]08:52
jjardonssam2: put    "\e[3~": delete-char     in /etc/inputrc ;)09:18
jjardonpersia: no09:19
persiaOdd.  I don't understand why you didn't have a good local cache for everything that wasn't changed for your update.09:19
ssam2jjardon: AWESOME09:21
ssam2I notice in Fedora I have a bunch of stuff in /etc/inputrc09:22
ssam2I think we should add that to fhs-dirs in Baserock so all the keys work :)09:22
* Kinnison worries that fhs-dirs is somehow becoming base-files09:22
persiaPerhaps it ought be split?09:24
persiaAnyway, shouldn't /etc/inputrc be installed by readline?09:24
jjardonpersia: I think so but AFAIK its not in the system because is GPLv309:24
persiaIf readline isn't in the system, what is expected to parse /etc/inputrc ?09:25
ssam2happy to rename fhs-dirs to base-files, or come up with some other place09:25
ssam2I just want delete to work :)09:25
*** ssam2 [] has quit ["Leaving"]09:25
*** ssam2 [] has joined #baserock09:25
Kinnisonis that the non-chorded ^D ?09:25
ssam2I don't know09:26
persiassam2: I'd suggest that /etc/inputrc belongs in whatever chunk contains the library parsing it.09:26
ssam2the one that says "Delete" on my keyboard09:26
persiaKinnison: No, that's "backward-word"09:26
ssam2persia: true, and I guess adding 'readline' fixes this because it installs that file09:26
ssam2but that seems to be impossible to get past the review process09:27
persiaOr if you're using something else as a readline library, stick it there.09:27
*** ssam2 [] has quit [Remote host closed the connection]09:41
*** ssam2 [] has joined #baserock09:42
ssam2fay: yes, it's the one from last week09:42
ssam2sorry, I found the magic "exit xchat suddenly" keypress09:42
ssam2oops, also this is the wrong channel09:42
petefothssam2: welcome to my world :)09:47
ssam2it'd be appreciated if those with personal branches in morph.git and definitions.git that have been merged and/or are no longer deleted could delete them10:36
*** tiagogomes [] has joined #baserock10:37
persiaNow that richard_maw has demonstrated success with a non-branch based merge process, is there a reason to continue to permit personal branches in those repos?10:43
persiaFor cases where a branch is really desireable, presumably one could provide a ref to another repo containing personal branches/10:43
persia(could even be g.b.o/baserock/personal-branches/definitions.git if that is a sensible hosting location for personal branches)10:43
petefothso long as the proposed new process is adequately documented for new and returning users10:43
* Kinnison notes that ssam2 probably meant "Feature branches" not "personal branches"10:44
Kinnisonit's just that for now, we tend to namespace our feature branches under baserock/username/10:44
ssam2persia: I've not seen that demo, is it a recent mail to baserock-dev@ ? I'm still catching up10:44
ssam2Kinnison: yes, I meant the extant branches which can currently be described as either of those things10:44
* Kinnison nods10:45
persiassam2: I think it was about a week ago or so: sending some stuff that needed git-am to process.10:45
Kinnisonssam2: persia is referring to where richard_maw did some Baserocking outside of his work persona10:45
ssam2persia: oh, I know what you mean. yes, it's possible to use git-am10:45
persiaKinnison: namespacing makes sense: I'm just wondering if those ought be put in the same repo, rather than a repo with explcit indications that it contains random code by random folk.10:46
ssam2on gcc 4.8:
Kinnisonpersia: For core-developed feature branches I strongly believe they should be in the core repositories10:47
persiaI don't believe there to be a workable semantic value for "core" to use in that statement currently.10:47
persiaBut I probably disagree with you more fundamentally, because I believe that you're envisioning a system where some developers are more equal to others.10:48
* Kinnison currently defines it as "entities with personae who have access to keys permitting them to push to those repositories"10:48
persiaAnd in a peer-reviewed codebase, I think that's something that should be explicitly avoided.10:48
KinnisonI appreciate this is perhaps an "old school" definitions10:49
persiaAn an example of the sort of odd behaviours this generates: I know of at least two cases where employees (at different firms) were let go because they "failed to gain core" in some project.10:49
persiaThis is despite them being active in upstream, well respected, and landing all the goals of the companies in question.10:50
Kinnisonssam2: regarding your confusion over why adding the SHA is useful on the tag -- it's because we're using upstream tag names and upstream can (and will, and perhaps more importantly HAVE) move tags.10:50
Kinnisonpersia: boggle10:50
persiaSo I'm actively against any definition of "core" you try to suggest, but I don't believe there is one now, so haven't started that argument :)10:50
KinnisonI see10:51
* Kinnison will ponder on that one10:51
radiofreeis there a bugzilla or something for baserock?10:52
ssam2Kinnison: but the old tag will be in the history of the baserock- tag, so we'll be able to detect if the upstream tag has changed anyway10:52
persiaradiofree: No  Send mail to the mailing list for now.10:52
* paulsherwood mutters GitLab under his breath10:53
persiassam2: You've convinced me that adding short-sha provides no benefit.  Thank you!  I like more readable names.  My only remaining worry is whether we can have a convention that lets those developing a definitions set have confidence they can avoid a tag name collision.10:54
* paulsherwood wonders what the convincing argument is10:55
persiaThat a local tag has the SHA of the upstream tag in it's history, so it's trivially derivable without humans needing to remember arbitrary hex sequences.10:56
ssam2it has the SHA1 of the commit that the upstream tag pointed to10:57
ssam2but that should be enough, I think?10:58
persiaAside from the possibility of collisions, I think so.10:58
paulsherwoodwhat is 'local tag'?11:00
persiaA tag a system integrator makes in a shared definitions.git from which they wish to generate reproducible builds.11:01
paulsherwoodok. so two years later, some new integrator (or saboteur) moves the tag, by accident or on purpose... how would we spot that?11:04
persiaAh, so embedding the shortsha in the name of the tag serves to confirm the tag has not moved?11:05
paulsherwoodyes. i thought i'd said that, twice :)11:06
persiaSomehow I forgot today, and was swayed.  Thank you for reminding me.11:06
* persia is no longer convinced we can do without the shortsha11:06
* paulsherwood cheers :)11:06
paulsherwoodnow for ssam2 :)11:06
ssam2can't gitano rules prevent people from moving the tag?11:07
persia1) We can't guarantee everyone stores their definitions.git in gitano, and 2) wouldn't that block moving tags for other purposes for other repos?11:08
paulsherwoodpossibly if we are mandating gitano. but what's the real overhead for doing it as i suggest? belt-and-braces just seems better to me11:08
ssam2Including sha name in tag is more robust, I agree11:09
ssam2I think we could mandate Gitano for people who want protection against the fairly rare case of an integrator forcepushing a tag, if we wanted11:10
ssam2I can't get my head around the exact overhead of your solution, I think actually trying it is probably the only way to know11:10
ssam2I have the impression I'd have to type hex strings a lot, but maybe I'm wrong11:10
paulsherwoodcan i count that as a +1?11:11
ssam2a +1 to try it out, certainly :)11:11
* paulsherwood uses cut and paste a lot11:11
paulsherwoodwe could have morph do the tagging, trivially...11:11
ssam2that would help a lot11:12
persia`morph commit`?11:12
persiaWe're still not protected against tag deletion, but we aren't protected against branch deletion now, so this isn't a regression.11:14
paulsherwoodpersia: i think this is more like 'morph publish'11:17
persiaI like that, not least because by using "publish", we avoid semantic confusion.11:19
paulsherwoodyes, i'm thinking we  should actively avoid clashing with git commands, but that's a bigger discussion11:20
persiaInteresting, the idea being that git+morph fit together well, but never use the same verb?11:22
* persia especially likes that this means doing away with all the morph verbs that don't seem to make sense11:23
paulsherwoodyes, possibly. but it needs more thought. and at least one of our customers has already flagged that he'd prefer to see only morph commands, no git commands in the workflow11:23
persiaThat's just a framing issue.11:24
paulsherwoodbut arguably that may be as a result of the current commands/naming11:24
paulsherwoodwhile we're on, i'd like to eradicate 'morphology' from the project vernacular11:25
persiaWhat do you suggest as an alternative?11:26
ssam2I think it'd be possible to more-or-less transparently wrap Git, by having Morph act as a 'filter' around Git commands, passing through any arguments it doesn't recognise to the underlying Git commands. I don't have time to think about this now, but I think it's a possible approach to investigate when we do look at solving this11:26
persiassam2: Why would one do that?  Wouldn't it be easier to just implement new verbs with git-foo?11:27
persia(e.g. git-build, git-deploy, etc.)11:27
ssam2persia: we need to override some, e.g. 'git push' so that it handles 'git-fat' transparently11:27
persiaI'm not advocating this, but it seems less fraught with danger than wrapping git11:27
persiaI still don't think it should do that :)11:27
straycat_radiofree, biff11:30
radiofreestraycat_: hi11:46
radiofreei linked to your branch, you only need the one commit now (for the filename stuff)11:46
radiofreethat gets past .morph.morph11:46
radiofreehowever i still can't build because of that "Failed to compute build graph: Problem with serialise-artifact" after that11:47
straycat_Right, that'll be my general lack of sleep kicking in11:51
rjekThis morning I got up thinking I could sleep for another month or so.11:52
straycat_Either you're trying to build some definitions that aren't on your trove or some other recent change has also broken serialise-artifact.11:53
radiofreeoooh i see11:54
radiofreemy definitions branch isn't pushed anyway, does that cause problems?11:54
straycat_Shouldn't, since build branches are pushed, I mean that your definitions have a ref that's not on your trove.11:56
radiofreehmm i wouldn't have thought that's the case, i'll double check though11:57
radiofreeah well actually i built all of this stuff on my development machine using the same trove, so i guess it is all there?11:57
persiaDo you have any unpushed commits in your defintions?  That would create a SHA not on the trove.11:57
Kinnisonssam2: So your suggestion is that the baserock/$UPSTREAMTAG is now no longer just a ref anchor for the same object as $UPSTREAMTAG but instead is somewhere we merge to?11:58
* persia doesn't like this interpretation11:58
radiofreepersia: no11:58
radiofreei used morph branch if that makes any difference11:59
persiaIt shouldn't.11:59
ssam2Kinnison: my suggestion is we don't have -2361436 on the end of the tag name, no other changes. Maybe I'm missing something11:59
straycat_When you used morph branch, was your trove-host and trove-id set to the same trove that the distbuild network is using?11:59
radiofreestraycat_: yep12:00
Kinnisonssam2: The intention was that baserock/$UPSTREAMTAG was simply a way to anchor the tagged objects in a namespace upstream can't mess with12:01
ssam2that makes sense12:01
ssam2and if upstream change their tag, ours won't change, so we'll notice that they changed it12:01
straycat_radiofree, Hrm, then I don't know. :/12:01
Kinnisonssam2: if you omit the -SHA bit then if upstream move what $UPSTREAMTAG points out, baserock/$UPSTREAMTAG will have to either migrate (losing history) or merge (potentially confusing)12:01
ssam2why can't it just stay the same?12:02
persiaThat's the "merge" case, which is confusing12:02
ssam2no it isn't12:02
ssam2it's the "continue to use the old tag" case12:02
radiofreestraycat_: i just did a morph checkout of the definitions branch i want to use, save trove etc as the distbuild network12:02
Kinnisonssam2: I'd be confused12:02
radiofreestill the same problem12:02
Kinnisonssam2: because I'd expect baserock/$UPSTREAMTAG to == $UPSTREAMTAG12:03
ssam2the average baserock user will be confused about the existence of the tags regardless of the policy, until they learn the policy12:03
persiassam2: Sorry: terminology: I interpreted Kinnison's "merge" as having a single repo containing both the moved upstream tag and the unmoved baserock tag, rather than having anything to do with `git merge`.12:03
ssam2the fact that you decided on one policy and would be confused if we chose a different one is neither here nor there12:04
ssam2i'm happy with either solution, btw, we don't need a long debate12:04
*** radiofree [radiofree@unaffiliated/radiofree] has quit [Quit: ZNC -]13:09
*** pedroalvarez [] has quit [No Ping reply in 180 seconds.]13:14
*** pedroalvarez [] has joined #baserock13:15
Mode #baserock +cnt by kornbluth.freenode.net13:15
*** ssam2 [] has joined #baserock13:15
*** radiofree [radiofree@unaffiliated/radiofree] has joined #baserock13:34
jjardonyay! Thanks ssam2 (and other reviewers) for the GTK3 merge! :)13:34
persiaIs there a gtk3 test system defined somewhere?13:35
jjardonBut I have a concern, I do not thing the stratum model is workin (at least for me). For example: to build a new GNOME component I need a new fuse or systemd, things that are in foundation, so I have to rebuild _a lot_ of stuff. The same happen with GLib, that its in foundation as well13:36
jjardonpersia: no, but Im using that stratum in a gnome system Im working on13:36
persiaCool.  If you can share, I'd like to give it a try when it works.13:37
jjardonspoiler: gtk3-demo runs here ;)13:37
jjardonI can create a super basic X system (or even wayland) so you can take a look13:38
persiaNo rush.  I won't have time to play with it that soon anyway: lots of upcoming travel.13:38
persiaOn strata: I think that it's a good concept, but perhaps the current strata haven't had the benefit of enough different types of systems?13:39
*** tiagogomes [~tiagogome@] has joined #baserock13:39
persiaI like the idea of several associated bits of software being managed together and built together, because dealing with interdependency maps for some things is incredibly painful (e.g. KDE).13:40
jjardonyeah, or maybe we should think how to divide the strata in different ways13:40
liw-orcI seem to have forgotten to update the ref for morph when I merged a patch from richard_maw to morph -- any objections to me updating that ref?13:40
persiaI'm not sure we can decde that a priori, but rather only by the process of adding more chunks, and seeing what logical strata develop from the associated interdependencies.13:40
persialiw-orc: which ref?13:40
liw-orcthe ref in tools.morph13:41
persiaYes.  To what value to do you wish to update it?  9a9445c47f25d919483a6e798e789dbf32e9310f ?13:41
liw-orcto head of master in morph.git, so yes13:41
persiaDo you think it makes sense to update the morph ref in cross-bootstrap.morph at the same time?13:42
persia+1 from me if both are updated, or if there's a really good reason not to update the cross-bootstrap morph (but in that case I'd like a ML post for documentation).13:43
* liw-orc updates13:44
jjardonpersia: the problem is that for example I need a new GLib (quite common thing) but I should not not need to rebuild X at all for that13:45
persiaThat makes sense.  For that matter, some folk will want X and not GLib.13:46
*** radiofree [radiofree@unaffiliated/radiofree] has quit [Quit: ZNC -]13:46
persiaMy thought is that it is probably reasonable to split/reorganise strata when the current structure is difficult, and point out how it is difficult for each difficulty in the patch series on the mailing list.13:47
persiaSometimes a little excess inconvenience will be worth the conceptual simplicity, but I suspect we'll want lots more strata then we have today.13:48
* persia suspects that users would find it convenient to be able to just write system definitions, without needing to fuss much about strata definitions except in special cases13:48
persiaThinking about this a bit more, I wonder if our ccache configuration is usefully capturing everything: if it works properly, rebuilding extra chunks in a stratum should be really fast.13:51
liw-orcassuming they're using C or another language ccache supports, and that there is no other parts of the build that take a lot time in the build13:51
liw-orcfor a lot small things, most of the overhead is in the configure phase13:51
persiaAh, so while ccache helps, extra rebuilds are still painful enough to try to avoid.13:52
jjardonanother though is: why we should rebuild everything? If I update systemd to a new version of a stable branch everything should still working fine without a rebuild13:53
persiaDepends on build system interdependencies as well as runtime interdependencies.13:55
persiaSome software might enable different features with a different version of systemd, so the rebuild creates a different object.13:56
jjardonI cant image a different way if we want to have an image ready as soon as possible after a commit is done (ie less than 5 minutes)13:56
persiaKey is wanting a trusted image.  There's lots of solutions that can produce working images in minutes from commit.13:56
jjardonpersia: is a stable branch, no new features, only bugfixes13:57
persiaThere's very few solutions that can assure that one can reproduce that image at all (let alone with the level of warrant in baserock)13:57
persiaI believe that sometimes folk make mistakes and sometimes there are regressions.  Were that never true, I'd agree.13:57
jjardonok, but then you have to renounce to CD13:58
jjardonif you cant build a final product in a moderate amount of time for every change you make13:59
persiaI think we can.  I think strata need some reorganisation.  I don't think we want one-stratum-per-chunk.14:00
persiaAnd I'm willing to trade *some* speed for assurance: there's a huge gap between "as soon as possible" and "not in a moderate amount of time".14:00
jjardonok, what amount of time you think? Im talking here to have feedback about a change in less than 5 min14:01
persiaFor me, it's hardware dependent.14:02
persiaAnd I don't believe Baserock should have specific hardware expectations.14:02
persiaIn the event that you have lots of fast hardware with unthrottled IO, I suspect it takes less than 5 minutes for nearly any operation.14:03
jjardonDo not think is hardware dependent. I think its a problem of not building things are not needed to build14:03
persiaOn a Tegra 2 laptop with no distbuild, I expect it takes more than 5 minutes even trying "as soon as possible".14:03
jjardonalso, not everyone has very powerful hardware, I talking about my laptop or a normal desktop14:04
jjardonpersia: and its not only compiling, its creating the images and test them14:05
jjardonevery commit should output a final product, thats how I understand CD14:05
persiaYes.  So the Tegra 2 laptops did all their IO over USB2.  This is part of why I picked this as an example.14:05
*** radiofree [quassel@unaffiliated/radiofree] has joined #baserock14:05
persiaA CD process needs to have infrastructure driving the CD, and the hardware selected to do this needs to be fast enough to keep up with the commit stream.14:06
persiaI don't think the actual time matters that much, as long as every commit results in product.14:06
*** radiofree [quassel@unaffiliated/radiofree] has quit [Quit: - Chat comfortably. Anywhere.]14:08
*** radiofree [quassel@unaffiliated/radiofree] has joined #baserock14:09
*** radiofree [quassel@unaffiliated/radiofree] has quit [Client Quit]14:10
*** radiofree [radiofree@unaffiliated/radiofree] has joined #baserock14:13
*** radiofree [radiofree@unaffiliated/radiofree] has quit [Remote host closed the connection]14:25
*** radiofree [radiofree@unaffiliated/radiofree] has joined #baserock14:25
jjardonpersia: ok, so tell me how can I work in a system where I track master of several components and every time I build, everything gets rebuilded because little changes in one of the modules14:33
persiaSplit the stata differerntly.  Make sure you have harware reources to mee your time goals.  Maybe do some proviling to determine optimisation cndidates.14:37
* ssam2 wonders about how much persia's proposed design costs in terms of electricty bills14:45
jjardonpersia: if the solution is to split the strata differently, whats the point to have strata in the first term? Mmm maybe make sense to have a strata per chunk in low level component and bigger strata when you go up in the stack14:57
jjardonMaybe foundation should be splitted completely: xorg should not depend on glib or fuse, but only in the components it needs. I think that would speed up things quite a lot15:00
*** fay_ [] has quit [Remote host closed the connection]15:05
radiofreeyes it is a bit maddening at the moment15:10
* jjardon search google for maddening15:11
paulsherwoodjjardon: not sure if we discussed this before, but if you are finding that you need to keep building foo and bar chunks, and they are triggering lots of rebuilds, you could consider having new versions foo-new and bar-new, in a new top-level stratum - do your work on them, then replace the originals when done?15:33
paulsherwood(me has done this, and it works for some use-cases at least)15:33
jjardonpaulsherwood: yeah, but id like a more general solution15:34
jjardonAlso is not the same: if im working in gtk i do not want to rebuild wayland and xorg every time a commit in glib is done15:35
paulsherwoodcan't have glib-new over the top, and make your gtk point to that?15:36
paulsherwoodi agree a general solution is required15:36
jjardonThat was an example, what if instead glib only they are several modules?15:40
paulsherwoodmake them all -new?15:40
jjardonI will take a look and see if foundation can be splitted in a way it saves as much as possible rebuilds15:41
jjardonpaulsherwood: that its a lot of work that will not benefit other people15:41
jjardonOr maybe we are envision  baserock differently? I see it as a tool where i can work with moving parts, so its easy to keep up to date, not where everything is static15:43
*** tiagogomes [~tiagogome@] has quit [Ping timeout: 240 seconds]15:50
paulsherwoodjjardon: that's the idea, yes. i think in some respects we achieve that well already, but there's still lots to do16:20
* ssam2 longs for a 'morph what-changed-such-that-you-think-you-need-to-rebuild-everything-above-eglibc' command16:36
ssam2I guess once again, CD will mitigate this issue16:37
*** jonathanmaw [] has quit [Quit: Leaving]16:41
petefothssam2: another requirement for which CI will be the silver bullet - hurrah!16:45
jjardonpetefoth: does not CD include CI ?17:07
*** ssam2 [] has quit [Quit: Leaving]17:18
*** tiagogomes [] has joined #baserock17:18
*** tiagogomes [] has quit [Quit: Leaving]18:08
*** sambishop [] has quit [Ping timeout: 255 seconds]19:10
persiajjardon: CD does not require CI, but it's best practice.  Consider a git repo with a post-commit deploy hook: this is simplistic CD, but means that the broken versions also get deployed.23:56

Generated by 2.15.3 by Marius Gedminas - find it at!