| *** gtristan has quit IRC | 03:37 | |
| *** gtristan has joined #baserock | 04:06 | |
| *** gtristan has quit IRC | 06:47 | |
| *** gtristan has joined #baserock | 06:48 | |
| *** gtristan has quit IRC | 07:11 | |
| *** toscalix has joined #baserock | 07:38 | |
| *** fay has joined #baserock | 07:41 | |
| *** fay is now known as Guest31139 | 07:41 | |
| *** gtristan has joined #baserock | 07:46 | |
| *** jonathanmaw has joined #baserock | 08:18 | |
| *** paulwaters_ has joined #baserock | 08:25 | |
| *** paulwaters_ has quit IRC | 08:54 | |
| *** locallycompact has joined #baserock | 09:30 | |
| *** jjardon has quit IRC | 09:47 | |
| *** jjardon has joined #baserock | 09:50 | |
| gtristan | paulsherwood, I've been re-reading your downstream metadata email | 10:09 |
|---|---|---|
| gtristan | as the problem blew up in my face today | 10:09 |
| paulsherwood | ack | 10:09 |
| paulsherwood | which way would you prefer us to go? | 10:09 |
| gtristan | I had some idea that this could potentially be handled with submodules (in one version of an idealistic future) | 10:10 |
| paulsherwood | too complicated imo | 10:10 |
| gtristan | that way we would not have a huge amount of irrelevant deadcode sitting in the same definitions source tree, and we could pull in someone's brewed strata & chunks on demand | 10:11 |
| pedroalvarez | that would bring all the headaches that yocto people have with layers | 10:12 |
| gtristan | but I find it complicated because it goes against other opinions that I have (mostly that stratum is broken and that they should be referring to chunks instead of containing them) | 10:12 |
| gtristan | maybe so | 10:12 |
| gtristan | or maybe the whole idea of trying to maintain all that stuff in the same source tree; is *also* a source of the same headaches that yocto people face | 10:12 |
| tiagogomes | You could also use git subtrees. They are complicated as well, but less odd and error-prone than submodules. The user wouldn't have to know much about them if instructions to update the downstream fork were provided. | 10:13 |
| locallycompact | what is this conversation about | 10:13 |
| gtristan | locallycompact, "downstream metadata" | 10:13 |
| gtristan | the email from last week | 10:13 |
| * paulsherwood still thinks if we enforce uniqueness, we're close to something workable | 10:13 | |
| locallycompact | move build-depends out of the strata | 10:13 |
| locallycompact | uniqueness is the worst possible solution to this | 10:13 |
| paulsherwood | locallycompact: you're starting to sound like persia | 10:14 |
| locallycompact | if you modify one strata you have to change every chunk the system | 10:14 |
| gtristan | long term, I think I clearly agree with locallycompact that build-depends belongs entirely at the chunk level and not in stratum | 10:14 |
| locallycompact | stop hypothesising and actually try to do this, change one stratum in the middle of a downstream system and see how much it hurts | 10:14 |
| locallycompact | you have to change every name of every chunk everywhere | 10:15 |
| locallycompact | move stratum depends to the parent type | 10:15 |
| *** rjek has quit IRC | 10:16 | |
| locallycompact | the system can govern the stratum depends, then if you want to alter a stratum you can just copy the stratum in particular, and copy the system and swap out that one stratum | 10:16 |
| gtristan | short term however (as in, lets have something that works yesterday already), I'm _really_ tempted to just remove _everything_ from the definitions tree that is not related to said downstream project, and just have a complete fork | 10:16 |
| gtristan | without any deadcode lying around to confuse me about what exactly in this definitions repo actually gets built | 10:17 |
| locallycompact | this I prefer | 10:17 |
| locallycompact | because then it will force a dialectic | 10:17 |
| *** rjek has joined #baserock | 10:17 | |
| paulsherwood | locallycompact: use easier words, please | 10:18 |
| paulsherwood | unless you don't *want* folks to understand what you're saying :) | 10:18 |
| * gtristan googled that | 10:18 | |
| locallycompact | I want to be precise | 10:18 |
| locallycompact | The scenario is: qt updates in either the one or the other repository, upstream or downstream it doesn't matter | 10:19 |
| locallycompact | And osmeone says hey we updated all the refs to 5.7 or whatever | 10:19 |
| locallycompact | and either way you just want to cherry-pick that file in a merge | 10:19 |
| locallycompact | the fact that one person or the other builds qt on top of this or that system is not relevant, they each have their own merge criteria, they just want to pull in the new refs and run the ci to check all the important systems are still functioning | 10:23 |
| locallycompact | so move the build-depends out of the stratum so they are just collections of repo/ref combinations with local ordering | 10:24 |
| locallycompact | they don't even need names, strictly | 10:24 |
| locallycompact | just chunks: | 10:25 |
| locallycompact | then you can cherry-pick that file in any situation | 10:25 |
| locallycompact | so a single stratum is just a set of "stratum build instructions" agnostic of where it might be situated | 10:27 |
| paulsherwood | works for me. i still want uniqueness, though :) | 10:27 |
| gtristan | well, maybe I dont understand this proposal exactly, but I think ordering inside the stratum loses even more dependency information (and we have a lot that is already missing) | 10:28 |
| locallycompact | maybe it works following this but doing it now is crazy | 10:28 |
| gtristan | i.e. I would really prefer that the whole thing worked with only chunks | 10:28 |
| locallycompact | we want to lose some dependency information | 10:28 |
| locallycompact | we don't want some chunk to claim "no I only build against gcc-4.6" or whatever | 10:28 |
| gtristan | except that the way we have lost dependency information, for instance, just because some corner case chunk has changed in foundation, *all of gnome.morph* needs to build for 4-6 hours | 10:29 |
| gtristan | when probably only a few chunks really need to be rebuilt | 10:29 |
| gtristan | locallycompact, except that we probably want to re-build the entire system when gcc version changes, except for say; installation of some python modules ? | 10:30 |
| gtristan | i.e. if we want repeatable builds (which I think is the value-add of strictly rebuilding everything) | 10:31 |
| locallycompact | when the gcc version changes is the purview of the system, the chunk is just a repo ref and build instructions | 10:36 |
| locallycompact | so my chunk is somewhere in some stratum | 10:37 |
| locallycompact | and now I swap out underneath for a different base and see if it still builds | 10:37 |
| locallycompact | none of the strata or chunks need change | 10:37 |
| *** Guest31139 has quit IRC | 10:37 | |
| locallycompact | build-essential may change if you don't care about keeping around old combinations of build-essential | 10:38 |
| locallycompact | but if you are holding build-essential-4.6 or whatever for some practical purpose it is still in lockstep with upstream | 10:39 |
| locallycompact | as is latest | 10:39 |
| locallycompact | then maybe you can get away with uniqueness after this point | 10:40 |
| locallycompact | but then I don't even see why chunk morphs or strata need even have name: or kind: anymore "chunk morphs" are not chunks, they're just build instructions | 10:41 |
| locallycompact | "chunk" morphs could be solid markdown for all it matters | 10:41 |
| pedroalvarez | hehe, we are throwing in all the ideas at the same time | 10:42 |
| * gtristan adds an organic chicken to the stew | 10:54 | |
| pedroalvarez | that won't make this easier to digest :P | 10:55 |
| paulsherwood | while i am very interested to improve the format, forcing uniqueness would be a quick win to reduce user confusion imo | 10:56 |
| paulsherwood | tangentially, one of the reasons i wanted nesting/generic composite things instead of strata, was the idea of (say) ruby, being ruby and stuff needed for ruby. rather than having ruby stratum contains ruby chunk, ruby composite would have repo+ref, plus other composites | 10:59 |
| paulsherwood | as opposed to the current 'foo contains foo' warnings | 11:00 |
| locallycompact | I get it but they are completely harmless and I'm concerned about stalling downstream development at this moment | 11:01 |
| *** Guest31139 has joined #baserock | 11:01 | |
| paulsherwood | ack | 11:02 |
| paulsherwood | i'll shut up, then | 11:02 |
| pedroalvarez | but moving downstream definitions to a different folder is not a bad idea, right? | 11:03 |
| pedroalvarez | I've considered doing that with genivi definitions, but always forgotten to actually do it | 11:03 |
| gtristan | pedroalvarez, I like that structure, I dislike having the rest of baserock in the same tree though, honestly, unless I'm actually using it | 11:04 |
| pedroalvarez | (even if genivi is not downstream, but hey) | 11:04 |
| pedroalvarez | if not using it, there isn't any point on having a fork, but I guess you mean that you may use "core" but not "openstack" strata | 11:05 |
| gtristan | hence why I thought submodules might be a good idea, but it would only be interesting if we really only had a few stratum that various definitions project trees would use as submodules | 11:05 |
| pedroalvarez | I'll keep screaming every time I read "submodules" | 11:06 |
| pedroalvarez | sorry | 11:06 |
| locallycompact | submodules are like sick buckets, good in a pinch but you don't want to leave it around for weeks on end | 11:06 |
| gtristan | yeah I got it, it's not great | 11:07 |
| franred | the problem is that you need to define a workflow where you generalize a definition from a "submodule" to the general if more than one system/strata uses it... or are we suggesting defining the same thing in multiple places? | 11:07 |
| gtristan | franred, I think I would prefer a completely separate definitions repo for each project, with variable maintenance levels | 11:09 |
| gtristan | franred, if some system became a "thing" at some point and then nobody cares about it, just let it die, at least it's not sitting there in definitions adding bloat | 11:09 |
| gtristan | submodules was just a (weak) idea of how code sharing might work in such a scenario | 11:10 |
| franred | ok, sounds sensible approach in one side, but if you need to maintain multiple projects, i.e. update a bug in gcc, you need to add these patch to all the repos/systems | 11:12 |
| franred | and you CI system won't apply per definitions but per project/system in this case as well | 11:12 |
| gtristan | franred, so if build-essential were a submodule, then maintainers of those separate projects would get to decide when their definitions repos pull in the new commits from the build-essential submodule | 11:13 |
| gtristan | I say this but also note that I do, generally dislike submodules; my experience with them is that they just add hassle | 11:14 |
| franred | it doesn't sounds bad for me. i.e. in that scenario we would have openstack-kilo working with old versions of its components but at least we would be sure it still alive as we configured and not what we have know which is a big unknown | 11:15 |
| franred | as an example of dead systems in definitions at the moment | 11:15 |
| gtristan | true, right now someone can just walk in and upgrade systemd without checking that every "maintained system" actually still boots to a graphic interface | 11:16 |
| gtristan | which is a problem | 11:16 |
| paulsherwood | franred: not entirely dead. i build them as part of ci | 11:16 |
| paulsherwood | but not deployed, tested | 11:16 |
| franred | paulsherwood, build doesn't means working | 11:16 |
| paulsherwood | no, true. but sometimes failed builds in the openstack have flagged a problem elsewhere | 11:17 |
| paulsherwood | eg the 'too many pies' issue | 11:17 |
| franred | paulsherwood, on the other hand, if it wouldn't change we would have that system open-running and deployable.. | 11:17 |
| paulsherwood | franred: ack. that just comes down to cost vs value | 11:18 |
| franred | gtristan, suggestion is something that jjardon and other members suggested in the past. We should think on what is valuable for the actual users of baserock. In that approach the systems will be still reproducible and provenance preserve. We lose the build all things with latest changes and make the maintenance of all the systems dependable on the interest of these systems maintainers. | 11:25 |
| franred | but also means that we will need to keep more metadata available in our servers? one for each version of the same package used in different definitions/repositories/systems | 11:26 |
| *** tiagogomes has quit IRC | 11:34 | |
| *** tiagogomes has joined #baserock | 11:42 | |
| *** gtristan has quit IRC | 12:33 | |
| locallycompact | the other monumental advantage to having strata only be a dictionary of chunks: is it would be easy write back changes in jq | 12:35 |
| locallycompact | so you could do single point insertions in shell | 12:35 |
| franred | jq? | 12:50 |
| locallycompact | json filter language https://stedolan.github.io/jq/tutorial/ | 12:52 |
| locallycompact | so you could for example move a chunk up or down a strata and that's basically one line of jq | 12:57 |
| *** cosm has joined #baserock | 12:58 | |
| edcragg | and use chunks in multiple strata? | 12:59 |
| *** cosm has quit IRC | 13:02 | |
| persia | strata exist for the convenience of system developers, so they don't have to understand chunks (assuming the chunk maintainers are sufficiently capable). | 13:02 |
| *** cosm has joined #baserock | 13:02 | |
| persia | Use of the same chunk in multiple strata is a common way to manage edge cases whilst developing the correct taxonomy for strata within a wider project. | 13:02 |
| persia | Downstream efforts are almost always going to be slightly at odds with any upstream taxonomy, meaning that it is hugely advantageous to be able to use chunk in multiple strata to allow separation of chunk maintenance and strata integration within the downstream project. | 13:03 |
| locallycompact | chunks *only* exist in strata, mereologically | 13:04 |
| persia | Yes, but not everyone thinks about it that way. | 13:06 |
| persia | The sense of a chunk being an independent thing with which one can do something remains strong, heavily influenced by the use of "packages" in distributions. | 13:06 |
| persia | And many downstream projects are more heavily influenced by those semantics than we. | 13:07 |
| pedroalvarez | paulsherwood: any findings in the CI after fixing latest bug? | 13:10 |
| paulsherwood | c.b.o seems to be down for me... | 13:15 |
| pedroalvarez | :S | 13:18 |
| pedroalvarez | paulsherwood: did you bring it to life? | 13:28 |
| pedroalvarez | it's up again | 13:28 |
| pedroalvarez | same thing happened a while back, maybe the hosting provider saving resources? idk | 13:29 |
| * paulsherwood needs to discuss with mwilliams | 13:48 | |
| paulsherwood | pedroalvarez: meanwhile i've kicked off a ci build on master definitions... so far so good 16-08-03 00:30:08 [248/1010/1010] [iptables] | 13:49 |
| pedroalvarez | good | 13:50 |
| paulsherwood | what's the deep change? | 13:50 |
| pedroalvarez | in anycase, I think a change in the cache-key is the thing to do | 13:50 |
| paulsherwood | for which? | 13:51 |
| pedroalvarez | the `sh -e -c` | 13:51 |
| pedroalvarez | is the right* thing to do | 13:51 |
| paulsherwood | oh, yes. that's done. i haven't released it yet, but the build i'm on has that | 13:51 |
| paulsherwood | s/on/running/ | 13:51 |
| pedroalvarez | cool | 13:51 |
| paulsherwood | am i right thinking morph has the same issue? | 13:51 |
| pedroalvarez | yse | 13:52 |
| pedroalvarez | very bad error :( | 13:52 |
| paulsherwood | amazing we've not noticed before now | 13:52 |
| paulsherwood | i guess most make installs are well-behaved | 13:52 |
| locallycompact | most open things have a make install at all | 13:54 |
| paulsherwood | ack. and for any congigure or build steps, the breakage would have been spotted during intgration in most cases | 13:59 |
| *** mwilliams_ct has joined #baserock | 14:01 | |
| mwilliams_ct | concourse's own website appears to be down at the moment, which is making debugging a tad tricky | 14:16 |
| mwilliams_ct | in fact apparently the whole .ci TLD is having issues | 14:17 |
| locallycompact | O_O | 14:18 |
| *** Guest31139 has quit IRC | 15:36 | |
| paulsherwood | pedroalvarez: i think we can call it ok 03:00:14 [924/1010/1010] [libgdata] | 16:20 |
| paulsherwood | hence i've bumped artifact-version: 6, and ybd 16.31 is released | 16:20 |
| pedroalvarez | we are really great integrators \o/ | 16:21 |
| *** jonathanmaw has quit IRC | 16:23 | |
| paulsherwood | pedroalvarez: what makes you say that, now? :-) | 16:24 |
| pedroalvarez | hehe, i wasn't 100% all the chunks were going to be ok | 16:26 |
| pedroalvarez | in fact, I don't think we have tested all of them after that change | 16:26 |
| pedroalvarez | but we are great :) | 16:26 |
| paulsherwood | :-) | 16:26 |
| *** toscalix has quit IRC | 16:51 | |
| paulsherwood | https://tech.slashdot.org/story/16/08/03/1635240/project-hosting-service-fosshub-compromised-embedding-malware-inside-hosted-files | 17:13 |
| paulsherwood | we need to get to reproducible builds as fast as possible | 17:13 |
| *** gtristan has joined #baserock | 17:31 | |
| persia | Perhaps with cryptographically verifiable content, just to be sure. | 17:31 |
| *** tiagogomes has quit IRC | 17:33 | |
| *** cosm_ has joined #baserock | 22:05 | |
| *** cosm has quit IRC | 22:10 | |
| *** cosm has joined #baserock | 22:14 | |
| *** cosm_ has quit IRC | 22:15 | |
| *** cosm_ has joined #baserock | 22:23 | |
| *** cosm has quit IRC | 22:27 | |
| paulsherwood | yup | 22:40 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!