*** locallycompact has quit IRC | 00:04 | |
*** JPohlman1 has joined #baserock | 02:11 | |
*** JPohlmann has quit IRC | 02:15 | |
*** edcragg has joined #baserock | 07:14 | |
*** toscalix has joined #baserock | 08:02 | |
*** fay_ has joined #baserock | 08:12 | |
*** fay_ is now known as faybrocklebank | 08:17 | |
*** gtristan has joined #baserock | 08:20 | |
*** edcragg has quit IRC | 08:50 | |
*** bfletcher has quit IRC | 08:52 | |
*** bashrc has joined #baserock | 08:58 | |
*** CTtpollard has quit IRC | 08:59 | |
*** CTtpollard has joined #baserock | 09:01 | |
*** tiagogomes_ has joined #baserock | 09:07 | |
*** edcragg has joined #baserock | 09:13 | |
*** bfletcher has joined #baserock | 09:18 | |
*** ctbruce has joined #baserock | 09:26 | |
*** jonathanmaw has joined #baserock | 09:27 | |
*** tiagogomes_ has quit IRC | 09:33 | |
*** edcragg has quit IRC | 09:35 | |
*** edcragg has joined #baserock | 09:38 | |
*** tiagogomes_ has joined #baserock | 09:40 | |
*** ssam2 has joined #baserock | 10:17 | |
*** ChanServ sets mode: +v ssam2 | 10:17 | |
*** ctbruce has quit IRC | 10:26 | |
*** paulw has joined #baserock | 10:27 | |
paulsherwood | gtristan: any chance WebkitGtk could be put lower down the stack, so it starts building earlier? | 10:38 |
---|---|---|
*** paulw has quit IRC | 10:38 | |
gtristan | paulsherwood, I had (have ?) a patch for that | 10:38 |
paulsherwood | ooh, cool! :) | 10:39 |
gtristan | what it comes down to is reorganization of strata | 10:39 |
paulsherwood | yup | 10:39 |
gtristan | my first attempt was denied because "i was creating too many strata" :) | 10:39 |
paulsherwood | well, i'd like to see gnome stratum split up - it makes a mess of concourse | 10:39 |
paulsherwood | how many strata did you have? | 10:40 |
gtristan | first I had one-per-chunk, then I reorganized it so that geoclue and a few more chunks lived inside a webkitgtk strata | 10:40 |
paulsherwood | -1 on one-per-chunk | 10:41 |
gtristan | now that we have webkitgtk and webkitgtk1 (the 2 apis)... we'd need one to hold the deps of both and then 2 separate webkit strata I guess | 10:41 |
gtristan | I would really love to see one-chunk-one-file with all deps explicitly listed... this would solve the whole layering problem :-/ | 10:42 |
paulsherwood | that creates different problems | 10:42 |
paulsherwood | let's make the current scheme work first, optimise later | 10:43 |
pedroalvarez | I imagine that one-per-chunk will create a really ugly concourse graph | 10:44 |
paulsherwood | true | 10:44 |
ssam2 | we could still define groups of some sort | 10:44 |
paulsherwood | also true | 10:44 |
paulsherwood | ssam2: did you get much feedback from your talk? | 10:44 |
ssam2 | not really | 10:46 |
paulsherwood | stunned silence :) | 10:46 |
paulsherwood | i enjoyed it, anyway | 10:46 |
ssam2 | nobody used the questions to prove it all totally wrong, which is a good sign | 10:47 |
paulsherwood | :) | 10:50 |
*** locallycompact has joined #baserock | 10:55 | |
*** gtristan has quit IRC | 11:16 | |
jjardon | webkitgtk1 is not part of the gnome core components, so if we split the apps in another stratum it would make things faster for people only interested on building the gnome core modules | 11:20 |
paulsherwood | +1 | 11:21 |
jjardon | uh, reorganizing this is going to be fun :) | 11:27 |
*** CTtpollard has quit IRC | 11:28 | |
*** CTtpollard has joined #baserock | 11:28 | |
paulsherwood | locallycompact: maybe you shouldn't be pushing to a production kbas when you're developing a chunk? :) | 11:33 |
paulsherwood | locallycompact: disable kbas-url, remove the chunk artifact | 11:34 |
paulsherwood | locallycompact: pls could you paste the build log for the artifact? | 11:34 |
locallycompact | I thought you had kbas-url hardcoded | 11:34 |
paulsherwood | password isn't :) and anyway you can override it | 11:35 |
paulsherwood | i think i reverted default kbas-url recently | 11:35 |
locallycompact | can you delete go* from the server so I can get a build log out | 11:36 |
pedroalvarez | lol | 11:36 |
paulsherwood | locallycompact: you can get a build log out by changing *anything* in your go.morph file | 11:36 |
* paulsherwood proposes not to tamper with the production kbas, but may change the password :-) | 11:37 | |
locallycompact | no change to either the chunk or the strata is changing the cache key or triggering a rebuild | 11:43 |
locallycompact | ah, got it | 11:50 |
locallycompact | paulsherwood, it doesn't warn if the chunk morph doesn't exist, I had mispelled it and it just built an empty thing | 11:51 |
paulsherwood | locallycompact: good catch! | 12:02 |
*** rdale has joined #baserock | 12:02 | |
*** rdale_ct_ has joined #baserock | 12:04 | |
*** rdale_ct has quit IRC | 12:05 | |
*** rdale has quit IRC | 12:07 | |
jjardon | mmm, Im getting this error in ybd: | 12:19 |
jjardon | https://www.irccloud.com/pastebin/EnmM3WOn/ | 12:19 |
jjardon | any idea? | 12:19 |
paulsherwood | jjardon: please use master? | 12:22 |
jjardon | paulsherwood: Nevermind, I was using morph: strata/NetworkManager/ModemManager.morph instead morph: strata/NetworkManager-common/ModemManager.morph | 12:22 |
* paulsherwood is considering tagging again | 12:23 | |
jjardon | paulsherwood: master error message for this case is not much better: | 12:24 |
jjardon | https://www.irccloud.com/pastebin/n7y18y6I/ | 12:25 |
paulsherwood | that's better, imo | 12:26 |
paulsherwood | it's saying you've not defined build-system, and it can't autodetect it | 12:26 |
paulsherwood | i decided to leave in the stack-trace, but maybe it's inelegant :) | 12:27 |
paulsherwood | i'll improve it, while fixing 11:51 < locallycompact> paulsherwood, it doesn't warn if the chunk morph doesn't exist, I had mispelled it and it just built an empty thing | 12:28 |
jjardon | sure, but the real problem is that I'm referencing the morph file incorrectly; anyway problem solved now :) | 12:28 |
*** gtristan has joined #baserock | 12:28 | |
paulsherwood | jjardon: what would be the ideal error message? | 12:28 |
paulsherwood | (assuming ybd can't be psychic) | 12:29 |
jjardon | paulsherwood: The error message from morph was enough for me: "ERROR: Couldn't find morphology: strata/NetworkManager/ModemManager.morph referenced in strata/NetworkManager-common.morph" | 12:31 |
paulsherwood | ok | 12:31 |
rdale_ct_ | i think the word 'morphology' is totally redundant | 12:35 |
rdale_ct_ | should be 'definition' | 12:35 |
paulsherwood | +1 | 12:36 |
rdale_ct_ | probably 'redundant' is the wrong word, 'confusing' might be better | 12:38 |
*** paulw has joined #baserock | 12:48 | |
jjardon | ok to move polkit to foundation instead of creating a stratum only for it? seems a common enough component in any Linux system | 12:50 |
pedroalvarez | something that is only currently in gnome>? | 12:52 |
jjardon | mmm, thinking twice I think Im going to separate it | 12:53 |
jjardon | paulsherwood: even systemd can use it if its present in the system | 12:53 |
jjardon | pedroalvarez: ^ | 12:53 |
jjardon | there are quite a lot stuff in the gnome stratum that really belongs to other layers | 12:53 |
pedroalvarez | yeah, i can see that | 12:53 |
pedroalvarez | but I'm not sure about including all of that components in foundation | 12:54 |
jjardon | indeed, does privileges-management look good as a stratum name? | 12:55 |
* paulsherwood can't comment on these components specifically | 12:55 | |
jjardon | ssam2: Hey! did you see yesterday comments about morph not using DEFAULTS? there is a patch to fix that on gerrit but I guess we have to increase the definitions version as well? | 13:02 |
pedroalvarez | I created a story for the real problem: https://storyboard.baserock.org/#!/story/72 | 13:02 |
ssam2 | oh, dear, so this is a Morph bug that means it ignores the DEFAULTS file even when VERSION = 7 ? | 13:07 |
pedroalvarez | yup | 13:07 |
ssam2 | i'm not sure we need to treat it as a new definitions version when Morph has a bug | 13:07 |
ssam2 | in this case | 13:08 |
pedroalvarez | the patch is not fixing morph for this, the patch includes a new build system | 13:08 |
ssam2 | the definitions format version 7 says that you should honour DEFAULTS | 13:08 |
ssam2 | so if Morph doesn't, that's a bug in morph, not a change in the spec | 13:08 |
ssam2 | oh, Morph still supports definitions V6, which affects https://gerrit.baserock.org/#/c/1788/ | 13:11 |
ssam2 | but there is already a patch with +2 to remove support for V6, it just needs rebasing I think | 13:11 |
* ssam2 comments in Gerrit | 13:11 | |
pedroalvarez | oh, good to see that there are some patches to fix the problem | 13:14 |
pedroalvarez | I wonder if we can persuade tiagogomes_ to at least tell us if any of the patches in https://gerrit.baserock.org/#/q/topic:cleanup-morphologyloader fixes this issue | 13:20 |
tiagogomes_ | my patches shouldn't fix anything. It was just some cleanups | 13:24 |
pedroalvarez | well, if they remove support for definitions version 6, that means that they should honour DEFINITIONS | 13:24 |
pedroalvarez | sorry, DEFAULTS | 13:24 |
jjardon | Does someone have some time to give some feedback about https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/definitions+branch:master+topic:reorganization_ofono_modemmanager ? | 13:25 |
ssam2 | it does look like my patches for DEFAULTS in Morph were totally and utterly broken | 13:25 |
ssam2 | that's embarassing | 13:25 |
rdale_ct_ | why is tiago copying json schemas from definitions into morph? | 13:26 |
ssam2 | I remember discussing this | 13:27 |
ssam2 | it's because i didn't want to make Morph's behaviour rely on having some arbitrary files in any definitions repo that it might ever want to build | 13:28 |
richard_maw | rdale_ct_: The tool that uses the data needs to be the one that owns the validation schema, not the data. | 13:28 |
richard_maw | Otherwise data could pass validation, but be completely incomprehensible to the tool. | 13:28 |
mwilliams_ct | rdale_ct_: https://irclogs.baserock.org/%23baserock.2015-12-04.log.html has some background I think | 13:28 |
rdale_ct_ | thanks for the link - i can see a discussion, but i don't see any agreement | 13:32 |
locallycompact | Me again. When I do these same commands in a build system it works, but done in the chunk morph it reports Read-only file system. https://paste.fedoraproject.org/321197/45519767/ | 13:38 |
richard_maw | o_O go builds in /var/tmp‽ | 13:38 |
locallycompact | ah, I can overwrite I think | 13:40 |
locallycompact | sec | 13:40 |
richard_maw | /var/tmp is not in the set of paths morph lets you write to IIRC | 13:40 |
*** toscalix has quit IRC | 13:44 | |
richard_maw | :¬/ that error is because go dist assumes that if TMPDIR is not set, it should fall back to using /var/tmp | 13:45 |
richard_maw | rather than /tmp, which would have been sensible | 13:45 |
richard_maw | so export TMPDIR=/tmp would fix it | 13:45 |
locallycompact | yeah | 13:46 |
richard_maw | ffs go! | 13:46 |
*** gtristan has quit IRC | 13:47 | |
*** toscalix has joined #baserock | 13:52 | |
jjardon | mmm, seems ybd is stuck in an infinite loop here: | 13:57 |
jjardon | https://www.irccloud.com/pastebin/QvHHta0a/ | 13:57 |
jjardon | paulsherwood: known bug? ^ | 13:57 |
ssam2 | it's like a 'which build tool is more broken?' contest in here today :-) | 13:57 |
jjardon | :) | 13:58 |
pedroalvarez | yocto wins | 13:58 |
paulsherwood | unfortunately i don't understand the splitting code.... rdale_ct_ ??? | 13:59 |
rdale_ct_ | i don't think the splitting code can cause a loop - does the polkit stratum have splitting in it? | 13:59 |
*** toscalix has quit IRC | 14:01 | |
pedroalvarez | locallycompact: you would help us a lot if you send your patches over gerrit :/ | 14:01 |
pedroalvarez | it's not that bad :( | 14:01 |
*** toscalix has joined #baserock | 14:02 | |
paulsherwood | jjardon: pls could you publish your definitions? | 14:02 |
rdale_ct_ | perhaps something is going wrong with the build of libndp, and an exception gets thrown before it writes out a libndp.meta file - that file is used to tell if something has already been built | 14:04 |
paulsherwood | i had hoped exceptions were exposed, now... but that sounds like a reasonable theory | 14:04 |
paulsherwood | jjardon: i assume you're using master? | 14:04 |
jjardon | paulsherwood: http://git.baserock.org/cgit/baserock/baserock/definitions.git/log/?h=jjardon/reorganization | 14:08 |
jjardon | paulsherwood: yes, Im using ybd master | 14:08 |
jjardon | (I've just tried and it seems to build with morph) | 14:08 |
paulsherwood | eek :) | 14:11 |
rdale_ct_ | is there anything in the libndp build log? | 14:15 |
pedroalvarez | jjardon: you might be interested on this one: https://gerrit.baserock.org/#/c/1794/1 | 14:17 |
paulsherwood | rdale_ct_: it's not being created | 14:21 |
paulsherwood | so problem is earlier than that | 14:21 |
jjardon | pedroalvarez: oh nice! thanks for the quick patch ssam2 ! | 14:22 |
* jjardon abandon his quick fix | 14:23 | |
jjardon | pedroalvarez: can we remove the -1 in https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/definitions+branch:master+topic:python3-distutils now? :) | 14:27 |
pedroalvarez | sure | 14:28 |
jjardon | paulsherwood: rdale_ct_ FYI, last tag of ybd fails in a different way: | 14:29 |
jjardon | https://www.irccloud.com/pastebin/Y0RCEka5/ | 14:29 |
paulsherwood | jjardon: unfortunately i'm fighting a headache, will get to this later | 14:31 |
jjardon | sure | 14:32 |
*** gtristan has joined #baserock | 14:36 | |
*** locallycompact has quit IRC | 14:38 | |
*** locallycompact has joined #baserock | 14:40 | |
*** locallycompact has quit IRC | 15:46 | |
*** inara has quit IRC | 15:52 | |
*** inara has joined #baserock | 16:02 | |
*** CTtpollard has quit IRC | 17:06 | |
*** jonathanmaw has quit IRC | 17:19 | |
paulsherwood | jjardon: my brain is not working very well, but removing some redundant build-depends.... http://paste.baserock.org/unefehatul | 17:25 |
*** locallycompact has joined #baserock | 17:28 | |
jjardon | paulsherwood: mmm, I like more explicit dependencies; things start breaking easily when you move components between strata if you use implicit dependencies | 17:32 |
*** ssam2 has quit IRC | 17:32 | |
paulsherwood | we went through this before | 17:32 |
paulsherwood | it became a nightmare | 17:32 |
locallycompact | Is there any way to make things from npm reproducible in baserock or is that another black hole I should just use ansible for? | 17:32 |
*** ssam2 has joined #baserock | 17:32 | |
*** ChanServ sets mode: +v ssam2 | 17:32 | |
paulsherwood | build-depends are transitive | 17:33 |
paulsherwood | (i hoped that this wasn't true, originally, but it was proved to be the case) | 17:33 |
rdale_ct_ | but that still shouldn't cause ybd to loop though | 17:34 |
paulsherwood | i agree. but i'm failing completely to figure out what's going on | 17:35 |
paulsherwood | and i do know that those depends are redundant | 17:35 |
paulsherwood | (and once removed, the loop doesn't happen) | 17:35 |
rdale_ct_ | ah | 17:35 |
paulsherwood | i'll clean it up so ybd reports it properly, but still i don't think it's right to allow the redundancy, even if morph builds it | 17:38 |
paulsherwood | npm? i thought we already did an importer for that | 17:39 |
paulsherwood | locallycompact: ^^ | 17:40 |
paulsherwood | i may be wrong | 17:40 |
ssam2 | zara did one, indeed | 17:40 |
ssam2 | as part of http://git.baserock.org/baserock/baserock/import.git | 17:40 |
Zara | yeah, over a year ago; I'd be very surprised if it worked :/ | 17:40 |
Zara | I think the npm registry has changed several things significantly since then. | 17:41 |
jjardon | paulsherwood: Id be happy if ybd warns me about redundant dependencies then | 17:41 |
* paulsherwood too :) | 17:41 | |
ssam2 | git.baserock.org seems super overloaded | 17:41 |
paulsherwood | jjardon: i hope to do that tonight | 17:42 |
* locallycompact prods gbo | 17:42 | |
paulsherwood | ssam2: probably concourse folks hammering it | 17:42 |
ssam2 | i can't even ssh in | 17:42 |
benbrown_ | everything is orange, I don't think it's us? | 17:43 |
paulsherwood | ouch. maybe ddos? | 17:43 |
ssam2 | oh, in the OpenStack status page it says 'Migrating' | 17:45 |
ssam2 | I presume that is something DataCentred are doing | 17:45 |
paulsherwood | :/ | 17:45 |
paulsherwood | was there any notification? | 17:45 |
ssam2 | didn't see any | 17:46 |
Zara | (oh, also, I don't think the npm importer guarantees things are reproducible, as npm uses semver for versioning, so the transitive dependencies could differ.) | 17:47 |
pedroalvarez | it supposed to be a 'live migration' | 17:47 |
Zara | (it's mainly a tool for generating morphs for an npm package and its dependencies faster than it would be to do it by hand. I *think* it should still work for that, but with more errors than a year ago.) | 17:49 |
paulsherwood | i think that's what locallycompact is seeking, actually | 17:50 |
locallycompact | ostensibly, yes I think so | 17:50 |
locallycompact | plus the lorries | 17:50 |
Zara | yeah, it should hopefully still work for that for many, though it's covered in cobwebs | 17:50 |
Zara | docs: http://wiki.baserock.org/guides/import-tool/npm/ , worth checking against reality | 17:52 |
Zara | I believe baserock itself has also changed significantly since. | 17:52 |
locallycompact | I'll let you know | 17:52 |
Zara | :) | 17:52 |
pedroalvarez | I've opened requested help to the host provider. I'll let you know whenever I got any news about git.baserock.org | 17:55 |
*** bashrc has quit IRC | 18:02 | |
*** toscalix has quit IRC | 18:06 | |
pedroalvarez | well, the host provider is aware of the issues. They are trying to fix it now | 18:10 |
*** locallycompact has quit IRC | 18:18 | |
*** faybrocklebank has quit IRC | 18:19 | |
*** edcragg has quit IRC | 18:34 | |
ssam2 | git.baserock.org is back | 18:44 |
paulsherwood | yay :) | 18:45 |
*** ssam2 has quit IRC | 18:45 | |
paulsherwood | rdale_ct_: are you still around? | 19:09 |
rdale_ct_ | yes | 19:50 |
paulsherwood | http://paste.baserock.org/zagawohasu | 20:02 |
paulsherwood | this is what's causing the loop for jjardon's usecase | 20:03 |
paulsherwood | claim() triggers because of that exception | 20:03 |
* paulsherwood is still trying to understand the splitting code | 20:03 | |
paulsherwood | regexp was never my strong suit | 20:05 |
rdale_ct_ | i not sure why it would need to chdir(), i'll have a look at the code | 20:05 |
paulsherwood | at that point, on a rerun, sandbox has: ls /src/tmp/tmp4j_Sn8 | 20:07 |
paulsherwood | baserock dev etc lib strata tmp usr | 20:07 |
rdale_ct_ | i suppose the chdir() reverts to where the app was before outside the scope of the with clause | 20:07 |
rdale_ct_ | why is it trying to write to /baserock/strata rather than /baserock that seems to be the problem? | 20:08 |
paulsherwood | quite :) | 20:09 |
paulsherwood | i wonder why strata exists at all? | 20:09 |
rdale_ct_ | in fact this part of the path is wrong too: /src/tmp/tmpPHvHrP/strata/privileges-management.inst/ | 20:09 |
rdale_ct_ | with an excess strata | 20:09 |
paulsherwood | that might be me, please hold. | 20:09 |
paulsherwood | no, seems it predates my recent messing | 20:12 |
paulsherwood | anyway, i think it's dinner time here | 20:12 |
paulsherwood | i'll take a deeper look later | 20:12 |
rdale_ct_ | ok, i'll look here again in a bit | 20:13 |
rdale_ct_ | the chunk name and path stuff is all a bit of a mess | 20:13 |
rdale_ct_ | in jjardon's diffs: +- name: strata/privileges-management | 20:18 |
rdale_ct_ | in the systems/gnome-system-x86_64.morph definition | 20:19 |
rdale_ct_ | why do you need both a 'name:' and a 'morph' field to specify a stratum apart from it being error prone in that their name values can conflict | 20:24 |
*** edcragg has joined #baserock | 20:32 | |
*** edcragg has quit IRC | 21:27 | |
paulsherwood | aha, yes. | 21:35 |
paulsherwood | we've been around this too. i would prefer name only, and insist on uniqueness, but all previous definitions would be broken that way | 21:36 |
paulsherwood | in the end, path is the only true way to distinguish foo from bar. but then we have a load of definitions with no path, because they don't need a morph file, being autodetected | 21:37 |
paulsherwood | hence one of the most horrible functions ever written in python... https://github.com/devcurmudgeon/ybd/blob/master/ybd/definitions.py#L143 | 21:38 |
* rjek makes a dwfinintion with a legit path of ERROR | 21:40 | |
paulsherwood | ack | 21:40 |
rjek | :-) | 21:40 |
rdale_ct_ | 'all previous definitions would be broken that way' - that's only a problem if you don't believe in migrations | 21:51 |
paulsherwood | rdale_ct_: i believe in making tools backward compatible as far as possible. we have clear evidence that some users are not even upgrading morph, for fear of breaking something | 22:03 |
paulsherwood | for ybd, as new kid on the block, i'm doing my best to make it deal with idiosyncrasies that arose before it existed | 22:05 |
paulsherwood | failing, in many cases... but i'm trying :) | 22:06 |
rdale_ct_ | yes, i'm impressed with the velocity of ybd development, but 'fear of breaking something' is because the migration process in morph hasn't been thougth through and users regard moving onto a new version of the schema is a problem. it shouldn't be imho | 22:19 |
paulsherwood | it's not just that, tbh. morph has expressly broken things for users a number of times, because of some fundamental difficulties as a result of early assumptions | 22:22 |
paulsherwood | the technical ground-breaking of morph has been great, but we didn't always take users into account properly | 22:23 |
paulsherwood | (and i speak as both a sometime contributor to morph, and a longtime user) | 22:24 |
paulsherwood | as ssam said in his talk, some of the things we are now confident about are expressly the result of doing the exact opposite previously | 22:26 |
paulsherwood | coming back to this bug... | 22:26 |
paulsherwood | ideally ybd would run json-schema against user's definitions and scream, long before a build has started | 22:27 |
paulsherwood | (and further... if we are confident in migrations, that step could notice an old version, report it, run migrations, build against that...) | 22:28 |
rdale_ct_ | i think we need to attack this on two fronts: firstly make migrations work and stop thinking that the idea of supporting 'everything that has every existed' is a good idea, and secondly decouple the process of defining baserock schemas from the projects consuming them by writing adapters for schema to project so that ybd or morph can use plugins from the latest upstream baserock schema or use a legacy schema with the default | 22:34 |
rdale_ct_ | Definitions parser | 22:34 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!