| *** paulwaters_ has joined #baserock | 07:00 | |
| *** toscalix has joined #baserock | 07:44 | |
| *** rdale has joined #baserock | 07:55 | |
| *** jonathanmaw has joined #baserock | 08:12 | |
| *** jonathanmaw has left #baserock | 08:12 | |
| *** tiagogomes has joined #baserock | 09:03 | |
| *** edcragg has joined #baserock | 09:27 | |
| *** locallycompact has joined #baserock | 09:34 | |
| jjardon | Hi, ok to lorry this 2 packages? https://gerrit.baserock.org/#/q/status:open+project:baserock/local-config/lorries+branch:master+topic:lorry-gnu-utilities | 09:56 |
|---|---|---|
| paulsherwood | jjardon: ok by me +1 | 09:58 |
| *** baserockgerrit has quit IRC | 10:02 | |
| *** gtristan has joined #baserock | 10:04 | |
| *** baserockgerrit has joined #baserock | 10:07 | |
| *** baserockgerrit_ has joined #baserock | 10:11 | |
| *** baserockgerrit_ is now known as baserockgerrit__ | 10:11 | |
| *** baserockgerrit has quit IRC | 10:12 | |
| *** baserockgerrit__ has quit IRC | 10:12 | |
| *** baserockgerrit has joined #baserock | 10:14 | |
| *** baserockgerrit has quit IRC | 10:19 | |
| radiofree | hello everyone! | 10:58 |
| radiofree | what are partial deployments and do they still work? | 10:58 |
| paulsherwood | SotK: ^^ | 10:58 |
| paulsherwood | radiofree: iirc it was just providing an untar of a single chunk as an upgrade deployment | 11:07 |
| paulsherwood | (unsafe, since no consideration of dependencies) | 11:07 |
| radiofree | ok | 11:07 |
| * pedroalvarez finds: http://wiki.baserock.org/guides/arm-howtos/#index4h2 | 11:09 | |
| paulsherwood | pedroalvarez: good spot! | 11:10 |
| pedroalvarez | I've never used it, but it looks like you can specify more than one chunk/strata to deploy | 11:10 |
| * paulsherwood thinks ybd *might* be ok with the same mechanism | 11:10 | |
| pedroalvarez | and also looks like you can deploy an upgrade, or a tar, or any kind of deployment | 11:11 |
| pedroalvarez | although deploying upgrades may need to include some chunks from foundation.morph that includes the upgrade scripts | 11:12 |
| radiofree | ok i'll give that a whirl, thanks pedroalvarez | 11:14 |
| *** rdale has quit IRC | 11:20 | |
| * SotK reiterates the "you shouldn't use this in production deployment" warning in that documentation | 12:00 | |
| *** rdale has joined #baserock | 12:43 | |
| paulsherwood | :) | 13:22 |
| jmacs | Should be a metadata flag for all source, that | 13:24 |
| paulsherwood | FIXME: make this safe for production :) | 13:25 |
| radiofree | paulsherwood: it works | 13:26 |
| radiofree | however i think it's just easier to ybd the/strata.morph and deploy that manually | 13:27 |
| paulsherwood | radiofree: cool | 13:27 |
| radiofree | since the partial-deploy tar ball is everything (with just the partially deployed thing built) | 13:27 |
| * paulsherwood would like to see ostree here, rather than all the weirdness | 13:27 | |
| pedroalvarez | what is "all the weirdness"? | 13:28 |
| pedroalvarez | the "non safe for production" bit? | 13:28 |
| paulsherwood | pedroalvarez: i may be wrong, but the deployment extensions struck me as weird in general | 13:30 |
| * paulsherwood is probably conflating things as usual | 13:31 | |
| radiofree | for upgrades, it still requires the system is btrfs right? | 13:31 |
| paulsherwood | yes | 13:31 |
| radiofree | deploying to tarball is useful and works, deploy to rawdisk isn't so useful if you can't use btrfs | 13:31 |
| radiofree | so that leaves us with no meaningful upgrade mechanism | 13:32 |
| jjardon | pedroalvarez: yes, and hard depend on btrfs. Also, ostree is a mature tool to manage updates of systems | 13:32 |
| paulsherwood | to be fair, ostree didn't exist when we first started with the btrfs upgrades :) | 13:32 |
| jjardon | true | 13:33 |
| pedroalvarez | does ostree include dependencies too? | 13:33 |
| pedroalvarez | erm.. | 13:33 |
| jjardon | pedroalvarez: what do you mean? | 13:33 |
| pedroalvarez | will ostree include dependencies too? as btrfs does? | 13:34 |
| paulsherwood | yes, it depends on some things | 13:34 |
| paulsherwood | it's already integrated in some baserock systems iiuc | 13:35 |
| pedroalvarez | as is btrfs, (integrated) but still causing pain | 13:35 |
| paulsherwood | ack | 13:36 |
| pedroalvarez | just asking questions because I'm curious, not criticizing | 13:36 |
| paulsherwood | pedroalvarez: aiui some folks here are working on a non-baserock baserock system... where btrfs will never be feasible... but upgrades are required | 13:38 |
| SotK | using ostree for updates would be nice | 13:38 |
| radiofree | SotK: did you and sam do some work on that in the past? | 13:39 |
| pedroalvarez | that work was more complicated | 13:39 |
| pedroalvarez | it was to use ostree in the whole building process | 13:39 |
| SotK | the previous ostree work was about using ostree in the artifact cache rather than a bunch of tarballs in order to speed up the building and deployment process | 13:40 |
| paulsherwood | did it succeed? | 13:40 |
| pedroalvarez | we found a big blocker | 13:40 |
| paulsherwood | oh? | 13:40 |
| pedroalvarez | this was a long time ago, and I didn't do the job, nor identify the blocker | 13:41 |
| richard_maw | paulsherwood: it did build faster, but Baserock filesystem images weren't able to cope with the deduplication of files, since we assume that files with the same contents may not be hardlinks of each other | 13:41 |
| richard_maw | paulsherwood: the result was that the systems didn't boot properly, because the first thing systemd did was write the machine-id to /etc/machine-id, which was an empty file that was deduplicated with all the other empty files on the system | 13:42 |
| paulsherwood | argh :) | 13:42 |
| richard_maw | paulsherwood: so for stuff where an empty file is a logically consistent configuration, they suddenly had an improperly formatted configuration file | 13:43 |
| pedroalvarez | richard_maw: I remember something about "usr merge" being a fix for some of the blockers of the ostree work | 13:43 |
| paulsherwood | strikes me this has to be either a corner case worth fixing in ostree, or a bug in our systems? | 13:43 |
| richard_maw | paulsherwood: bug in our systems, ostree is designed for read-only /usr and empty /etc | 13:44 |
| richard_maw | easiest fix for systemd-based systems would be to have something move the contents of /etc into /usr/lib/factory/etc and have systemd copy it out of there into /etc on first boot | 13:45 |
| richard_maw | otherwise you might get away with some other "unsharing" of files in /etc | 13:45 |
| locallycompact | paulsherwood, where does the megayaml thing you mentioned end up | 13:50 |
| paulsherwood | definitions.yml in the definitions dir | 13:57 |
| paulsherwood | richard_maw: can you refresh my memory about the loom project, please? did it see the light of day? | 13:58 |
| richard_maw | it did not | 13:59 |
| paulsherwood | nothing in public anywhere? | 13:59 |
| jjardon | there are some issues opened about that: https://storyboard.baserock.org/#!/story/11 , https://storyboard.baserock.org/#!/story/36 and https://storyboard.baserock.org/#!/story/48 | 14:00 |
| richard_maw | the patches to yarn to make its library consumable for it went public, but weren't merged | 14:00 |
| paulsherwood | richard_maw: was there any discussion anywhere about the concepts? | 14:01 |
| richard_maw | not that I recall | 14:17 |
| richard_maw | I did a friday talk at Codethink about it, but don't recall if it was recorded. persia_ might know more since he was involved in drafting the idea. | 14:18 |
| *** jjardon_matrix has quit IRC | 14:18 | |
| *** gtristan has quit IRC | 14:20 | |
| mwilliams_ct | Hey folks! I am making steady headway on getting a trove running on Baserock, thanks to pedroalvarez! One thing we are confused by, where/when/how is git-daemon-export-ok created for repos in /home/git/repos? | 14:40 |
| mwilliams_ct | s/on Baserock/on Debian | 14:40 |
| richard_maw | mwilliams_ct: git-daemon-export-ok is created by gitano | 14:41 |
| mwilliams_ct | richard_maw: ack, that's what pedroalvarez was suspecting. something to do with the ACLs presumably, do I need to play about with the gitano-admin repo (subbing gitano-admin for whatever the repo is called)? | 14:42 |
| richard_maw | mwilliams_ct: https://git.gitano.org.uk/gitano.git/tree/doc/admin/000.mdwn?h=liw/admindoc might be handy if you need a primer into how rules work | 14:43 |
| richard_maw | it's WIP at this point, but feedback would probably be graciously accepted in #gitano | 14:43 |
| mwilliams_ct | thanks richard_maw, will have a look through | 14:44 |
| persia | richard_maw: I never saw such a video recording. I may have an older version of the code somewhere. My memory is that after the initial implementation, Kinnison was drafting some guidelines for integration with Mustard, but I haven't heard since, and that was a long time ago now. | 14:46 |
| persia | paulsherwood: My memory of the loom tooling was that it parsed YAML containing requirements, and then used that to process tests, where the requirements were expressed as a user story, with tags to identify verification criteria as scenarios, and scenarions. Looking through my documentation, I appear to have the loom requirements, in the loom format, but not the code to parse them. | 14:51 |
| paulsherwood | persia: any reason the requirements can not be made public? | 14:52 |
| paulsherwood | reason is i'm trying to flush out any prior art on traceability from requirements to tests, at https://lists.veristac.io/pipermail/trustable-software/2016-September/000044.html | 14:53 |
| * richard_maw has found the code for loom, and the mustard for loom that was its test suite | 14:54 | |
| persia | paulsherwood: I don't know of any, although I don't have a license to redistribute, and some copyright is claimed ("Copyright 2014 Codethink"). | 14:54 |
| persia | richard_maw: As you're the author of most of what I have, you're probably best placed to license it so it can be distributed :) | 14:54 |
| * richard_maw is looking for a branch of the branch of cmdtest that has the yarn changes in | 14:55 | |
| richard_maw | paulsherwood: the branch of cmdtest that has the changes required for loom is http://git.baserock.org/cgit/delta/cmdtest.git/log/?h=baserock/richardmaw/S10275/factor-out-modules-v3 the loom code is in the codethink git server in ct098/loom | 14:56 |
| paulsherwood | richard_maw: ack, thanks. i'll need to get someone to push that somewhere public | 14:57 |
| *** jjardon_matrix has joined #baserock | 14:59 | |
| persia | paulsherwood: Some other prior art on tracing tests to requirements: http://pythonhosted.org/behave/ , https://cucumber.io/ , http://jbehave.org/ , http://behat.org/en/v3.0/ , http://jasmine.github.io/ , and http://blog.liw.fi/posts/yarn/ | 15:08 |
| persia | Note that all of those suffer from compliance with the given-when-then format: none contain any means of tracking that to user stories directly. | 15:09 |
| persia | (which tends to lead to the degenerate format of user stories commonly found in agile (e.g. "As a user, I want to submit identifying characteristics of my device for cross-site tracking.") | 15:10 |
| *** locallycompact has quit IRC | 17:27 | |
| radiofree | should changing the DEFAULTS (adding a splitting rule) force a rebuild of everything? | 17:33 |
| radiofree | we don't actually have a rule for debug symbols, so they end up in misc | 17:33 |
| persia | If you change the split rules, you change the content of all artifacts, so yes, I think it should force a rebuild of everything. | 17:35 |
| persia | radiofree: Are these debug files that are stripped with --only-keep-debug for later troubleshooting of executables? | 17:36 |
| * persia would be very excited if that had been implemented | 17:36 | |
| radiofree | yes, we (correctly) strip debug symbols into /usr/lib/debug/ | 17:37 |
| radiofree | however there's actually no rule for debug splitting in a chunk | 17:37 |
| radiofree | i think that may have been an omission since the stratum level devel refers to -debug | 17:37 |
| persia | Yes, that does seem to be an omission | 17:43 |
| *** toscalix has quit IRC | 17:44 | |
| *** tiagogomes has quit IRC | 17:53 | |
| paulsherwood | radiofree: no, it shouldn't force a rebuild | 17:54 |
| paulsherwood | in ybd artifacts are created with everything | 17:54 |
| paulsherwood | the splitting only applies on creation of strata (which are just manifests) and image assembly | 17:54 |
| radiofree | But the manifest is part of the chunk? | 18:00 |
| radiofree | Or does that get overwritten when you deploy? | 18:00 |
| paulsherwood | radiofree: manifest for a chunk contains all of the splits. stratum manifest identifies specific splits (eg foo-minimal) | 18:01 |
| radiofree | But I added a rule for chunk splitting? | 18:06 |
| radiofree | At what point will they be applied? | 18:06 |
| paulsherwood | stratum creation applies the rules to decide what shoudl be in the stratum (just generate the lists). system creation applies the lists from the strata to create the actual image | 18:07 |
| paulsherwood | (iirc) | 18:08 |
| radiofree | OK, so when will my chunks be split according to the new rules? | 18:09 |
| * radiofree will test this with. deploy at somepoint | 18:11 | |
| radiofree | So why is the chunk manifest in the artifact for a chunk? | 18:11 |
| radiofree | If I create a new splitting rule for a chunk surely something will have to add that to the manifest at some point? | 18:12 |
| * paulsherwood starts to wonder if he's wrong | 18:12 | |
| radiofree | Well before my rule, you'd have /usr/lib/debug/foo.so in "misc" for the chunk | 18:13 |
| radiofree | I add a splitting rule in defaults to move that to debug instead, and build the system again | 18:13 |
| radiofree | But it didn't rebuild anything, and the manifest that says foo.so is in misc is still part of the chunk aftifact | 18:14 |
| radiofree | So at what point is that go to debug (the statum splitting has devel which contains debug) | 18:15 |
| paulsherwood | i think you've found a hole/bug | 18:17 |
| paulsherwood | let me do some investigation | 18:17 |
| paulsherwood | sadly istm that chunks shouldn't care about splits at all, and all splitting should be done at stratum and system | 18:53 |
| paulsherwood | currently the stratum splitting is using split info from the chunk metadata, which means the current scheme can't support radiofree's usecase :/ | 18:54 |
| paulsherwood | https://gitlab.com/baserock/ybd/issues/242 | 18:58 |
| paulsherwood | radiofree: can you show me your new defaults please? | 19:01 |
| * paulsherwood is wondering if it's possible just to ignore the chunk metadata | 19:02 | |
| radiofree | paulsherwood: not currently, I'm in the pub, I'll pop back on the way home | 19:21 |
| radiofree | paulsherwood: maybe write the metadata at deploy stage? | 19:22 |
| radiofree | Entire chunk is the artifact, splitting should happen on that | 19:24 |
| radiofree | I don't see why the manifest needs to be in the artifact | 19:24 |
| paulsherwood | i agree | 19:29 |
| paulsherwood | this can wait, enjoy your night | 19:29 |
| *** gtristan has joined #baserock | 19:54 | |
| radiofree | paulsherwood: is there a default kbas server? | 21:07 |
| * radiofree can come up with a quick example if there's cache for build essential | 21:07 | |
| paulsherwood | http://artifacts1.baserock.org:8000 | 21:32 |
| radiofree | ta | 21:35 |
| radiofree | paulsherwood: does that build master? | 21:35 |
| radiofree | well it wants to build stage1 binutils so i guess it does | 21:36 |
| radiofree | s/does/doesn't | 21:37 |
| radiofree | again with the latest tag, i think https://gitlab.com/jamesthomas/simpleproject/tree/master may have to wait until tomorrow | 21:37 |
| radiofree | kbas-url: 'http://artifacts1.baserock.org:8000' is all i need in my config? | 21:37 |
| radiofree | unless you don't build 32-bit? | 21:38 |
| * radiofree try's a 64-bit chroot | 21:42 | |
| radiofree | no cache in 64-bit either... | 21:55 |
| radiofree | i get a 500 error with http://artifacts1.baserock.org:8000/ | 21:55 |
| radiofree | so i guess it's not actually working | 21:55 |
| * radiofree gives up for the evening | 21:56 | |
| radiofree | however, there's a very simple set of rule fixes that should be able to save the project i'm working on quite a significant amount of deploy and testing time | 21:56 |
| radiofree | simple by virtue of not having to extract such a fat system to a usb | 21:57 |
| *** gtristan has quit IRC | 22:07 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!