IRC logs for #baserock for Friday, 2015-01-16

jjardon_ is now known as jjardon00:38
*** genii [~quassel@ubuntu/member/genii] has quit [Remote host closed the connection]03:07
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]07:06
*** zoli_ [] has joined #baserock07:06
*** zoli_ [] has quit [Changing host]07:06
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock07:06
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]07:06
*** zoli_ [] has joined #baserock07:08
*** zoli_ [] has quit [Changing host]07:08
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock07:08
*** petefoth [] has joined #baserock07:27
*** mdunford [] has joined #baserock08:04
*** mariaderidder [~maria@] has joined #baserock08:16
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]08:22
*** mariaderidder [~maria@] has quit [Ping timeout: 245 seconds]08:45
*** mariaderidder [~maria@] has joined #baserock08:49
*** wdutch [] has joined #baserock09:02
Mode #baserock +v pedroalvarez_ by ChanServ09:17
pedroalvarez is now known as pedroalvarez09:19
Mode #baserock +o pedroalvarez by ChanServ09:19
*** jonathanmaw [] has joined #baserock09:24
*** mariaderidder [~maria@] has quit [Quit: Ex-Chat]09:29
Kinnison_ is now known as Kinnison09:31
*** tiagogomes_ [] has joined #baserock09:37
paulsherwoodrdale: sorry, but the world at large has no interest in ct internal systems, so no point in lorrying those repos at gbo09:38
rdaleah ok09:38
* rjek_ applauds straycat's vim change.09:44
rjek_ is now known as rjek09:44
SotKCould I get a quick review on this lorry? I want to use it when caching tree SHAs.
Kinnisonwhy using https rather than git:// ?09:46
KinnisonAnd do you truly need an LRU cache?09:47
Kinnisonrather than just keeping the active set of SHAs needed in a file in the definitions?09:47
persiaI'm not a huge fan of the file solution, because the existence of a file invites folk to try to change it, regardless of the documentation09:48
KinnisonI see09:48
SotKKinnison: oops, I pasted and forgot to change to git://09:49
KinnisonSotK: I don't mind either way09:49
* Kinnison allows +1 for either git:// or https:// (but I feel git:// will be faster)09:49
franred_Kinnison, ummm, we have a lot of https://bla@github .. would be worth to change them to git:// ?09:50
Kinnisondon't change stuff unless you need to09:50
KinnisonIn *theory* we should prefer https09:51
Kinnisonbut since we don't actually validate the certificates, there's no security gain09:51
persiaI thought that with the ca-certificates chunk, we *could* validate the certificates.  Should we start doing so?09:52
pedroalvarezfranred_: lorry controller ignores them09:54
pedroalvarezignores, or not uses them 09:54
De|ta_ is now known as De|ta09:55
Kinnisonpersia: I think l-c needs a way to be given extra certificates to trust before we can turn that on09:55
pedroalvarezfranred_: the use of https I mean, no the lorry itself09:55
persiaKinnison: Why l-c specifically?  We have a means to add certificates generally.09:55
Kinnisonpersia: because it's configuration data for lorrying09:55
*** franred_ [] has quit [Quit: Leaving]09:56
Kinnisonpersia: and I don't know about anyone else, but I don't want to have to redeploy a lorrying system (trove/whatever) just to get a trust anchor in place for lorrying09:56
*** franred [] has joined #baserock09:56
persiaAh, as in the lorry file should specify the certificate in some way, rather than just blindly trusting a set of CAs?09:56
* Kinnison is, as always, a statistical outlier09:56
Kinnisonpersia: essentially I'm thinking there should be an *additional* ca bundle in local-config/lorries.git09:56
persiaAnd if we're just adding a new certificate, redploy should be fast (tiny delta).09:57
Kinnison*iff* we get to the point of not needing reboots09:57
persiaHow long does a reboot take?  Isn't it just a couple minutes?09:59
persiaOr is the issue that the lorry controller is running in the same environment as the git server, the cache server, etc. so everything falls down at once?09:59
KinnisonNot if there're lorries in progress which would have to be waited for, etc10:00
KinnisonI'm just imagining lorrying now10:00
Kinnisonignoring the shared context that trove *currently* causes10:00
Kinnisonif you have a lorrying effort in progress, cancelling or waiting for it, in order to add a ca certificate just for lorrying seems sad10:00
persiaAh, right, and given a sensible system size, there are probably enough components to assume constant lorrying.10:03
* Kinnison should really have explained that at the start of his commentary10:03
* Kinnison has difficulty realising that what is in his head, is often *only* in his head10:03
persiaIn that case, even if the reboot issue is resolved, unless there is a way to continue without breaking lorries-in-progress, it makes sense to have a separate CA store for l-c.10:04
persiaSince it is a small config change to cause an additional CA store to be used, and there is an online command that refreshes the current CA database, this is not a blocking issue.10:04
Kinnisonwell, without a need to reboot, updating the ca bundle should not adversely affect any running processes10:04
persiaAh right, and we don't care if currently-running lorries don't have the updated certificates.10:06
KinnisonIf we were to go down this route, I'd also like it if we could put ssh host keys into the config10:07
Kinnisonso that in case of mob-ssh pull uris we can pre-accept the keys, ditto for upstream troves10:07
Kinnisonrather than the current approach10:07
*** Krin [] has joined #baserock10:11
persiaSounds sensible.10:12
persiaIf you have a good idea how this might work, it is probably worth documenting the model on the wiki, and referencing it from the stories page.10:12
KinnisonSadly all I have is nebulous burblings at this time10:13
DavePageKinnison: DNSSEC and SSH keys in DNS \o/10:14
* Kinnison needs to sort DNSSEC for his domains10:14
* Kinnison also apparently needs to renew his client cert and thence SSL certificates anchored with it10:14
Kinnisonhere goes another few hundred quid :-)10:14
persiaDavePage: Yes, but having that as the *only* way to do things is annoying, especially for folk trying to use the software in large organisations.10:15
*** lachlanmackenzie [] has joined #baserock10:21
*** locallycompact [] has joined #baserock10:22
lachlanmackenzieHow many other toolchains have been generated to build baserock (apart from baserock itself)?10:23
KinnisonA long time ago we used a Debian toolchain to bootstrap us into Baserock, but since then, only toolchains built within Baserock have been used IIRC10:24
KinnisonThese days, morph depends on an awful lot of stuff such that trying to bring it up outside of a Baserock environment will likely be hard10:25
persialachlanmackenzie: What are you trying to accomplish?  There's a lot of flexibility in the definition of the "Baserock toolchain" that may meet your goals.10:27
lachlanmackenziepersia: Hi I'm just doing some research about the practicalities of porting Baserock to a MIPS architecture as part of a project.10:29
persiaSomeone was doing a cross-bootstrap to MIPS some months ago, but I don't remember patches hitting the list.10:30
persiaAside from the mechanical stuff, I think there were some issues with the compiler support at the time, but given the increasing use of MIPS in Debian, I think that's probably sorted now.10:30
persiaIf you try a cross-bootstrap, does anything break?10:30
lachlanmackenzieAt the minute it's just some research, I will have a look at what's been done on MIPS.10:37
persiaMy recommendation would be to ignore that: it was done a long time ago, and with an older version of gcc than the devel systems use today.10:39
persiaJust try a cross-bootstrap, and report what breaks.10:39
persiaWe can probably suggest where to look to work around things, until it works.10:39
KinnisonI believe tiagogomes_ has been working on updating binutils to 2.25 and gcc to 2.9.2 if that helps you10:40
pedroalvarezthat's good.10:44
pedroalvarezDo we know how is that looking?10:44
KinnisonLast I heard from him, he's tracking down an issue in syslinux (-Werror or something)10:45
KinnisonI imagine he'll be posting on baserock-dev soon10:45
tiagogomes_Kinnison is correct.10:46
*** zoli_ [] has joined #baserock10:46
*** zoli_ [] has quit [Changing host]10:46
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock10:46
KinnisonVindication at last!10:46
*** jonathanmaw [] has quit [Ping timeout: 244 seconds]10:46
*** jonathanmaw [] has joined #baserock10:55
*** locallycompact [] has quit [Remote host closed the connection]10:56
*** locallycompact [] has joined #baserock11:13
*** locallycompact [] has quit [Ping timeout: 265 seconds]12:16
straycatrjek, :)12:26
* SotK notices that the ref for morph in the cross-bootstrap stratum is from November, as opposed to January 2nd in morph-utils...13:09
persiaIs there a way to play with strata that would allow cross-bootstrap to automatically pick up that sort of update?13:10
SotKpersia: you mean somehow include morph-utils in cross-bootstrap?13:12
SotKpatch to fix:
persiaThat was my thought,yes.  I'm not sure how to accomplish it with the current semantics though.13:13
richard_mawpersia: we ought to be able to build the "build" system as part of the cross-bootstrap. The cross-bootstrap stratum only exists because we didn't have fine grained strata when it was created, and to ease cycle time we created a stripped-down stratum for it13:13
persialachlanmackenzie: Even if it takes a while to get to master, you might want to apply that patch for cross-bootstrapping13:13
persiarichard_maw: That sounds like a good improvement.  Every time someone tries cross-bootstrap, it seems we find something that we forgot to migrate previously.13:14
mwilliams_cthi folks, I'm trying to spin up a jetson and not having a lot of success. after following the guide on w.b.o, I'm getting the error "*** Warning - bad CRC, using default environment" -- not sure if I've missed a step?13:43
Kinnisonsounds like u-boots environment is corrupt13:45
Kinnisoncan you saveenv from u-boot to clear that?13:45
mwilliams_ctsave'd env and will reboot13:46
mwilliams_ctWekk bew errirs at least! "Retrieving file: /extlinux.conf; btrfs proble failed"13:47
mwilliams_cts/wekk bew errirs/well, new errors/  -- it's definitely a friday13:48
Kinnisonpedroalvarez: regarding the bug you and I diagnosed the other day -- I've finally sent the patch to baserock-dev13:48
Zara_mwilliams_ct: I thought that was some kind of programmer slang I was just unfamiliar with.13:49
persiamwilliams_ct: At least you've successfully demonstrated your classical touchtyping strategy :)13:49
mwilliams_ctpersia: I remember one day noticing that I was touchtyping. never tried to learn it, it just happened!13:50
persiaAre you trying to boot from MMC, or SD?  If SD, how did you generate the SD?13:50
mwilliams_ctI've just followed this guide
persiaTry flashing again.13:53
mwilliams_ctpersia: shall give that a shot13:53
mwilliams_ctthis may take some time13:54
* Kinnison has a question regarding morph and its use in definitions -- in morph-utils the SHA1 looks to be a non-trivial number of commits back on master and the one in cross-bootstrap is from november last year. Is this oversight or deliberate for making things work?13:54
* SotK is pretty sure its an oversight, it always has been before anyway13:55
KinnisonSomeone who can usefully analyse what changed from 30aaa46e250a1bd8081283839abd7d5aab97fb1e probably wants to check and update morph-utils.morph, and then possibly cross-bootstrap.morph too13:56
KinnisonOr perhaps cross-bootstrap.morph could be altered and/or dropped in favour of morph-utils to some extent?13:56
* persia gets confused by a long string of hex13:57
persiaDo you mean between that commit and master?13:58
Kinnisonthat commit being what morph-utils refers to for morph13:58
persiaThe cross-bootstrap one was discussed earlier and thought to be oversight.  richard_maw suggested that we may be able to just build "build" in cross-bootstrap now, as we have more fine grained strata.13:59
persiaThis would reduce the issues in the future.14:00
Kinnisonthat would be nice14:00
KinnisonBut don't build systems include BSPs?14:00
paulsherwoodcan we make cross-bootstrap part of build-system?14:00
KinnisonIIRC we don't do BSP building in cross-bootstrap because it's very awkward14:00
persiaHow do we expect the cross-bootstrap environment to boot without a target-appropriate BSP?14:01
paulsherwoodinitially by taking a provided bsp from elsewher,e i think14:01
* persia reads the docs again before complaining at more length14:01
persiaOh Ugh.14:07
persiaWe should be able to generate a cross-built BSP for the target, and deploy that system.14:08
persiaThe current model is such that Baserock cannot be used for hardware bringup.14:08
richard_mawIt can provide the userland for bringup, it just doesn't do the whole job14:09
richard_mawassuming we share an understanding of what hardware bringup means, the result of cross-bootstrap is a tarball that can be unpacked onto storage of the new machine, and set init= in the kernel command-line14:11
KinnisonOkay, so without getting into the complexities of building BSPs using bootstrap compilers, what would we prefer the shape of things to be?14:11
persiarichard_maw: Interesting.  Perhaps we should recommend that method in the docs?14:12
persiaKinnison: My preferred shape would be able to cross-bootstrap a system that could be deployed to a target, for remaining bootstrap.14:13
KinnisonSince cross-bootstrapping is often done very early in the project's interaction with a given architecture, that's not really something I'd aim for directly14:13
persiaSo the user runs some command to build stuff, then another command to deploy stuff, then possibly some extra fiddling to get the deployment in the right place, then boots the target, and it completes the bootstrap.14:13
KinnisonAlthough I can see how it'd be useful later for verifying bootstrap14:13
persiaKinnison: My assumption is that I would not have working userspace for a target *except* via cross-bootstrap.14:14
KinnisonThat's rarely the case for stuff we're working on, but it might happen14:14
Kinnisonesp. if a HW vendor takes us on14:15
persiaI have several bits of hardware for which I don't have shell access to userspace, but have source for kernels.14:15
persiaI've been using OpenWRT to TFTP-boot them into something from which I could install something more interesting.14:16
persiaBut I'd rather not have to use that sort of method.14:16
KinnisonAre those bits of hardware things you'd be using to complete Baserock bootstraps?!14:16
KinnisonRemember, we're talking about bootstrapping support for a CPU architecture, not a given target unit14:16
persiaAt least one of the devices that I've used OpenWRT to access is an architecture not currently supported by definitions master.14:17
persiaI can't authoritatively say what I might encounter in the future.14:17
* Kinnison believes you're aiming for a nebulous idea of what you might encounter rather than something more concrete for architecture bootstrap14:17
KinnisonI won't complain if you make it so we can do as you suggested, but I don't personally see the benefit in the effort14:18
persiaI don't know how to so make it, but you asked what shape I wanted :)14:20
KinnisonCertainly I agree that the current shape is wasteful and causing issues14:20
persiaI've definitely tried to use Baserock on a few architectures that were unsupported, but not yet managed to cause cross-bootstrap to do what I wanted.14:20
persiaIt may be just that I'm playing with the wrong kit :)14:23
KinnisonI wouldn't say that :-)14:23
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]14:41
jmacsOops, we're lorrying delta/libyajl2-gem and delta/ruby-gems/libyajl2-gem now...15:17
*** CTtpollard [] has quit [Ping timeout: 272 seconds]15:26
KinnisonOdd sequence of log:15:34
Kinnison2015-01-16 15:33:58 [Build 3/173] [stage2-linux-api-headers] Not updating git repository upstream:linux because it already contains sha1 df2e1b9168a7ab5dd8149e38b5ac70cdef86d1fa15:34
Kinnison2015-01-16 15:33:58 # git remote update origin --prune15:34
Kinnisonanyone know why that might happen?15:34
richard_mawI can't think of a sensible reason for it to do that15:35
KinnisonCan you think of any non-sensible reason?15:36
richard_mawaccidental inversion of the test15:36
straycatLooking at SotK's series, I'm a little unsure of the need for all the caching we currently have, we check the local checkout, then the remote, finally the remote. Given all the morphs are in defs is not enough to just check the local checkout? I guess distbuild might stop us from doing that?15:39
straycat*then the local cache, finally the remote15:39
persiastraycat: Does morph always check, or only on a cache-miss?15:39
straycatonly on miss15:40
persiaI suspect that on a cache-miss we ought update the local cache, rather than go checking remotely.15:40
*** CTtpollard [] has joined #baserock15:40
straycatpersia, huh? if the local checkout doesn't contain the morph we want, we look in the local cache, if the local cache doesn't contain the morph we want, we look in the remote cache.15:41
persiaThat doesn't make any sense to me.15:41
straycatpersia, which part?15:42
persiaFor morphologies, if the local checkout doesn't contain it, then I think we want to error out.15:42
SotKpersia: why?15:42
persiaSotK: Because if I'm building from the working area (something I don't like, but believe to be current morph behaviour), and I delete a file, I don't want it randomly showing up because it existed in some cache.15:42
persiaThinking about that more, is this to support the old behaviour of morphologies-in-source-repos?15:44
SotKpretty much, yes15:44
straycatwhich is why it's looking overly complex to me right now15:44
persiaBut everything I've said should be ignored.15:45
persiaBecause it doesn't apply.15:46
persiaActually, it would be nice to add a "version" file to definitions.  If present, and containing the single character "1", the legacy behaviours can be skipped.15:46
* SotK wants ssssam's temporary build branch patch merged15:47
persiaIf not present, they should be retained.  If present with a different value, morph should complain that it does not understand how to parse that version of definitions.15:47
*** zoli_ [] has joined #baserock15:49
*** zoli_ [] has quit [Changing host]15:49
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock15:49
straycatSotK, can look at that next?15:49
SotKstraycat: thanks :)15:49
* paulsherwood would love to shred this logic, but cannot participate now15:55
SotKstraycat: thanks for the review!16:31
jmacsI apologise in advance for the Chef/Ceph patch that's about to hit the mailing list but I've been rebasing it for a week and no longer know who I am or what words are.16:33
Zara_drat, I was hoping that rebasing would stop having that effect on me, once I had some years of experience16:37
straycatSotK, :]16:38
* paulsherwood doesn't rebase. he does git reset --hard, and git cherry-pick16:38
paulsherwoodbut didn't JPohlmann once talk about how easy and exciting interactive rebase is16:39
richard_mawhe did16:39
* paulsherwood wishes he'd seen that talk16:39
straycatit makes us all feel happy inside16:39
De|ta"Don't fear the rebase"16:40
paulsherwoodthat was the one16:40
* paulsherwood can hear Blue Öyster Cult already16:40
* robtaylor too16:40
* SotK three16:41
Kinnisonall I can hear is the hDD in the server behind me 16:41
Kinnisonand some people who type loudly16:42
jmacsI seem to have accumulated 55G of /src/cache/artifacts... is there any way to determine what's what in here so I can sensibly remove some?16:43
straycatjmacs, might want to check out morph gc16:44
paulsherwoodstraycat: it won't do anything unless he's short of space, iiuc16:44
straycatoh yeah you have to set --cachedir-min-space to something high for it to do that >.>16:45
paulsherwoodeven then, there's no reason gc will be particularly intelligent in its choices, i think?16:45
* paulsherwood may be mistaken16:46
jmacsOh, morph gc does clear them.16:46
straycatit removes everything older than some set constant16:46
jmacsNo idea what it's removed, but it's done now16:46
jmacsOK, that sounds fine.16:46
richard_mawthe rules are 1. everything older than configured value, 2. least recently used that's older than a configured value16:47
* persia likes rebase, and used to do it for several parallel patch series multiple times daily16:48
* franred likes rebase too, it is pretty useful and good for clean the result of using the "commit often" commandment 16:50
Zara_whoa, just saw jmac's patch @_@16:50
* Krin is slowly working through that patch16:51
jmacsAlready spotted a reference to github in there :(16:51
paulsherwoodjmacs: that could be fixed on merge ;)16:55
jmacsAs I said, it needs testing anyway, and I'm sure people will find bigger problems than that16:57
paulsherwoodjmacs: how much of a box does it need for testing? could i build it on my machine, for example?16:58
* Krin holds up his hands after getting to the 4th of 19 patches "your doing stuff that i dont know about, i cant really review it when i am unsure on the result of commands your using jmacs" 16:58
jmacspaulsherwood: You can't do anything useful with it as it needs some upstream repositories changing to ones on my github account17:00
jmacsYou could attempt to build it, but you'll need a baserock machine with at least 8GB of RAM17:00
*** CTtpollard [] has quit [Quit: Ex-Chat]17:02
paulsherwoodah, my physical machine doesn't have that much17:20
*** jonathanmaw [] has quit [Quit: Leaving]17:34
*** Krin [] has quit [Remote host closed the connection]17:40
*** wdutch [] has quit [Quit: Quit]17:52
*** franred [] has quit [Remote host closed the connection]18:14
*** rdale [] has quit [Ping timeout: 240 seconds]18:18
*** grahamfinney_ [] has joined #baserock18:22
*** tiagogomes_ [] has quit [Quit: Leaving]18:29
*** grahamfinney_ [] has quit [Quit: Ex-Chat]18:30
*** lachlanmackenzie [] has quit [Ping timeout: 245 seconds]19:02
*** grahamfinney [] has joined #baserock19:07
*** grahamfinney [] has quit [Client Quit]19:08
*** grahamfinney [] has joined #baserock19:12
*** grahamfinney_ [] has joined #baserock19:19
*** grahamfinney [] has quit [Read error: Connection reset by peer]19:19
*** grahamfinney [] has joined #baserock19:23
*** grahamfinney [] has quit [Client Quit]19:23
*** grahamfinney_ [] has quit [Ping timeout: 276 seconds]19:25
*** genii [~quassel@ubuntu/member/genii] has joined #baserock20:44

Generated by 2.14.0 by Marius Gedminas - find it at!