| *** gtristan has quit IRC | 04:45 | |
| *** gtristan has joined #baserock | 05:07 | |
| *** gtristan has quit IRC | 08:31 | |
| *** ctbruce has joined #baserock | 08:32 | |
| *** tiagogomes has joined #baserock | 08:44 | |
| *** mariaderidder has joined #baserock | 09:05 | |
| *** bashrc_ has joined #baserock | 09:06 | |
| *** toscalix__ has joined #baserock | 09:14 | |
| *** gtristan has joined #baserock | 09:27 | |
| *** edcragg has joined #baserock | 09:38 | |
| *** ssam2 has joined #baserock | 09:49 | |
| *** ChanServ sets mode: +v ssam2 | 09:49 | |
| tiagogomes | gtristan, thanks for clarifying me on your last email | 09:49 |
|---|---|---|
| *** nowster has joined #baserock | 09:54 | |
| *** toscalix__ has quit IRC | 09:54 | |
| gtristan | tiagogomes, no problem :) | 09:58 |
| gtristan | ssam2, says that I do not have to go into such level of detail in a proposal, but in my experience you do - otherwise people invariably have misunderstandings :) | 09:59 |
| * gtristan waits for jjardon to arrive... | 10:00 | |
| *** mariaderidder has quit IRC | 10:09 | |
| *** dabukalam has quit IRC | 10:12 | |
| *** pedroalvarez has quit IRC | 10:12 | |
| *** pedroalvarez has joined #baserock | 10:12 | |
| *** ChanServ sets mode: +v pedroalvarez | 10:12 | |
| *** dabukalam has joined #baserock | 10:13 | |
| *** toscalix__ has joined #baserock | 10:16 | |
| * gtristan hopes he kept his pre-systemd-upgrade artifacts in tact... | 10:19 | |
| gtristan | hmmm, this does not make sense | 10:21 |
| gtristan | [1/17/413] [samba] ... | 10:21 |
| gtristan | the gnome stratum depends on the samba stratum... and the gnome stratum contains *way* more than 17 chunks | 10:21 |
| paulsherwood | gtristan: i think ybd's counter is misbehaving | 10:21 |
| * gtristan does not understand | 10:21 | |
| paulsherwood | are you using last tag, or later? | 10:22 |
| gtristan | damn, and I was hoping that I could recover | 10:22 |
| gtristan | last tag | 10:22 |
| gtristan | last time when you told me there was one | 10:22 |
| gtristan | lets see what it does... I have a lingering hope that I can test pre-systemd-upgrade without rebuilding webkit | 10:23 |
| paulsherwood | the problem i'm aware of is after last tag | 10:23 |
| gtristan | Ah ! | 10:23 |
| paulsherwood | (in fact it's introduced by the last commit) | 10:23 |
| gtristan | I know | 10:23 |
| gtristan | yay | 10:23 |
| gtristan | ok its going to work | 10:23 |
| gtristan | this is nice :) | 10:23 |
| paulsherwood | :-) | 10:23 |
| paulsherwood | what's the explanation? | 10:24 |
| gtristan | I removed the samba artifact | 10:24 |
| gtristan | but I have all the others | 10:24 |
| gtristan | ybd found that sambas cache key needs to be rebuilt for this config | 10:24 |
| gtristan | but, all the other cache keys are present, so depending chunks wont need a rebuild :) | 10:24 |
| paulsherwood | :) | 10:24 |
| gtristan | it will take about 30min I guess to verify this... | 10:25 |
| * gtristan prepares to pounce on jjardon | 10:25 | |
| *** mariaderidder has joined #baserock | 10:37 | |
| rjek | gtristan: Get ready to pounce. | 11:09 |
| *** jjardon has joined #baserock | 11:14 | |
| tiagogomes | now! | 11:15 |
| gtristan | ahem | 11:15 |
| gtristan | jjardon | 11:15 |
| gtristan | either one of two things happened | 11:15 |
| gtristan | One of them, is that you have a different setup, VM, machine than I do, and that everything works fine for you (I doubt this, but, benefit of the doubt and all) | 11:16 |
| gtristan | The other possibility, is that you thought it was acceptable to push a systemd upgrade to master without verifying that we still reach a graphical boot | 11:17 |
| gtristan | For those who enjoy upgrading low level components, let them also enjoy verifying dependent system integrity, otherwise, please - lets keep those upgrades outside of definitions master. | 11:19 |
| gtristan | I have lost several hours without even identifying with certainty that that is the actual faulty commit, actually | 11:20 |
| ssam2 | in fairness, it was actually me who pushed it, and I admit to not doing any amount of testing. i'm happy to stop +2'ing definitions patches if it's causing problems | 11:20 |
| ssam2 | I mostly do it due to the paucity of reviewers | 11:20 |
| gtristan | It can be anything after b18ff44 (Adding gnome-control-center) and before and up until cca2b07 (systemd upgrade) | 11:20 |
| ssam2 | but I did think that 'Tested this with a x86_64 weston system' meant that it was tested :-) | 11:21 |
| gtristan | unfortunately, with baserock, performing a bisection, manually or otherwise - takes at least a day of lag in rebuilds | 11:23 |
| paulsherwood | gtristan: it wouldn't if you were using a fast AWS machine | 11:23 |
| gtristan | the point is that, when comparing this project to a project like, say, GTK+ or an application, ensuring that your commits dont break something before pushing upstream is even more critical | 11:25 |
| gtristan | the cost of picking up the pieces and identifying exactly what broke is more | 11:25 |
| ssam2 | this problem was solved a while ago for Morph uses with https://mason-x86-64.baserock.org/ | 11:26 |
| ssam2 | although it's really rubbish, it does autobuild stuff as soon as it gets committed | 11:26 |
| jjardon | gtristan: whats the problem | 11:26 |
| ssam2 | would something similar for ybd reduce the pain? | 11:27 |
| ssam2 | (i don't have time to do that at present, just wondering) | 11:27 |
| ssam2 | (but should be easy for anyone else to set up) | 11:27 |
| gtristan | I have verified that master is broken, and I have verified that my pending patch set works when staged against my pre-systemd-update branch | 11:27 |
| gtristan | jjardon, boot up master, black screen; no gnome-initial-setup, automatically enabled gdm = bricked | 11:28 |
| gtristan | ssam2, it is not sufficient to know that "things built" | 11:28 |
| jjardon | gtristan: what is broken exactly? ah, the gnome system | 11:28 |
| pedroalvarez | gtristan: that is true | 11:29 |
| ssam2 | gtristan: but it means that bisect is pretty quick because you can download the prebuilt artifacts | 11:29 |
| ssam2 | gtristan: i agree much more automated testing is needed, but seems there's no resources/willing people to make that happen at present | 11:29 |
| gtristan | ssam2, what is needed is a stricter policy when making changes to low level strata | 11:30 |
| jjardon | gtristan: I only tested the weston one, Id say revert the commit with a note | 11:31 |
| gtristan | I dont know how you would automatically test that a change in the init system, which *probably* requires system integration changes, would actually result in a working PAM setup and login | 11:31 |
| ssam2 | didn't gnome-continous used to do that? | 11:33 |
| ssam2 | seems to have stopped doing that these days, but it used to show a screenshot of logged-in desktop for each build | 11:33 |
| gtristan | ssam2, I'm not sure that it does really, it runs installed tests, which is a step up from stuff we dont have | 11:34 |
| gtristan | it apparently takes screenshots, but that does not mean that it accounts for a properly integrated gdm | 11:34 |
| ssam2 | ok. I figured that if you get to graphical desktop them GDM must be working | 11:34 |
| gtristan | ssam2, it may very well bypass that by running gnome-session without gdm, but perhaps, I'm not sure | 11:34 |
| gtristan | the thing to be clear about, is that GNOME assumes the system works - below the GNOME stack is "not GNOME's problem" | 11:35 |
| *** brlogger` has joined #baserock | 11:38 | |
| *** brlogger has quit IRC | 11:38 | |
| gtristan | such CI would have to be done in a separate repo | 11:38 |
| jjardon | gtristan: https://git.gnome.org/browse/gnome-continuous/tree/manifest.json#n102 | 11:38 |
| jjardon | oh, sam was faster :) | 11:38 |
| radiofree | does anyone know how they automate testing the wayland backend? it's in qemu right? | 11:40 |
| jjardon | gtristan: if something is broken you can tag the component until it get fixed | 11:40 |
| gtristan | Anyway, at this point we have a choice; offload this problem to me, so that I lose time focusing on my actual mandate, or verify that that is infact the broken commit and revert it until it can be introduced without breaking things | 11:40 |
| jjardon | gtristan: sure, revert the systemd upgrade commit if that breaks the gnome system | 11:41 |
| gtristan | I can try master with the reverted systemd, it will take the rest of the day to validate that it is not something else in that handful of 10 commits (at-spi changes ? doubtful they can brick a system) | 11:41 |
| ssam2 | as a cheap hack to avoid the full rebuild, you can add a new systemd stratum at the top of the dependency graph with old systemd | 11:44 |
| ssam2 | the old version will overwrite the new version in the resulting system | 11:44 |
| gtristan | looks like it doesnt have system integration hooks | 11:47 |
| gtristan | so maybe I would have enough faith that that would be correct :-/ | 11:48 |
| gtristan | There is another alternative, which is that I push my most recent patch set, which I have verified against an older master (based on: "b18ff44 Adding gnome-control-center to GNOME strata") | 11:52 |
| gtristan | And not claim responsibility for doing the detective work and verifying that the systemd upgrade is the one in b18ff44...cca2b07 which causes the issue | 11:53 |
| jjardon | gtristan: push your branch, but that commit is more than a week old ... | 12:01 |
| gtristan | I'm rebuilding, do push my branch would be to perpetuate the idea that it's ok to push stuff without validating first | 12:12 |
| jjardon | gtristan: I do not think anyone is pushing stuff without validating first | 12:13 |
| gtristan | jjardon, fwiw, I understand its a mistake - it's just of paramount importance that it be clear | 12:13 |
| gtristan | that was not my impression, to be honest | 12:14 |
| gtristan | I am unsure if any of my commits have compromised stability of adjacent systems, this worries me in general | 12:14 |
| jjardon | gtristan: but the amount of validation a individual can do is limited; for exameple I can not build every single system in definitions to check a change is not regressing in any of them | 12:14 |
| gtristan | jjardon, however you should, and I should | 12:15 |
| gtristan | or else I should not make any change to a low level stratum | 12:15 |
| *** mariaderidder has quit IRC | 12:15 | |
| gtristan | that is the responsibility that comes with sharing low level components | 12:15 |
| gtristan | otherwise we send the message that we just dont really care about the general stability of the systems we maintain | 12:16 |
| jjardon | gtristan: agree, we should, but thats not really a realistic thing to demand to a contributor: I do not even have the hardware to test some of the systems. thats why we need propper ci to test this for us | 12:20 |
| gtristan | at least we need a staging area | 12:21 |
| gtristan | the cost of validating every outcome, and being careful to do so when modifying low level components in general, is lower than picking up the pieces | 12:21 |
| gtristan | when things break | 12:22 |
| gtristan | also, if there are system morphs in the systems/ directory which do not have a dedicated maintainer, somebody responsible and building those, they should probably be dropped | 12:22 |
| gtristan | it makes no sense to publish a system which we know "builds" for so long, but nobody has a clue if it actually works | 12:23 |
| radiofree | that makes sense | 12:23 |
| gtristan | that reflects worse on the project than being so proud of having many systems | 12:23 |
| jjardon | yeah, I suggested that several times; we dont even know if they builds | 12:23 |
| jjardon | gtristan: actually, I suggest that we only offer systems that are in the ci; there is no other way to know that they at least compile | 12:24 |
| gtristan | I would wish we had some separation for those | 12:25 |
| gtristan | sending a clear message of which systems we vouch for their stability and are maintained and receive regular attention, vs. which systems have merely passed some automated CI | 12:25 |
| jjardon | gtristan: I agree, but the CI is better than nothing. Also I guess you have to define what "maintained" and "regular attention" means; Is there a person that occasionally build the system and checks it works? is there a machine that runs some funtional test in the generated system? ... | 12:34 |
| *** ssam2 has quit IRC | 12:35 | |
| gtristan | jjardon, first, stable verified releases | 12:36 |
| gtristan | stable branches | 12:36 |
| *** ssam2 has joined #baserock | 12:36 | |
| *** ChanServ sets mode: +v ssam2 | 12:36 | |
| gtristan | jjardon, if we first sent a message that we do maintain stable branches, and that we take stability of specific systems very seriously, we send a message that we are a venue for contributing patches back upstream | 12:38 |
| gtristan | before we demonstrate that responsibility and at least organization, our baselines will not receive contributions from any user base | 12:38 |
| pedroalvarez | I want to point out that we don't have CI | 12:39 |
| gtristan | we need to demonstrate that if a security bug is found in a downstream IVI system, that contributing the patch upstream will be kept in a stable branch somewhere | 12:39 |
| gtristan | performing all of the manual validation ourselves is impossible, we need to collect bugs from the field and make sure we always improve stability | 12:40 |
| jjardon | pedroalvarez: mason? | 12:41 |
| gtristan | CI is another nice-to-have feature, to be sure - but being well organized and proving that we have a stable vs experimental branching scheme is, IMO a higher priority and brings more long term value | 12:41 |
| gtristan | we need to coordinate between "systems" maintainers to make a choice of what version of systemd and other low level components we agree upon for the "next stable branch" (current experimental) | 12:42 |
| gtristan | and *not* upgrade things dramatically in stable, other than collected security fixes and other important patches, pull strictly from agreed upon upstream stable branches | 12:43 |
| jjardon | gtristan: we branch the entire definitions or every system has its own branch? | 12:43 |
| persia | jjardon: mason is not CI | 12:43 |
| persia | (or at least not mason v2, which is what is deployed) | 12:43 |
| gtristan | jjardon, I think A.) We reduce definitions to 3 or 4 maintained systems B.) We branch definitions as a whole | 12:44 |
| *** mariaderidder has joined #baserock | 12:44 | |
| persia | gtristan: It depends on the goals. Baserock has moved back and forth between two goals over time: that of being a sensible curated software collection and that of being a set of tools to build software collections. | 12:44 |
| gtristan | jjardon, maybe we can have something like gstreamer (i.e. gstreamer-plugins-bad) for a separate repo of not-really-maintained systems | 12:45 |
| persia | Most of the folk using Baserock today ignore the reference systems entirely, and are interested in the tools: more because what they are doing has nothing to do with the reference systems than because of any policies surrounding the reference systems. | 12:45 |
| jjardon | mmm, so what happen with external contributors that want to add a new systems ? | 12:45 |
| gtristan | persia, it cannot really be successful at the latter without being successful as the former | 12:45 |
| gtristan | afaics | 12:45 |
| persia | gtristan: If you like: as I said, none of the current users of the tools use the reference systems in any way, or have expressed any interest in doing so. | 12:46 |
| pedroalvarez | sorry to say this, but Mason is not a real CI | 12:46 |
| persia | But that doesn't mean there aren't folk who care about that sort of thing. | 12:46 |
| gtristan | jjardon, they have to demonstrate their capability of maintaining that system over time within the baserock community | 12:46 |
| persia | Why "external contributors"? The current model seems to work without any "internal/external" requirements. Let it stay that way. | 12:47 |
| gtristan | I think the distinction is more "new contributors" | 12:47 |
| jjardon | persia: pedroalvarez what is mason then? and what its missing to not be a real CI? | 12:47 |
| gtristan | The point is we cannot, as a project, advertise that we support a system as stable, if nobody is willing to step up and do maintenance of that system | 12:48 |
| persia | jjardon: Mason v2 performs some limited build validation post-merge, but doesn't have facilities for extended testing, nor for any sort of pre-merge validation. | 12:48 |
| persia | mason v3 moves stuff into pre-merge, and has some framework for wider testing, but I never heard of it being completed. | 12:48 |
| pedroalvarez | mason(v1, the one currently in use) is a devel system that is pulling every minute, and if the sha1 changes it builds and deploys ci.morph | 12:48 |
| persia | pedroalvarez: v1 is the jenkins one: current is v2 | 12:49 |
| pedroalvarez | sorry looks like the one in production is v2 | 12:49 |
| pedroalvarez | yes | 12:49 |
| jjardon | persia: is pre-merging a requirement for a CI? I wouldn't think so | 12:50 |
| persia | jjardon: pre-merge is a requirement to not land failed builds in master, but no, not a requirement for CI. Being able to take action on commit is a requirement for CI, rather than a scheduled activity. | 12:51 |
| persia | mason v2 does neither. | 12:52 |
| jjardon | persia: you can say that in our case that action is generate a red or green line in a webpage | 12:53 |
| persia | jjardon: Except that the action isn't based on a commit happening. | 12:54 |
| persia | mason v2 polls the repo occasionally, and runs a build if it sees a difference, but whether that is one commit or a million commits is unknown to Mason v2. | 12:54 |
| persia | So it isn't possible to know which commit caused the red line except by coincidence. | 12:54 |
| jjardon | ah, I get you know | 12:55 |
| persia | Mason v2 is a reasonable system to do snapshot testing: I'd like more comprehensive tests, but it works. I don't mean to suggest it doesn't have value, only to support the argument that it fails to be "continuous". | 12:56 |
| jjardon | but then again; is a requirement for a CI to build changes/ present result for _every_ commit; the definitions in wikipedia says "several times a day", it doesnt specify how often | 12:56 |
| persia | It depends on one's semantics. | 12:57 |
| jjardon | persia: sure, im only curious why some people say "is not CI" | 12:57 |
| persia | For my personal semantics, anything post-merge isn't "CI", but only "CD". | 12:57 |
| persia | I think most people currently have semantics that limit "CI" to every-commit systems. | 12:57 |
| persia | (although there is debate on the pre-merge/post-merge side of things) | 12:57 |
| ssam2 | mason v2 also isn't integrating anything, so at best it's "continuous build" | 12:58 |
| persia | ssam2: To be fair, Travis also doesn't "integrate", and folk call that "CI". | 12:59 |
| persia | I think "Integrate" in "CI" has come to mean "build + test" | 12:59 |
| pedroalvarez | "continuous build but I'm not going to build every commit, and it's going to take some days to give you any feedback" | 12:59 |
| pedroalvarez | I've been trying to put the baserock.org ssl certificate in g.b.o | 13:00 |
| pedroalvarez | but looks like cloning from https still doesn't work | 13:00 |
| persia | :( | 13:01 |
| jjardon | I think there si a patch from straycat in the mailing list to fix that | 13:01 |
| pedroalvarez | jjardon: that's another issue, and the patch is to fix the ca-certificates chunk | 13:03 |
| pedroalvarez | which is broken since we moved to python3 | 13:03 |
| pedroalvarez | I can't clone using https even from debian | 13:03 |
| *** toscalix_ has joined #baserock | 13:05 | |
| pedroalvarez | I don't know exactly how this SSL certs work. Are we expecting the user to install it manually in their systems? | 13:06 |
| pedroalvarez | should we install the baserock.org SSL cert in baserock systems by default? | 13:06 |
| pedroalvarez | or maybe it should just work without any manual steps | 13:07 |
| pedroalvarez | ? | 13:07 |
| *** toscalix__ has quit IRC | 13:08 | |
| tiagogomes | I don't know how it works, but you don't need to install anything on the systems where you are running `git clone` | 13:09 |
| pedroalvarez | tiagogomes: I guess that depends on how trusted is the certificate and how well it has been distributed | 13:11 |
| pedroalvarez | but I can be wrong, as I said "I don't know exactly how this SSL certs work." | 13:11 |
| persia | Which CA asserts the b.o cert is trustworthy? | 13:18 |
| persia | If it is not one pre-delegated as a trust authority by users, it is likely to raise an alert. | 13:19 |
| persia | The set of currently trusted certs from Mozilla.org is a reasonable proxy for the certs trusted by most users. | 13:20 |
| persia | err, CAs delegated | 13:20 |
| jjardon | gtristan: I found the commit that broke the session: https://gerrit.baserock.org/#/c/1390/ | 13:23 |
| pedroalvarez | I'm failing to understand and answer your question | 13:23 |
| persia | pedroalvarez: Is the public part of the cert somewhere I can inspect? | 13:24 |
| pedroalvarez | yes | 13:24 |
| persia | Essentially a certificate is signed by a certificate authority, to indicate that that certificate authority believes that certificate to be genuine. | 13:24 |
| pedroalvarez | http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/infrastructure.git/tree/certs/baserock.org-ssl-certificate-temporary-dsilverstone.full.cert | 13:24 |
| SotK | pedroalvarez: I think the question is "which CA signed the b.o cert", but I could be totally mistaken as I also only have a minimal understanding | 13:24 |
| jjardon | Can anyone take a look to gerrit to lorry some pending GNOME components, please? | 13:24 |
| * gtristan 's internet is crapping out here... loading url | 13:24 | |
| gtristan | jjardon, not systemd ? | 13:24 |
| persia | And most computers have a common set of certificate authorities to whom the delegate the decision as to whether a certificate is valid. | 13:25 |
| jjardon | gtristan: nope | 13:25 |
| pedroalvarez | maybe: StartCom Class 2 Primary Intermediate Server CA ? | 13:25 |
| persia | So, if a certificate is signed by one of the delegated authorities, then that certificate is considered valid. If not, the user is usually prompted. | 13:25 |
| pedroalvarez | and these certificates are usually more expensive, right? | 13:26 |
| persia | Some certificate authorities will accept money as proof of authenticity. Others have administrative fees as part of the approval process. Others have no-fee processes of variable reliability. | 13:27 |
| ssam2 | the one we have for baserock.org web services is trusted by Firefox out-of-the-box | 13:27 |
| gtristan | jjardon, the revert happens before the commit it reverts ? | 13:27 |
| ssam2 | our ca-certificates chunk comes from Debian | 13:28 |
| ssam2 | so if Debian trusts the baserock.org certificate out-of-the-box, no problem | 13:28 |
| ssam2 | since kinnison provided the cert and is a Debian user, I imagine he'd spot if the CA provider he used wasn't trusted by Debian. | 13:28 |
| persia | pedroalvarez: Indeed, startcom is the relevant CA here. | 13:28 |
| gtristan | so a new revert on top of the commit after it's revert, I see | 13:29 |
| gtristan | jjardon, alright I'll remove my systemd revert from my tree and restart my build | 13:29 |
| gtristan | it should complete over night | 13:29 |
| * gtristan rebases against master | 13:29 | |
| pedroalvarez | ssam2: I guess the problem is somewhere else, given that I can `curl https://mason-x86-64.baserock.org/` | 13:29 |
| pedroalvarez | (from debian) | 13:30 |
| pedroalvarez | and from baserock | 13:30 |
| persia | pedroalvarez: When you `git clone ...` do you get "fatal: unable to access 'https://git.baserock.org/git/baserock/baserock/infrastructure.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none" ? | 13:31 |
| pedroalvarez | and I can't do the same with git.baserock.org | 13:31 |
| pedroalvarez | persia: yes | 13:31 |
| gtristan | ah, total clusterfsck, commit, revert, commit again... now revert again | 13:32 |
| * gtristan wonders how that made it into master | 13:32 | |
| persia | pedroalvarez: What do you get if you run `openssl verify ${KEY}` | 13:33 |
| persia | I get failure locally, and I get the same error as you trying to run `git clone ...` | 13:34 |
| pedroalvarez | persia: http://paste.baserock.org/kedudifubo | 13:35 |
| persia | Same for me. Frustratingly, I seem to have StartCom_Certification_Authority_2.crt enabled in my ca-certificates configuration. | 13:36 |
| persia | For the record, the other incredibly useful openssl command is `openssl x509 -in ${CERT} -text -noout`, which prints the details of the certificate. | 13:37 |
| persia | My suggestion is to ask Kinnison for guidance on whether the certificate is one he can verify on his setup. | 13:38 |
| pedroalvarez | yeah, sounds sensible | 13:38 |
| persia | If not, then perhaps something went wonky in the request process. If so, then it becomes interesting to explore what differences exist between there and yours/mine. | 13:38 |
| *** gtristan has quit IRC | 13:39 | |
| pedroalvarez | persia: my guess is that I haven't installed it properly, given that: http://paste.baserock.org/uhigozebej | 13:43 |
| persia | pedroalvarez: That is the same error message as http://paste.baserock.org/kedudifubo | 13:44 |
| straycat | 13:03 +pedroalvarez$ jjardon: that's another issue, and the patch is to fix the ca-certificates chunk | 13:44 |
| persia | The problem is with the trust of the certificate, not with how the server is configured. | 13:45 |
| straycat | the https clone issue is fixed by the patch i sent to the list a couple of days ago | 13:45 |
| persia | The fact that you get the same error is an indication that you installed it correctly. | 13:45 |
| pedroalvarez | straycat: yes, this is not to fix curl in baserock, is to fix https in g.b.o | 13:45 |
| persia | straycat: slightly different: your patch fixes running git or curl on baserock hosts. The issue we're debugging is about using git or curl on non-Baserock hosts to access g.b.o | 13:46 |
| straycat | okay i didn't actually read the backscroll | 13:46 |
| pedroalvarez | persia: but, I believe that the same cert is installed in mason, and it doesn't fail | 13:46 |
| straycat | since no one replied to my patch yet i figured people had forgotten | 13:46 |
| persia | pedroalvarez: I don't believe it is the same cert | 13:46 |
| pedroalvarez | hm.. | 13:47 |
| pedroalvarez | straycat: sorry, I didn't feel ready to review the patch :/ | 13:47 |
| straycat | that's fine i thought about it today and decided that my past self wouldn't have been able to review it easily either :) | 13:48 |
| pedroalvarez | I should review it just for learning | 13:49 |
| pedroalvarez | persia: hm.. it should be, I've done something wrong. I'll investigate | 13:50 |
| * SotK notices that the lighttpd config in trove-setup points to /etc/lighttpd/certs/lighttpd.pem | 13:50 | |
| SotK | which appears to be created here: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/trove-setup.git/tree/ansible/roles/trove-setup/tasks/lighttpd.yml#n6 | 13:50 |
| pedroalvarez | and what i've done so far is, replaced that .pem file, and restarted lighttpd-git | 13:52 |
| *** gtristan has joined #baserock | 13:55 | |
| *** toscalix_ has quit IRC | 14:02 | |
| edcragg | pedroalvarez: i hear you have some ideas on how an ARM artifact cache has been done before, or could be done in the future? | 14:18 |
| pedroalvarez | yes | 14:18 |
| pedroalvarez | currently we have arm artifacts in cache.baserock.org | 14:18 |
| pedroalvarez | it is a non-public mason, running on a jetson on my desk, but pushing artifacts every build | 14:19 |
| pedroalvarez | before we had jetsons in DataCentred | 14:19 |
| paulsherwood | artifacts1.baserock.org:8000 is serving arm artifacts for current definitions | 14:19 |
| paulsherwood | (ybd only) | 14:19 |
| radiofree | edcragg: use a mustang? | 14:20 |
| radiofree | hint hint | 14:20 |
| pedroalvarez | moonshot? | 14:20 |
| paulsherwood | i'm considering a cron job to trigger git pulls and run ybd automatically to keep it populated | 14:20 |
| radiofree | do we have access to them anymore? | 14:20 |
| paulsherwood | radiofree: yes | 14:20 |
| pedroalvarez | if you manage to publish the logs, and a status page, then will have reinvented mason | 14:21 |
| radiofree | it's the same anyway, you just want to build armv7lhf on aarch64, there's a guide on the mailing list | 14:21 |
| radiofree | do that and push the cache somewhere | 14:21 |
| pedroalvarez | that was one of the ideas I was going to suggest | 14:21 |
| radiofree | using a cron job as paulsherwood suggested would suffice | 14:21 |
| paulsherwood | pedroalvarez: yes, but faster :) | 14:21 |
| radiofree | although i think the preferred way is a systemd timer now | 14:21 |
| edcragg | this sounds promising | 14:22 |
| pedroalvarez | for now it only sounds confusing | 14:22 |
| paulsherwood | edcragg: if you're just seeking existing artifacts, and are willing to use ybd, all of arm7lhf ci.morph should already be at http://artifacts1.baserock.org:8000/* | 14:22 |
| radiofree | i don't think it's confusing | 14:22 |
| *** gtristan has quit IRC | 14:23 | |
| pedroalvarez | radiofree: I see no point on using a cron job, that will only publish the artifacts on the same way that I'm doing now | 14:23 |
| pedroalvarez | maybe faster, but hey | 14:23 |
| radiofree | apparent from requiring a pedroalvarez to do it manually | 14:23 |
| pedroalvarez | and maybe not in my desk | 14:23 |
| radiofree | s/apparent/apart | 14:24 |
| *** gtristan has joined #baserock | 14:24 | |
| pedroalvarez | I'm not doing it manually | 14:24 |
| radiofree | oh right, so we're already supplying arm caches? | 14:24 |
| pedroalvarez | yes | 14:24 |
| pedroalvarez | as I said is a non-public mason | 14:24 |
| pedroalvarez | working on the same way as the others | 14:25 |
| radiofree | ah cool, why not make it public then? | 14:25 |
| radiofree | is that building on a moonshot? | 14:25 |
| pedroalvarez | <pedroalvarez> it is a non-public mason, running on a jetson on my desk, but pushing artifacts every build | 14:25 |
| radiofree | ah ok, it would be ideal to use a moonshot/mustang rather than a single jetson | 14:26 |
| pedroalvarez | <pedroalvarez> that was one of the ideas I was going to suggest | 14:26 |
| paulsherwood | is there an echo in here? :-) | 14:26 |
| pedroalvarez | <pedroalvarez> yes | 14:27 |
| paulsherwood | lol | 14:27 |
| pedroalvarez | just kidding in the last one | 14:27 |
| pedroalvarez | but the others were answers to similar questions | 14:27 |
| pedroalvarez | Ok, edcragg and I are trying to evaluate what would be good for the project and what not | 14:28 |
| pedroalvarez | before doing any work :) | 14:28 |
| edcragg | so a moonshot would be good then | 14:34 |
| edcragg | especially since it's phsically close the the cache server, i presume | 14:34 |
| edcragg | running mason | 14:34 |
| pedroalvarez | yes | 14:34 |
| edcragg | on baserock | 14:34 |
| pedroalvarez | hm... | 14:34 |
| edcragg | or would the 32 bit chroot build work on ubuntu? | 14:35 |
| pedroalvarez | the way to use moonshot for amrv7 now is using a chroot | 14:35 |
| pedroalvarez | I think the answer here (IIUC) is yes | 14:35 |
| edcragg | guess that would be the easiest way, unless we wanted to improve the baserock support for moonshot | 14:37 |
| persia | I have previously used 64-bit Ubuntu running on armv8 to chroot into 32-bit Baserock with some success. The key thing to remember is to run `linux32 chroot ..." when one enters the chroot. | 14:38 |
| paulsherwood | edcragg: what problems precisely are you aiming to solve? | 14:38 |
| edcragg | paulsherwood: an arm arifact cache, to help building on smaller arm systems where building locally would be impratical or impossible | 14:39 |
| edcragg | it would lower the bar for entry on these platforms, perhaps with smaller hobbyist boards | 14:39 |
| paulsherwood | well, to pitch for the ybd approach... its artifact server is properly atomic, can receive uploads from multiple build machines at once, can handle any format of artifact, has minimal dependencies, can run on any linux. i'd hope that a simple engineering assessment would lead to it being chosen over 'morph-cache-server', but i've been wrong here before. | 14:40 |
| paulsherwood | plus, any ybd developer can run it on his/her own machine to serve artifacts to his/her colleagues | 14:40 |
| pedroalvarez | ok, then problem solved | 14:40 |
| paulsherwood | the artifacts1.baserock.org is an instance running on Scaleway... $6 per month. it's been populated so far using moonshot build machines | 14:41 |
| paulsherwood | pedroalvarez: there's always room for improvement... | 14:42 |
| paulsherwood | i don;t know how scaleable it is (currently it's a bottle app with cherrypy) | 14:42 |
| paulsherwood | security could do with more thought | 14:42 |
| paulsherwood | etc | 14:42 |
| paulsherwood | and it has no tests :-) | 14:43 |
| persia | paulsherwood: So artifacts1.baserock.org has current ARM cache? | 14:43 |
| paulsherwood | yup | 14:43 |
| persia | heh. | 14:43 |
| SotK | is its api the same as morph-cache-server's? | 14:44 |
| paulsherwood | it's not automatically updating... i haven't got around to setting up any git-pull-build cron | 14:44 |
| persia | I think it is probably more interesting to consume the gerrit event stream, and build on merge, rather than using cron. | 14:45 |
| paulsherwood | SotK: not for ybd, and not all of m.c.s apo... but we did manage to serve artifacts to morph using it... | 14:45 |
| paulsherwood | https://github.com/devcurmudgeon/ybd/blob/master/kbas/__main__.py#L80 | 14:45 |
| paulsherwood | s/apo/api/ | 14:46 |
| persia | (could build-on-candidate, but that ends up being a lot of builds, most speculative) | 14:46 |
| paulsherwood | persia: that would be interesting... i'd be delighted to accept patches :) | 14:46 |
| pedroalvarez | patches for? | 14:46 |
| pedroalvarez | (just curious) | 14:46 |
| paulsherwood | anything that makes it better, pretty much :) | 14:47 |
| pedroalvarez | right | 14:47 |
| * persia read that as patches to bring artifacts1.baserock.org management into the infrastructure repo, along with an update to the system definition to consume the gerrit event stream and rebuild on merge. | 14:47 | |
| paulsherwood | but i specifically meant persia's suggestion of consuming gerrit, somehow triggering git pull and run ybd | 14:47 |
| pedroalvarez | Implementing this "to consume the gerrit event stream, and build on merge" sounds like doing something that is already done | 14:48 |
| paulsherwood | in mason, you mean? | 14:48 |
| persia | There's lots of things that consume the gerrit event stream. I don't think any of them run ybd as a result. | 14:48 |
| pedroalvarez | I can create a gocd instance that consumes gerrit patches, and gives feedback to gerrit | 14:48 |
| persia | Mason v3 does a very similar thing with morph. | 14:48 |
| *** toscalix_ has joined #baserock | 14:49 | |
| persia | pedroalvarez: Be careful of the "everything looks like a nail" problem. gocd is way more than you need just to trigger builds. | 14:49 |
| pedroalvarez | yes | 14:52 |
| paulsherwood | persia: true, but we do need to improve CD here :) | 14:52 |
| pedroalvarez | and yes | 14:52 |
| pedroalvarez | I've been investigating about CI systems that provide elastic agents support | 14:52 |
| pedroalvarez | I failed to find one so far | 14:53 |
| persia | Mason v3 does | 14:53 |
| pedroalvarez | jenkins has a plugin for aws, nothing for openstack I believe | 14:53 |
| persia | paulsherwood: Using a CD system to populate a cache is overkill, unless there is some other point to the CD. | 14:54 |
| persia | I'm not opposed to CD, but I am opposed to using a CD system because it can do a job, rather than because there is some expected delivery chain. | 14:54 |
| pedroalvarez | if we only want to populate the cache here, that problem is solved | 14:55 |
| paulsherwood | ack | 14:55 |
| pedroalvarez | I'm of course interested in something else | 14:56 |
| persia | That's fine, but be explicit about your agenda. | 14:58 |
| pedroalvarez | persia: I don't remember masonv3 being elastic | 14:58 |
| pedroalvarez | persia: hehe, yes | 14:58 |
| persia | pedroalvarez: Mason v3 uses zuul to consume gerrit, and dispatches work via gearman to an arbitrary number of workers, unless I read the wrong design documents. I never tested the final version. | 14:59 |
| pedroalvarez | the conversation started with poulating the cache, then I said I had a private mason, then people suggested to make it public,and then we started talking about cron jobs, gerrit streams, and CI | 14:59 |
| SotK | persia: that is correct, though it was only ever tested with one worker | 14:59 |
| persia | I don't think Mason v3 ever got nodepool support though | 14:59 |
| SotK | and we never did anything to get the workers elastically created | 15:00 |
| pedroalvarez | I see, so it will be ok with workers going up and down, but not actually turning them on and off | 15:00 |
| SotK | it'll be ok with them turning on and off too I think, though the test they were running would need to be retriggered | 15:03 |
| * SotK has been meaning to try to rebase that work onto master in his free time, but doesn't have much of that at the moment | 15:04 | |
| *** mariaderidder has quit IRC | 15:16 | |
| *** mariaderidder has joined #baserock | 15:16 | |
| *** toscalix_ has quit IRC | 15:18 | |
| jjardon | paulsherwood: are you OK if we add ybd to the build systems so everyone can use that arm cache? | 15:21 |
| pedroalvarez | I don't see why he would mind | 15:22 |
| pedroalvarez | don't put it in "morph-utils", it has the workd "morph" :) | 15:22 |
| paulsherwood | please do, jjardon :) | 15:23 |
| pedroalvarez | jjardon: don't use master | 15:24 |
| paulsherwood | jjardon: it could serve x86 artifacts too, if that would help | 15:26 |
| *** toscalix__ has joined #baserock | 15:54 | |
| gary_perkins | Hi, can anyone help with this? I'm trying to deploy a trove to DC's Openstack. The Glance image-create is producing the following error https://paste.codethink.co.uk/?15 There is an error and a couple of warnings in there. Not really sure what's going on | 15:55 |
| pedroalvarez | i can't see that paste | 15:55 |
| persia | gary_perkins: paste.baserock.org is maybe more useful :) | 15:56 |
| jjardon | paulsherwood: what is the parameter in config/ybd.conf to use artifacts1.baserock.org? same as morph, artifac-cache-server? | 15:56 |
| tiagogomes | gary_perkins, your username is incorrect. Set an appropriate OPENSTACK_USER in the cluster | 15:57 |
| pedroalvarez | I don't think that is the problem | 15:59 |
| gary_perkins | tiagogomes: that's correct for DC, also it would have failed the credentials check if that was so | 15:59 |
| pedroalvarez | gary_perkins: sounds like a hack, but run ` pip install eventlet==0.17.4` | 15:59 |
| pedroalvarez | I need to fix this in baserock systems | 15:59 |
| gary_perkins | http://paste.baserock.org/ibiseqiwub | 16:00 |
| gary_perkins | ok, I'll try that, thanks pedroalvarez | 16:00 |
| gary_perkins | also, is it possible to add a new parameter to the image-create? as DC require a '--property architecture=amd64' so it will spin up a VM on the right arch | 16:02 |
| *** wdutch has quit IRC | 16:02 | |
| *** wdutch has joined #baserock | 16:02 | |
| paulsherwood | jjardon: kbas-url: http://artifacts1.baserock.org:8000/ | 16:03 |
| jjardon | paulsherwood: Its working! thanks :) | 16:06 |
| pedroalvarez | patch sent: https://gerrit.baserock.org/#/c/1391/ | 16:10 |
| paulsherwood | jjardon: great! | 16:14 |
| *** mariaderidder has quit IRC | 16:16 | |
| *** mariaderidder has joined #baserock | 16:16 | |
| edcragg | for some reason i don't seem to be able to clone the bsp-support repo - http://paste.baserock.org/oweqefoyon | 16:20 |
| tiagogomes | edcragg, try `git clone git://git.baserock.org/baserock/baserock/bsp-support` | 16:21 |
| edcragg | tiagogomes: yes, i suspect that might work, but this is morph, using 'upstream:bsp-support' | 16:22 |
| *** ctbruce has quit IRC | 16:23 | |
| straycat | edcragg, do you not want baserock/baserock/bsp-support ? | 16:23 |
| straycat | so, baserock:bsp-support | 16:24 |
| tiagogomes | edcragg that is a badly constructed morph then, because 'upstream' is not part of the default alias | 16:24 |
| edcragg | oh, is that new? | 16:24 |
| straycat | edcragg, baserock is an alias that points to your trove | 16:24 |
| straycat | well, yourtrove/baserock | 16:25 |
| edcragg | right, it has always worked before, i've never come across this 'baserock:' | 16:27 |
| *** toscalix__ has quit IRC | 16:27 | |
| *** toscalix has joined #baserock | 16:28 | |
| straycat | upstream is just an alias for yourtrove/delta, so if you somehow created delta/bsp-support on a trove you were using then it might have worked there | 16:28 |
| edcragg | i'm just using g.b.o, and always have been | 16:29 |
| *** mariaderidder has quit IRC | 16:30 | |
| straycat | that's odd then, since there is no bsp-support in delta, since i created bsp-support on gbo | 16:30 |
| straycat | does it work if you switch to baserock:bsp-support? | 16:30 |
| edcragg | just trying now | 16:32 |
| edcragg | ERROR: Cannot find remote git repository: baserock:bsp-support | 16:33 |
| SotK | edcragg: can you paste your morph config somewhere? | 16:34 |
| edcragg | SotK: http://paste.baserock.org/zubuvobaye | 16:36 |
| SotK | o.O | 16:38 |
| SotK | is there anything useful in the morph log around the time of the error? | 16:39 |
| *** persia_ has joined #baserock | 17:01 | |
| straycat | edcragg, baserock:baserock/bsp-support isn't it, sorry | 17:11 |
| ssam2 | baserock:baserock/bsp-support not baserock:bsp-support surely ? | 17:11 |
| *** paulwaters_ has quit IRC | 17:13 | |
| edcragg | SotK: http://paste.baserock.org/lajazixeki | 17:13 |
| tiagogomes | it should be baserock:baserock/bsp-support | 17:14 |
| edcragg | guess that's the wrong url, though | 17:14 |
| edcragg | yep | 17:14 |
| edcragg | still confused why it worked before | 17:14 |
| *** toscalix has quit IRC | 17:15 | |
| edcragg | and there's a big precedent for 'upstream:' in definitions, 880 references, but 18 references to 'baserock:' | 17:15 |
| edcragg | 'baserock:baserock/bsp-support' does seem to have worked, though | 17:18 |
| edcragg | thanks! | 17:19 |
| tiagogomes | upstream are used for projects which are under a delta prefix in the trove, confusing | 17:21 |
| tiagogomes | baserock is set to git.baserock.org unless trove_host was given | 17:21 |
| edcragg | yep, i understand now :) | 17:27 |
| edcragg | are these things documented somewhere? | 17:27 |
| tiagogomes | mm, not sure | 17:29 |
| tiagogomes | nothing better as looking into the code :) | 17:29 |
| ssam2 | the repo alias thing is a bit undocumented | 17:36 |
| ssam2 | although it's basically the same idea as https://en.wikipedia.org/wiki/CURIE | 17:36 |
| ssam2 | http://wiki.baserock.org/definitions/current/ should definitely document how this works | 17:44 |
| ssam2 | but doesn't right now | 17:44 |
| *** CTtpollard has quit IRC | 17:46 | |
| *** bashrc_ has quit IRC | 17:56 | |
| *** nowster has quit IRC | 18:07 | |
| *** ssam2 has quit IRC | 18:19 | |
| tiagogomes | grrr, can I resize the comment box on gerrit? I can't write anything useful there | 18:19 |
| *** gary_perkins has quit IRC | 18:20 | |
| *** gary_perkins has joined #baserock | 18:21 | |
| edcragg | tiagogomes: i can, by clicking and dragging the bottom right corner | 18:23 |
| tiagogomes | I can't do that. I am talking about that box with the -1,+1 radio button | 18:24 |
| edcragg | pedroalvarez: i have tested deployment for the remaining SoCFPGA dev kit definitions - https://gerrit.baserock.org/#/c/799/9 | 18:24 |
| tiagogomes | edcragg it did work? | 18:25 |
| edcragg | tiagogomes: yes, that one | 18:25 |
| edcragg | tiagogomes: partitioning you mean? | 18:25 |
| tiagogomes | yep | 18:25 |
| edcragg | yes :) | 18:25 |
| edcragg | it worked all along, all i had to do to get it to work with the merged partitioining extension was add the USE_PARTITONING=yes | 18:26 |
| edcragg | had to also fix some things in the systems which were already merged, too | 18:27 |
| tiagogomes | cool | 18:32 |
| *** tiagogomes has quit IRC | 18:58 | |
| *** edcragg has quit IRC | 19:23 | |
| *** paulwaters_ has joined #baserock | 20:16 | |
| *** paulwaters_ has quit IRC | 20:19 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!