IRC logs for #baserock for Friday, 2015-11-06

*** gtristan has quit IRC04:45
*** gtristan has joined #baserock05:07
*** gtristan has quit IRC08:31
*** ctbruce has joined #baserock08:32
*** tiagogomes has joined #baserock08:44
*** mariaderidder has joined #baserock09:05
*** bashrc_ has joined #baserock09:06
*** toscalix__ has joined #baserock09:14
*** gtristan has joined #baserock09:27
*** edcragg has joined #baserock09:38
*** ssam2 has joined #baserock09:49
*** ChanServ sets mode: +v ssam209:49
tiagogomesgtristan, thanks for clarifying me on your last email09:49
*** nowster has joined #baserock09:54
*** toscalix__ has quit IRC09:54
gtristantiagogomes, no problem :)09:58
gtristanssam2, says that I do not have to go into such level of detail in a proposal, but in my experience you do - otherwise people invariably have misunderstandings :)09:59
* gtristan waits for jjardon to arrive...10:00
*** mariaderidder has quit IRC10:09
*** dabukalam has quit IRC10:12
*** pedroalvarez has quit IRC10:12
*** pedroalvarez has joined #baserock10:12
*** ChanServ sets mode: +v pedroalvarez10:12
*** dabukalam has joined #baserock10:13
*** toscalix__ has joined #baserock10:16
* gtristan hopes he kept his pre-systemd-upgrade artifacts in tact...10:19
gtristanhmmm, this does not make sense10:21
gtristan[1/17/413] [samba] ...10:21
gtristanthe gnome stratum depends on the samba stratum... and the gnome stratum contains *way* more than 17 chunks10:21
paulsherwoodgtristan: i think ybd's counter is misbehaving10:21
* gtristan does not understand10:21
paulsherwoodare you using last tag, or later?10:22
gtristandamn, and I was hoping that I could recover10:22
gtristanlast tag10:22
gtristanlast time when you told me there was one10:22
gtristanlets see what it does... I have a lingering hope that I can test pre-systemd-upgrade without rebuilding webkit10:23
paulsherwoodthe problem i'm aware of is after last tag10:23
gtristanAh !10:23
paulsherwood(in fact it's introduced by the last commit)10:23
gtristanI know10:23
gtristanyay10:23
gtristanok its going to work10:23
gtristanthis is nice :)10:23
paulsherwood:-)10:23
paulsherwoodwhat's the explanation?10:24
gtristanI removed the samba artifact10:24
gtristanbut I have all the others10:24
gtristanybd found that sambas cache key needs to be rebuilt for this config10:24
gtristanbut, all the other cache keys are present, so depending chunks wont need a rebuild :)10:24
paulsherwood:)10:24
gtristanit will take about 30min I guess to verify this...10:25
* gtristan prepares to pounce on jjardon10:25
*** mariaderidder has joined #baserock10:37
rjekgtristan: Get ready to pounce.11:09
*** jjardon has joined #baserock11:14
tiagogomesnow!11:15
gtristanahem11:15
gtristanjjardon11:15
gtristaneither one of two things happened11:15
gtristanOne of them, is that you have a different setup, VM, machine than I do, and that everything works fine for you (I doubt this, but, benefit of the doubt and all)11:16
gtristanThe other possibility, is that you thought it was acceptable to push a systemd upgrade to master without verifying that we still reach a graphical boot11:17
gtristanFor those who enjoy upgrading low level components, let them also enjoy verifying dependent system integrity, otherwise, please - lets keep those upgrades outside of definitions master.11:19
gtristanI have lost several hours without even identifying with certainty that that is the actual faulty commit, actually11:20
ssam2in fairness, it was actually me who pushed it, and I admit to not doing any amount of testing. i'm happy to stop +2'ing definitions patches if it's causing problems11:20
ssam2I mostly do it due to the paucity of reviewers11:20
gtristanIt can be anything after b18ff44 (Adding gnome-control-center) and before and up until cca2b07 (systemd upgrade)11:20
ssam2but I did think that 'Tested this with a x86_64 weston system' meant that it was tested :-)11:21
gtristanunfortunately, with baserock, performing a bisection, manually or otherwise - takes at least a day of lag in rebuilds11:23
paulsherwoodgtristan: it wouldn't if you were using a fast AWS machine11:23
gtristanthe point is that, when comparing this project to a project like, say, GTK+ or an application, ensuring that your commits dont break something before pushing upstream is even more critical11:25
gtristanthe cost of picking up the pieces and identifying exactly what broke is more11:25
ssam2this problem was solved a while ago for Morph uses with https://mason-x86-64.baserock.org/11:26
ssam2although it's really rubbish, it does autobuild stuff as soon as it gets committed11:26
jjardongtristan: whats the problem11:26
ssam2would something similar for ybd reduce the pain?11:27
ssam2(i don't have time to do that at present, just wondering)11:27
ssam2(but should be easy for anyone else to set up)11:27
gtristanI have verified that master is broken, and I have verified that my pending patch set works when staged against my pre-systemd-update branch11:27
gtristanjjardon, boot up master, black screen; no gnome-initial-setup, automatically enabled gdm = bricked11:28
gtristanssam2, it is not sufficient to know that "things built"11:28
jjardongtristan: what is broken exactly? ah, the gnome system11:28
pedroalvarezgtristan: that is true11:29
ssam2gtristan: but it means that bisect is pretty quick because you can download the prebuilt artifacts11:29
ssam2gtristan: i agree much more automated testing is needed, but seems there's no resources/willing people to make that happen at present11:29
gtristanssam2, what is needed is a stricter policy when making changes to low level strata11:30
jjardongtristan: I only tested the weston one, Id say revert the commit with a note11:31
gtristanI dont know how you would automatically test that a change in the init system, which *probably* requires system integration changes, would actually result in a working PAM setup and login11:31
ssam2didn't gnome-continous used to do that?11:33
ssam2seems to have stopped doing that these days, but it used to show a screenshot of logged-in desktop for each build11:33
gtristanssam2, I'm not sure that it does really, it runs installed tests, which is a step up from stuff we dont have11:34
gtristanit apparently takes screenshots, but that does not mean that it accounts for a properly integrated gdm11:34
ssam2ok. I figured that if you get to graphical desktop them GDM must be working11:34
gtristanssam2, it may very well bypass that by running gnome-session without gdm, but perhaps, I'm not sure11:34
gtristanthe thing to be clear about, is that GNOME assumes the system works - below the GNOME stack is "not GNOME's problem"11:35
*** brlogger` has joined #baserock11:38
*** brlogger has quit IRC11:38
gtristansuch CI would have to be done in a separate repo11:38
jjardongtristan: https://git.gnome.org/browse/gnome-continuous/tree/manifest.json#n10211:38
jjardonoh, sam was faster :)11:38
radiofreedoes anyone know how they automate testing the wayland backend? it's in qemu right?11:40
jjardongtristan: if something is broken you can tag the component until it get fixed11:40
gtristanAnyway, at this point we have a choice; offload this problem to me, so that I lose time focusing on my actual mandate, or verify that that is infact the broken commit and revert it until it can be introduced without breaking things11:40
jjardongtristan: sure, revert the systemd upgrade commit if that breaks the gnome system11:41
gtristanI can try master with the reverted systemd, it will take the rest of the day to validate that it is not something else in that handful of 10 commits (at-spi changes ? doubtful they can brick a system)11:41
ssam2as a cheap hack to avoid the full rebuild, you can add a new systemd stratum at the top of the dependency graph with old systemd11:44
ssam2the old version will overwrite the new version in the resulting system11:44
gtristanlooks like it doesnt have system integration hooks11:47
gtristanso maybe I would have enough faith that that would be correct :-/11:48
gtristanThere is another alternative, which is that I push my most recent patch set, which I have verified against an older master (based on: "b18ff44 Adding gnome-control-center to GNOME strata")11:52
gtristanAnd not claim responsibility for doing the detective work and verifying that the systemd upgrade is the one in b18ff44...cca2b07 which causes the issue11:53
jjardongtristan: push your branch, but that commit is more than a week old ...12:01
gtristanI'm rebuilding, do push my branch would be to perpetuate the idea that it's ok to push stuff without validating first12:12
jjardongtristan: I do not think anyone is pushing stuff without validating first12:13
gtristanjjardon, fwiw, I understand its a mistake - it's just of paramount importance that it be clear12:13
gtristanthat was not my impression, to be honest12:14
gtristanI am unsure if any of my commits have compromised stability of adjacent systems, this worries me in general12:14
jjardongtristan: but the amount of validation a individual can do is limited; for exameple I can not build every single system in definitions to check a change is not regressing in any of them12:14
gtristanjjardon, however you should, and I should12:15
gtristanor else I should not make any change to a low level stratum12:15
*** mariaderidder has quit IRC12:15
gtristanthat is the responsibility that comes with sharing low level components12:15
gtristanotherwise we send the message that we just dont really care about the general stability of the systems we maintain12:16
jjardongtristan: agree, we should, but thats not really a realistic thing to demand to a contributor: I do not even have the hardware to test some of the systems. thats why we need propper ci to test this for us12:20
gtristanat least we need a staging area12:21
gtristanthe cost of validating every outcome, and being careful to do so when modifying low level components in general, is lower than picking up the pieces12:21
gtristanwhen things break12:22
gtristanalso, if there are system morphs in the systems/ directory which do not have a dedicated maintainer, somebody responsible and building those, they should probably be dropped12:22
gtristanit makes no sense to publish a system which we know "builds" for so long, but nobody has a clue if it actually works12:23
radiofreethat makes sense12:23
gtristanthat reflects worse on the project than being so proud of having many systems12:23
jjardonyeah, I suggested that several times; we dont even know if they builds12:23
jjardongtristan: actually, I suggest that we only offer systems that are in the ci; there is no other way to know that they at least compile12:24
gtristanI would wish we had some separation for those12:25
gtristansending a clear message of which systems we vouch for their stability and are maintained and receive regular attention, vs. which systems have merely passed some automated CI12:25
jjardongtristan: I agree, but the CI is better than nothing. Also I guess you have to define what "maintained" and "regular attention" means; Is there a person that occasionally build the system and checks it works? is there a machine that runs some funtional test in the generated system? ...12:34
*** ssam2 has quit IRC12:35
gtristanjjardon, first, stable verified releases12:36
gtristanstable branches12:36
*** ssam2 has joined #baserock12:36
*** ChanServ sets mode: +v ssam212:36
gtristanjjardon, if we first sent a message that we do maintain stable branches, and that we take stability of specific systems very seriously, we send a message that we are a venue for contributing patches back upstream12:38
gtristanbefore we demonstrate that responsibility and at least organization, our baselines will not receive contributions from any user base12:38
pedroalvarezI want to point out that we don't have CI12:39
gtristanwe need to demonstrate that if a security bug is found in a downstream IVI system, that contributing the patch upstream will be kept in a stable branch somewhere12:39
gtristanperforming all of the manual validation ourselves is impossible, we need to collect bugs from the field and make sure we always improve stability12:40
jjardonpedroalvarez: mason?12:41
gtristanCI is another nice-to-have feature, to be sure - but being well organized and proving that we have a stable vs experimental branching scheme is, IMO a higher priority and brings more long term value12:41
gtristanwe need to coordinate between "systems" maintainers to make a choice of what version of systemd and other low level components we agree upon for the "next stable branch" (current experimental)12:42
gtristanand *not* upgrade things dramatically in stable, other than collected security fixes and other important patches, pull strictly from agreed upon upstream stable branches12:43
jjardongtristan: we branch the entire definitions or every system has its own branch?12:43
persiajjardon: mason is not CI12:43
persia(or at least not mason v2, which is what is deployed)12:43
gtristanjjardon, I think A.) We reduce definitions to 3 or 4 maintained systems  B.) We branch definitions as a whole12:44
*** mariaderidder has joined #baserock12:44
persiagtristan: It depends on the goals.  Baserock has moved back and forth between two goals over time: that of being a sensible curated software collection and that of being a set of tools to build software collections.12:44
gtristanjjardon, maybe we can have something like gstreamer (i.e. gstreamer-plugins-bad) for a separate repo of not-really-maintained systems12:45
persiaMost of the folk using Baserock today ignore the reference systems entirely, and are interested in the tools: more because what they are doing has nothing to do with the reference systems than because of any policies surrounding the reference systems.12:45
jjardonmmm, so what happen with external contributors that want to add a new systems ?12:45
gtristanpersia, it cannot really be successful at the latter without being successful as the former12:45
gtristanafaics12:45
persiagtristan: If you like: as I said, none of the current users of the tools use the reference systems in any way, or have expressed any interest in doing so.12:46
pedroalvarezsorry to say this, but Mason is not a real CI12:46
persiaBut that doesn't mean there aren't folk who care about that sort of thing.12:46
gtristanjjardon, they have to demonstrate their capability of maintaining that system over time within the baserock community12:46
persiaWhy "external contributors"?  The current model seems to work without any "internal/external" requirements.  Let it stay that way.12:47
gtristanI think the distinction is more "new contributors"12:47
jjardonpersia: pedroalvarez what is mason then? and what its missing to not be a real CI?12:47
gtristanThe point is we cannot, as a project, advertise that we support a system as stable, if nobody is willing to step up and do maintenance of that system12:48
persiajjardon: Mason v2 performs some limited build validation post-merge, but doesn't have facilities for extended testing, nor for any sort of pre-merge validation.12:48
persiamason v3 moves stuff into pre-merge, and has some framework for wider testing, but I never heard of it being completed.12:48
pedroalvarezmason(v1, the one currently in use) is a devel system that is pulling every minute, and if the sha1 changes it builds and deploys ci.morph12:48
persiapedroalvarez: v1 is the jenkins one: current is v212:49
pedroalvarezsorry looks like the one in production is v212:49
pedroalvarezyes12:49
jjardonpersia: is pre-merging a requirement for a CI? I wouldn't think so12:50
persiajjardon: pre-merge is a requirement to not land failed builds in master, but no, not a requirement for CI.  Being able to take action on commit is a requirement for CI, rather than a scheduled activity.12:51
persiamason v2 does neither.12:52
jjardonpersia: you can say that in our case that action is generate a red or green line in a webpage12:53
persiajjardon: Except that the action isn't based on a commit happening.12:54
persiamason v2 polls the repo occasionally, and runs a build if it sees a difference, but whether that is one commit or a million commits is unknown to Mason v2.12:54
persiaSo it isn't possible to know which commit caused the red line except by coincidence.12:54
jjardonah, I get you know12:55
persiaMason v2 is a reasonable system to do snapshot testing: I'd like more comprehensive tests, but it works.  I don't mean to suggest it doesn't have value, only to support the argument that it fails to be "continuous".12:56
jjardonbut then again; is a requirement for a CI to build changes/ present result for _every_ commit; the definitions in wikipedia says "several times a day", it doesnt specify how often12:56
persiaIt depends on one's semantics.12:57
jjardonpersia: sure, im only curious why some people say "is not CI"12:57
persiaFor my personal semantics, anything post-merge isn't "CI", but only "CD".12:57
persiaI think most people currently have semantics that limit "CI" to every-commit systems.12:57
persia(although there is debate on the pre-merge/post-merge side of things)12:57
ssam2mason v2 also isn't integrating anything, so at best it's "continuous build"12:58
persiassam2: To be fair, Travis also doesn't "integrate", and folk call that "CI".12:59
persiaI think "Integrate" in "CI" has come to mean "build + test"12:59
pedroalvarez"continuous build but I'm not going to build every commit, and it's going to take some days to give you any feedback"12:59
pedroalvarezI've been trying to put the baserock.org ssl certificate in g.b.o13:00
pedroalvarezbut looks like cloning from https still doesn't work13:00
persia:(13:01
jjardonI think there si a patch from straycat in the mailing list to fix that13:01
pedroalvarezjjardon: that's another issue, and the patch is to fix the ca-certificates chunk13:03
pedroalvarezwhich is broken since we moved to python313:03
pedroalvarezI can't clone using https even from debian13:03
*** toscalix_ has joined #baserock13:05
pedroalvarezI don't know exactly how this SSL certs work. Are we expecting the user to install it manually in their systems?13:06
pedroalvarezshould we install the baserock.org SSL cert in baserock systems by default?13:06
pedroalvarezor maybe it should just work without any manual steps13:07
pedroalvarez?13:07
*** toscalix__ has quit IRC13:08
tiagogomesI don't know how it works, but you don't need to install anything on the systems where you are running `git clone`13:09
pedroalvareztiagogomes: I guess that depends on how trusted is the certificate and how well it has been distributed13:11
pedroalvarezbut I can be wrong, as I said "I don't know exactly how this SSL certs work."13:11
persiaWhich CA asserts the b.o cert is trustworthy?13:18
persiaIf it is not one pre-delegated as a trust authority by users, it is likely to raise an alert.13:19
persiaThe set of currently trusted certs from Mozilla.org is a reasonable proxy for the certs trusted by most users.13:20
persiaerr, CAs delegated13:20
jjardongtristan: I found the commit that broke the session: https://gerrit.baserock.org/#/c/1390/13:23
pedroalvarezI'm failing to understand and answer your question13:23
persiapedroalvarez: Is the public part of the cert somewhere I can inspect?13:24
pedroalvarezyes13:24
persiaEssentially a certificate is signed by a certificate authority, to indicate that that certificate authority believes that certificate to be genuine.13:24
pedroalvarezhttp://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/infrastructure.git/tree/certs/baserock.org-ssl-certificate-temporary-dsilverstone.full.cert13:24
SotKpedroalvarez: I think the question is "which CA signed the b.o cert", but I could be totally mistaken as I also only have a minimal understanding13:24
jjardonCan anyone take a look to gerrit to lorry some pending GNOME components, please?13:24
* gtristan 's internet is crapping out here... loading url13:24
gtristanjjardon, not systemd ?13:24
persiaAnd most computers have a common set of certificate authorities to whom the delegate the decision as to whether a certificate is valid.13:25
jjardongtristan: nope13:25
pedroalvarezmaybe: StartCom Class 2 Primary Intermediate Server CA ?13:25
persiaSo, if a certificate is signed by one of the delegated authorities, then that certificate is considered valid.  If not, the user is usually prompted.13:25
pedroalvarezand these certificates are usually more expensive, right?13:26
persiaSome certificate authorities will accept money as proof of authenticity.  Others have administrative fees as part of the approval process.  Others have no-fee processes of variable reliability.13:27
ssam2the one we have for baserock.org web services is trusted by Firefox out-of-the-box13:27
gtristanjjardon, the revert happens before the commit it reverts ?13:27
ssam2our ca-certificates chunk comes from Debian13:28
ssam2so if Debian trusts the baserock.org certificate out-of-the-box, no problem13:28
ssam2since kinnison provided the cert and is a Debian user, I imagine he'd spot if the CA provider he used wasn't trusted by Debian.13:28
persiapedroalvarez: Indeed, startcom is the relevant CA here.13:28
gtristanso a new revert on top of the commit after it's revert, I see13:29
gtristanjjardon, alright I'll remove my systemd revert from my tree and restart my build13:29
gtristanit should complete over night13:29
* gtristan rebases against master13:29
pedroalvarezssam2: I guess the problem is somewhere else, given that I can `curl https://mason-x86-64.baserock.org/`13:29
pedroalvarez(from debian)13:30
pedroalvarezand from baserock13:30
persiapedroalvarez: When you `git clone ...` do you get "fatal: unable to access 'https://git.baserock.org/git/baserock/baserock/infrastructure.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none" ?13:31
pedroalvarezand I can't do the same with git.baserock.org13:31
pedroalvarezpersia: yes13:31
gtristanah, total clusterfsck, commit, revert, commit again... now revert again13:32
* gtristan wonders how that made it into master13:32
persiapedroalvarez: What do you get if you run `openssl verify ${KEY}`13:33
persiaI get failure locally, and I get the same error as you trying to run `git clone ...`13:34
pedroalvarezpersia: http://paste.baserock.org/kedudifubo13:35
persiaSame for me.  Frustratingly, I seem to have StartCom_Certification_Authority_2.crt enabled in my ca-certificates configuration.13:36
persiaFor the record, the other incredibly useful openssl command is `openssl x509 -in ${CERT} -text -noout`, which prints the details of the certificate.13:37
persiaMy suggestion is to ask Kinnison for guidance on whether the certificate is one he can verify on his setup.13:38
pedroalvarezyeah, sounds sensible13:38
persiaIf not, then perhaps something went wonky in the request process.  If so, then it becomes interesting to explore what differences exist between there and yours/mine.13:38
*** gtristan has quit IRC13:39
pedroalvarezpersia: my guess is that I haven't installed it properly, given that: http://paste.baserock.org/uhigozebej13:43
persiapedroalvarez: That is the same error message as http://paste.baserock.org/kedudifubo13:44
straycat13:03 +pedroalvarez$ jjardon: that's another issue, and the patch is to fix the ca-certificates chunk13:44
persiaThe problem is with the trust of the certificate, not with how the server is configured.13:45
straycatthe https clone issue is fixed by the patch i sent to the list a couple of days ago13:45
persiaThe fact that you get the same error is an indication that you installed it correctly.13:45
pedroalvarezstraycat: yes, this is not to fix curl in baserock, is to fix https in g.b.o13:45
persiastraycat: slightly different: your patch fixes running git or curl on baserock hosts.  The issue we're debugging is about using git or curl on non-Baserock hosts to access g.b.o13:46
straycatokay i didn't actually read the backscroll13:46
pedroalvarezpersia: but, I believe that the same cert is installed in mason, and it doesn't fail13:46
straycatsince no one replied to my patch yet i figured people had forgotten13:46
persiapedroalvarez: I don't believe it is the same cert13:46
pedroalvarezhm..13:47
pedroalvarezstraycat: sorry, I didn't feel ready to review the patch :/13:47
straycatthat's fine i thought about it today and decided that my past self wouldn't have been able to review it easily either :)13:48
pedroalvarezI should review it just for learning13:49
pedroalvarezpersia: hm.. it should be, I've done something wrong. I'll investigate13:50
* SotK notices that the lighttpd config in trove-setup points to /etc/lighttpd/certs/lighttpd.pem13:50
SotKwhich appears to be created here: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/trove-setup.git/tree/ansible/roles/trove-setup/tasks/lighttpd.yml#n613:50
pedroalvarezand what i've done so far is, replaced that .pem file, and restarted lighttpd-git13:52
*** gtristan has joined #baserock13:55
*** toscalix_ has quit IRC14:02
edcraggpedroalvarez: i hear you have some ideas on how an ARM artifact cache has been done before, or could be done in the future?14:18
pedroalvarezyes14:18
pedroalvarezcurrently we have arm artifacts in cache.baserock.org14:18
pedroalvarezit is a non-public mason, running on a jetson on my desk, but pushing artifacts every build14:19
pedroalvarezbefore we had jetsons in DataCentred14:19
paulsherwoodartifacts1.baserock.org:8000 is serving arm artifacts for current definitions14:19
paulsherwood(ybd only)14:19
radiofreeedcragg: use a mustang?14:20
radiofreehint hint14:20
pedroalvarezmoonshot?14:20
paulsherwoodi'm considering a cron job to trigger git pulls and run ybd automatically to keep it populated14:20
radiofreedo we have access to them anymore?14:20
paulsherwoodradiofree: yes14:20
pedroalvarezif you manage to publish the logs, and a status page, then will have reinvented mason14:21
radiofreeit's the same anyway, you just want to build armv7lhf on aarch64, there's a guide on the mailing list14:21
radiofreedo that and push the cache somewhere14:21
pedroalvarezthat was one of the ideas I was going to suggest14:21
radiofreeusing a cron job as paulsherwood suggested would suffice14:21
paulsherwoodpedroalvarez: yes, but faster :)14:21
radiofreealthough i think the preferred way is a systemd timer now14:21
edcraggthis sounds promising14:22
pedroalvarezfor now it only sounds confusing14:22
paulsherwoodedcragg: if you're just seeking existing artifacts, and are willing to use ybd, all of arm7lhf ci.morph should already be at http://artifacts1.baserock.org:8000/*14:22
radiofreei don't think it's confusing14:22
*** gtristan has quit IRC14:23
pedroalvarezradiofree:  I see no point on using a cron job, that will only publish the artifacts on the same way that I'm doing now14:23
pedroalvarezmaybe faster, but hey14:23
radiofreeapparent from requiring a pedroalvarez to do it manually14:23
pedroalvarezand maybe not in my desk14:23
radiofrees/apparent/apart14:24
*** gtristan has joined #baserock14:24
pedroalvarezI'm not doing it manually14:24
radiofreeoh right, so we're already supplying arm caches?14:24
pedroalvarezyes14:24
pedroalvarezas I said is a non-public mason14:24
pedroalvarezworking on the same way as the others14:25
radiofreeah cool, why not make it public then?14:25
radiofreeis that building on a moonshot?14:25
pedroalvarez<pedroalvarez> it is a non-public mason, running on a jetson on my desk, but pushing artifacts every build14:25
radiofreeah ok, it would be ideal to use a moonshot/mustang rather than a single jetson14:26
pedroalvarez<pedroalvarez> that was one of the ideas I was going to suggest14:26
paulsherwoodis there an echo in here? :-)14:26
pedroalvarez<pedroalvarez> yes14:27
paulsherwoodlol14:27
pedroalvarezjust kidding in the last one14:27
pedroalvarezbut the others were answers to similar questions14:27
pedroalvarezOk, edcragg and I are trying to evaluate what would be good for the project and what not14:28
pedroalvarezbefore doing any work :)14:28
edcraggso a moonshot would be good then14:34
edcraggespecially since it's phsically close the the cache server, i presume14:34
edcraggrunning mason14:34
pedroalvarezyes14:34
edcraggon baserock14:34
pedroalvarezhm...14:34
edcraggor would the 32 bit chroot build work on ubuntu?14:35
pedroalvarezthe way to use moonshot for amrv7 now is using a chroot14:35
pedroalvarezI think the answer here (IIUC) is yes14:35
edcraggguess that would be the easiest way, unless we wanted to improve the baserock support for moonshot14:37
persiaI have previously used 64-bit Ubuntu running on armv8 to chroot into 32-bit Baserock with some success.  The key thing to remember is to run `linux32 chroot ..." when one enters the chroot.14:38
paulsherwoodedcragg: what problems precisely are you aiming to solve?14:38
edcraggpaulsherwood: an arm arifact cache, to help building on smaller arm systems where building locally would be impratical or impossible14:39
edcraggit would lower the bar for entry on these platforms, perhaps with smaller hobbyist boards14:39
paulsherwoodwell, to pitch for the ybd approach... its artifact server is properly atomic, can receive uploads from multiple build machines at once, can handle any format of artifact, has minimal dependencies, can run on any linux. i'd hope that a simple engineering assessment would lead to it being chosen over 'morph-cache-server', but i've been wrong here before.14:40
paulsherwoodplus, any ybd developer can run it on his/her own machine to serve artifacts to his/her colleagues14:40
pedroalvarezok, then problem solved14:40
paulsherwoodthe artifacts1.baserock.org is an instance running on  Scaleway... $6 per month. it's been populated so far using moonshot build machines14:41
paulsherwoodpedroalvarez: there's always room for improvement...14:42
paulsherwoodi don;t know how scaleable it is (currently it's a bottle app with cherrypy)14:42
paulsherwoodsecurity could do with more thought14:42
paulsherwoodetc14:42
paulsherwoodand it has no tests :-)14:43
persiapaulsherwood: So artifacts1.baserock.org has current ARM cache?14:43
paulsherwoodyup14:43
persiaheh.14:43
SotKis its api the same as morph-cache-server's?14:44
paulsherwoodit's not automatically updating... i haven't got around to setting up any git-pull-build cron14:44
persiaI think it is probably more interesting to consume the gerrit event stream, and build on merge, rather than using cron.14:45
paulsherwoodSotK: not for ybd, and not all of m.c.s apo... but we did manage to serve artifacts to morph using it...14:45
paulsherwoodhttps://github.com/devcurmudgeon/ybd/blob/master/kbas/__main__.py#L8014:45
paulsherwoods/apo/api/14:46
persia(could build-on-candidate, but that ends up being a lot of builds, most speculative)14:46
paulsherwoodpersia: that would be interesting... i'd be delighted to accept patches :)14:46
pedroalvarezpatches for?14:46
pedroalvarez(just curious)14:46
paulsherwoodanything that makes it better, pretty much :)14:47
pedroalvarezright14:47
* persia read that as patches to bring artifacts1.baserock.org management into the infrastructure repo, along with an update to the system definition to consume the gerrit event stream and rebuild on merge.14:47
paulsherwoodbut i specifically meant persia's suggestion of consuming gerrit, somehow triggering git pull and run ybd14:47
pedroalvarezImplementing this "to consume the gerrit event stream, and build on merge" sounds like doing something that is already done14:48
paulsherwoodin mason, you mean?14:48
persiaThere's lots of things that consume the gerrit event stream.  I don't think any of them run ybd as a result.14:48
pedroalvarezI can create a gocd instance that consumes gerrit patches, and gives feedback to gerrit14:48
persiaMason v3 does a very similar thing with morph.14:48
*** toscalix_ has joined #baserock14:49
persiapedroalvarez: Be careful of the "everything looks like a nail" problem.  gocd is way more than you need just to trigger builds.14:49
pedroalvarezyes14:52
paulsherwoodpersia: true, but we do need to improve CD here :)14:52
pedroalvarezand yes14:52
pedroalvarezI've been investigating about CI systems that provide elastic agents support14:52
pedroalvarezI failed to find one so far14:53
persiaMason v3 does14:53
pedroalvarezjenkins has a plugin for aws, nothing for openstack I believe14:53
persiapaulsherwood: Using a CD system to populate a cache is overkill, unless there is some other point to the CD.14:54
persiaI'm not opposed to CD, but I am opposed to using a CD system because it can do a job, rather than because there is some expected delivery chain.14:54
pedroalvarezif we only want to populate the cache here, that problem is solved14:55
paulsherwoodack14:55
pedroalvarezI'm of course interested in something else14:56
persiaThat's fine, but be explicit about your agenda.14:58
pedroalvarezpersia: I don't remember masonv3 being elastic14:58
pedroalvarezpersia: hehe, yes14:58
persiapedroalvarez: Mason v3 uses zuul to consume gerrit, and dispatches work via gearman to an arbitrary number of workers, unless I read the wrong design documents.  I never tested the final version.14:59
pedroalvarezthe conversation started with poulating the cache, then I said I had a private mason, then people suggested to make it public,and then we started talking about cron jobs, gerrit streams, and CI14:59
SotKpersia: that is correct, though it was only ever tested with one worker14:59
persiaI don't think Mason v3 ever got nodepool support though14:59
SotKand we never did anything to get the workers elastically created15:00
pedroalvarezI see, so it will be ok with workers going up and down, but not actually turning them on and off15:00
SotKit'll be ok with them turning on and off too I think, though the test they were running would need to be retriggered15:03
* SotK has been meaning to try to rebase that work onto master in his free time, but doesn't have much of that at the moment15:04
*** mariaderidder has quit IRC15:16
*** mariaderidder has joined #baserock15:16
*** toscalix_ has quit IRC15:18
jjardonpaulsherwood: are you OK if we add ybd to the build systems so everyone can use that arm cache?15:21
pedroalvarezI don't see why he would mind15:22
pedroalvarezdon't put it in "morph-utils", it has the workd "morph" :)15:22
paulsherwoodplease do, jjardon  :)15:23
pedroalvarezjjardon: don't use master15:24
paulsherwoodjjardon: it could serve x86 artifacts too, if that would help15:26
*** toscalix__ has joined #baserock15:54
gary_perkinsHi, can anyone help with this? I'm trying to deploy a trove to DC's Openstack. The Glance image-create is producing the following error https://paste.codethink.co.uk/?15 There is an error and a couple of warnings in there. Not really sure what's going on15:55
pedroalvarezi can't see that paste15:55
persiagary_perkins: paste.baserock.org is maybe more useful :)15:56
jjardonpaulsherwood: what is the parameter in config/ybd.conf to use artifacts1.baserock.org? same as morph, artifac-cache-server?15:56
tiagogomesgary_perkins, your username is incorrect. Set an appropriate OPENSTACK_USER in the cluster15:57
pedroalvarezI don't think that is the problem15:59
gary_perkinstiagogomes: that's correct for DC, also it would have failed the credentials check if that was so15:59
pedroalvarezgary_perkins: sounds like a hack, but run ` pip install eventlet==0.17.4`15:59
pedroalvarezI need to fix this in baserock systems15:59
gary_perkinshttp://paste.baserock.org/ibiseqiwub16:00
gary_perkinsok, I'll try that, thanks pedroalvarez16:00
gary_perkinsalso, is it possible to add a new parameter to the image-create? as DC require a '--property architecture=amd64' so it will spin up a VM on the right arch16:02
*** wdutch has quit IRC16:02
*** wdutch has joined #baserock16:02
paulsherwoodjjardon: kbas-url: http://artifacts1.baserock.org:8000/16:03
jjardonpaulsherwood: Its working! thanks :)16:06
pedroalvarezpatch sent: https://gerrit.baserock.org/#/c/1391/16:10
paulsherwoodjjardon: great!16:14
*** mariaderidder has quit IRC16:16
*** mariaderidder has joined #baserock16:16
edcraggfor some reason i don't seem to be able to clone the bsp-support repo - http://paste.baserock.org/oweqefoyon16:20
tiagogomesedcragg, try `git clone git://git.baserock.org/baserock/baserock/bsp-support`16:21
edcraggtiagogomes: yes, i suspect that might work, but this is morph, using 'upstream:bsp-support'16:22
*** ctbruce has quit IRC16:23
straycatedcragg, do you not want baserock/baserock/bsp-support ?16:23
straycatso, baserock:bsp-support16:24
tiagogomesedcragg that is a badly constructed morph then, because 'upstream' is not part of the default alias16:24
edcraggoh, is that new?16:24
straycatedcragg, baserock is an alias that points to your trove16:24
straycatwell, yourtrove/baserock16:25
edcraggright, it has always worked before, i've never come across this 'baserock:'16:27
*** toscalix__ has quit IRC16:27
*** toscalix has joined #baserock16:28
straycatupstream is just an alias for yourtrove/delta, so if you somehow created delta/bsp-support on a trove you were using then it might have worked there16:28
edcraggi'm just using g.b.o, and always have been16:29
*** mariaderidder has quit IRC16:30
straycatthat's odd then, since there is no bsp-support in delta, since i created bsp-support on gbo16:30
straycatdoes it work if you switch to baserock:bsp-support?16:30
edcraggjust trying now16:32
edcraggERROR: Cannot find remote git repository: baserock:bsp-support16:33
SotKedcragg: can you paste your morph config somewhere?16:34
edcraggSotK: http://paste.baserock.org/zubuvobaye16:36
SotKo.O16:38
SotKis there anything useful in the morph log around the time of the error?16:39
*** persia_ has joined #baserock17:01
straycatedcragg, baserock:baserock/bsp-support isn't it, sorry17:11
ssam2baserock:baserock/bsp-support not baserock:bsp-support surely ?17:11
*** paulwaters_ has quit IRC17:13
edcraggSotK: http://paste.baserock.org/lajazixeki17:13
tiagogomesit should be baserock:baserock/bsp-support17:14
edcraggguess that's the wrong url, though17:14
edcraggyep17:14
edcraggstill confused why it worked before17:14
*** toscalix has quit IRC17:15
edcraggand there's a big precedent for 'upstream:' in definitions, 880 references, but 18 references to 'baserock:'17:15
edcragg'baserock:baserock/bsp-support' does seem to have worked, though17:18
edcraggthanks!17:19
tiagogomesupstream are used for projects which are under a delta prefix in the trove, confusing17:21
tiagogomesbaserock is set to git.baserock.org unless trove_host was given17:21
edcraggyep, i understand now :)17:27
edcraggare these things documented somewhere?17:27
tiagogomesmm, not sure17:29
tiagogomesnothing better as looking into the code :)17:29
ssam2the repo alias thing is a bit undocumented17:36
ssam2although it's basically the same idea as https://en.wikipedia.org/wiki/CURIE17:36
ssam2http://wiki.baserock.org/definitions/current/ should definitely document how this works17:44
ssam2but doesn't right now17:44
*** CTtpollard has quit IRC17:46
*** bashrc_ has quit IRC17:56
*** nowster has quit IRC18:07
*** ssam2 has quit IRC18:19
tiagogomesgrrr, can I resize the comment box on gerrit? I can't write anything useful there18:19
*** gary_perkins has quit IRC18:20
*** gary_perkins has joined #baserock18:21
edcraggtiagogomes: i can, by clicking and dragging the bottom right corner18:23
tiagogomesI can't do that. I am talking about that box with the -1,+1 radio button18:24
edcraggpedroalvarez: i have tested deployment for the remaining SoCFPGA dev kit definitions - https://gerrit.baserock.org/#/c/799/918:24
tiagogomesedcragg it did work?18:25
edcraggtiagogomes: yes, that one18:25
edcraggtiagogomes: partitioning you mean?18:25
tiagogomesyep18:25
edcraggyes :)18:25
edcraggit worked all along, all i had to do to get it to work with the merged partitioining extension was add the USE_PARTITONING=yes18:26
edcragghad to also fix some things in the systems which were already merged, too18:27
tiagogomescool18:32
*** tiagogomes has quit IRC18:58
*** edcragg has quit IRC19:23
*** paulwaters_ has joined #baserock20:16
*** paulwaters_ has quit IRC20:19

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!