| *** gtristan has quit IRC | 03:38 | |
| *** gtristan has joined #baserock | 04:10 | |
| *** toscalix has joined #baserock | 08:22 | |
| *** rdale has joined #baserock | 08:29 | |
| *** ctbruce has joined #baserock | 08:49 | |
| *** gtristan has quit IRC | 08:51 | |
| *** bashrc has joined #baserock | 09:05 | |
| *** edcragg has joined #baserock | 09:10 | |
| *** CTtpollard has quit IRC | 09:27 | |
| *** jonathanmaw has joined #baserock | 09:27 | |
| *** gtristan has joined #baserock | 09:35 | |
| *** gtristan has quit IRC | 09:35 | |
| *** gtristan_ has joined #baserock | 09:48 | |
| *** CTtpollard has joined #baserock | 09:57 | |
| *** ctbruce has quit IRC | 10:03 | |
| *** ctbruce has joined #baserock | 10:03 | |
| *** ssam2 has joined #baserock | 10:06 | |
| *** ChanServ sets mode: +v ssam2 | 10:06 | |
| *** ssam2 has quit IRC | 10:13 | |
| *** gtristan_ has quit IRC | 10:13 | |
| *** gtristan_ has joined #baserock | 10:14 | |
| *** gtristan_ is now known as gtristan | 10:14 | |
| paulsherwood | not much, i guess :) | 10:21 |
|---|---|---|
| mwilliams_ct | hi! I'm currently working on trying to use concourse to build a base-system. One of the problems we've had is that concourse only accepts branches as a reference to where work is, not tags or specific commits | 10:25 |
| mwilliams_ct | I tried to approximate this by using unpetrify-ref to equal branches, but this has failed as some chunks use tags as their unpetrify-ref | 10:25 |
| *** ssam2 has joined #baserock | 10:25 | |
| *** ChanServ sets mode: +v ssam2 | 10:25 | |
| mwilliams_ct | would there be interest here in making a branch for all currently tagged only chunks? My suspicion is not as it may not be practical to branch for everything gbo hosts | 10:26 |
| radiofree | fix concourse to use sha's/tags would be the better option | 10:26 |
| paulsherwood | do you have any idea how many unpetrify-refs are affected in the whole of master definitions? | 10:26 |
| paulsherwood | +1 to radiofree's suggestion, if concourse upstream will take it. seems like a bug there | 10:27 |
| mwilliams_ct | there are 15 in base-system alone so I'm guessing it's a large number though I dont know the exact figure | 10:27 |
| mwilliams_ct | I agree, we should try and get this upstream to concourse | 10:27 |
| *** Lachlan1975 has joined #baserock | 10:29 | |
| pedroalvarez | concourse maintainers may think: Why on earth do you want to track a sha1 on a CI pipeline? :) | 11:00 |
| mwilliams_ct | pedroalvarez: I've phrased it to them as that unlike smaller projects, always tracking HEAD can cause things to break quite easily | 11:01 |
| pedroalvarez | thing is.. if you have a pipeline configured to track a (or multiple) sha1, the pipeline is not going to be triggered ever by them. | 11:02 |
| pedroalvarez | so, why would you? | 11:02 |
| pedroalvarez | s/by them/by changes in them/ | 11:03 |
| pedroalvarez | I guess something in the pipeline tracks a real branch of definitions. In that case, is the concourse pipeline going to be re-configured for every change on this branch? | 11:04 |
| mwilliams_ct | pedroalvarez: you saw what I had yesterday right, the pipeline for base-system? The idea will be to have definitions feeding (with trigger: true) into a first job "parse-definitions" which will generate and deploy a pipeline | 11:06 |
| pedroalvarez | gotcha | 11:06 |
| pedroalvarez | makes sense | 11:07 |
| paulsherwood | pedroalvarez: 'the pipeline is not going to be triggered ever by them.' no. but things (dependencies) before them in the pipeline wil cause them to rebuild | 11:15 |
| mwilliams_ct | paulsherwood: what do you mean by "things (dependencies)" and "them"? | 11:16 |
| mwilliams_ct | (in Baserock terms) | 11:16 |
| pedroalvarez | I think they are concourse terms | 11:17 |
| paulsherwood | changes in, say, build-essential will cause foo to be rebuilt even if foo is locked to a tag | 11:18 |
| pedroalvarez | and is exactly what you have said, and what I have guessed | 11:18 |
| pedroalvarez | aah.. no, I was wrong then | 11:18 |
| paulsherwood | i was trying to clarify vs 11:02 <+pedroalvarez> thing is.. if you have a pipeline configured to track a (or multiple) sha1, the pipeline is not going to be triggered ever by them. | 11:18 |
| paulsherwood | i don't know if i've achieved that, though :) | 11:19 |
| pedroalvarez | yes and no. This is a very complex problem, and it's going to be full of implementation details | 11:19 |
| paulsherwood | he | 11:20 |
| paulsherwood | heh | 11:20 |
| paulsherwood | pedroalvarez: i heard elsewhere that storyboard kanban and email notifications are now working... is s.b.o updated with that functionality? | 11:27 |
| pedroalvarez | paulsherwood: of course :) | 11:28 |
| paulsherwood | oh, ok! so if i login i should be able to see it working? | 11:29 |
| pedroalvarez | yes | 11:29 |
| paulsherwood | pedroalvarez: maybe worth announcing this onlist? | 11:29 |
| pedroalvarez | That's a good idea | 11:31 |
| jjardon | Hi, can I have another review of https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/definitions+branch:master+topic:move_libxkbfile to save rebuilding the whole gnome stratum every time something changes in the x-generic stratum? | 11:31 |
| paulsherwood | pedroalvarez: lovely new colours :) | 11:32 |
| pedroalvarez | paulsherwood: I was going crazy when using openstack's and baserock's at the same time.. a different color was mandatory.. :) | 11:33 |
| paulsherwood | heh! | 11:33 |
| CTtpollard | cd .. | 11:34 |
| CTtpollard | oops | 11:34 |
| Zara | pedroalvarez: hahaha, that's why my dev instance is yellow! it got bad when I had four different instances open, all the same colour (mine, SotK's, s.o.o and baserock's)... | 11:35 |
| radiofree | is there a "clean up unpacked folders" after build option for ybd? | 11:44 |
| * radiofree only has a 50G ssd and the unpacked folders take up quite a bit of space | 11:45 | |
| paulsherwood | radiofree: rm -fr $artifacts/*/*-unpacked ? | 11:59 |
| radiofree | i have a script to do it but i was wondering if there was anything i could specify in the config | 12:00 |
| paulsherwood | i'd be delighted to accept a patch, or an issue :) | 12:07 |
| * radiofree might also patch checking the .unpacked folder to make sure everything in the manifest is actually in there before using it | 12:08 | |
| *** edcragg has quit IRC | 12:33 | |
| pedroalvarez | right, StoryBoard annouced in baserock-dev :) | 13:01 |
| *** edcragg has joined #baserock | 13:08 | |
| Zara | \o/ | 13:10 |
| paulsherwood | pedroalvarez: please can i be subscribed to that board? | 13:17 |
| pedroalvarez | paulsherwood: sure, you should be able to see it now in your Boards Dashboard: https://storyboard.baserock.org/#!/dashboard/boards | 13:20 |
| paulsherwood | pedroalvarez: thanks! | 13:21 |
| paulsherwood | interesting... so "morph: allow to not have build-depends" is in 'bugs', yet merged | 13:22 |
| pedroalvarez | paulsherwood: I just populated the board a bit with random tasks before sending the email | 13:23 |
| paulsherwood | aha :) | 13:23 |
| paulsherwood | makes sense now | 13:23 |
| pedroalvarez | I noticed when doing so that our storyboard could do with a bit of gardening | 13:24 |
| paulsherwood | maybe with the boards, we can do that better/easier | 13:25 |
| paulsherwood | rdale: i've been wondering about this splitting thing, wrt systems and strata | 13:36 |
| paulsherwood | currently ybd doesn't actually create a stratum artifact, because it's just a set of chunk artifacts untarred | 13:36 |
| paulsherwood | i wonder if we could *only* do splitting at deploy time? | 13:37 |
| paulsherwood | and/or system creation time | 13:37 |
| paulsherwood | ie not bother doing anything for indivual strata? | 13:38 |
| rdale | no, i think we should do a copy of the stratum as it is now but filter the built chunks according to the splitting rules, and then tar up the filtered version of the stratum | 13:39 |
| paulsherwood | oh ok, am i misremembering? does ybd actually create stratum artifacts? | 13:39 |
| rdale | step 1: create the stratum as now, step2: generate a set of meta file in the baserock dir of the stratum with a sub set of the chunk's artifacts, step 3: copy the stratum according to the lists of files for the chunks from the .meta files | 13:40 |
| rdale | i think ybd creates stratum artifacts, as they get cached | 13:41 |
| rdale | so i think the chunks should be filtered via the splitting rules before caching - i don't like the idea of doing everytime you deploy | 13:42 |
| paulsherwood | ok i'm interested to see it working :) | 13:43 |
| rdale | here is a .meta file example: http://paste.baserock.org/motimokame | 13:46 |
| rdale | not a good one as it's from stage1, so there is only one big list of files for the '-misc' artifact | 13:47 |
| rdale | when a file like that is copied to the /baserock dir for a stratum, the products whose artifact name don't match the splitting rules are dropped | 13:48 |
| paulsherwood | ack | 13:50 |
| *** jonathanmaw_ has joined #baserock | 13:56 | |
| *** jonathanmaw has quit IRC | 14:00 | |
| *** ssam2 has quit IRC | 14:13 | |
| *** ssam2 has joined #baserock | 14:20 | |
| *** ChanServ sets mode: +v ssam2 | 14:20 | |
| *** jonathanmaw has joined #baserock | 14:21 | |
| *** jonathanmaw_ has quit IRC | 14:25 | |
| jjardon | mason is failing: 2016-01-19 11:11:45 Progress: Transferring linux-x86-64-generic-misc to shared artifact cache | 14:30 |
| jjardon | 2016-01-19 11:11:56 Build of linux-x86-64-generic failed. | 14:30 |
| jjardon | out of space again? | 14:30 |
| *** gtristan has quit IRC | 14:45 | |
| *** ctbruce has quit IRC | 14:49 | |
| ssam2 | maybe, thanks for the heads up | 14:53 |
| pedroalvarez | I've already created some space | 14:55 |
| pedroalvarez | come on jjardon, I put a PROGRESS row in mason so that you can see what's going on right now :) | 14:55 |
| jjardon | can we have a timer that removes the oldest artifacts but the ones from releases, so we avoid this problem? | 14:59 |
| ssam2 | one day | 15:01 |
| ssam2 | it's not trivial, i did it once but it didn't get merged for some reason | 15:01 |
| ssam2 | i forget the details | 15:01 |
| pedroalvarez | removing chunk artifacts was causing some build fails, because the build tool was assuming that the existence of the stratum artifact meant all the chunk artifacts of that stratum existed | 15:04 |
| pedroalvarez | well, I think that was one of the reasons | 15:04 |
| paulsherwood | ybd copes with that :) | 15:04 |
| paulsherwood | and culls old artifacts :) | 15:05 |
| mwilliams_ct | historic question: at some point was unpetrify-ref required to be a branch? | 15:05 |
| mwilliams_ct | (a propos of nothing, was just curious) | 15:05 |
| paulsherwood | i think by convention it was originally, but users started setting tags because it made more sense in many cases | 15:06 |
| mwilliams_ct | that makes sense. lots of "baserock/morph" branches that look like they were older tags | 15:06 |
| jjardon | mwilliams_ct: I think so, yes | 15:06 |
| *** ratmice______ has quit IRC | 15:07 | |
| *** ratmice______ has joined #baserock | 15:07 | |
| jjardon | mwilliams_ct: morph files used to be in the chunk repo, so a branch was needed | 15:07 |
| mwilliams_ct | jjardon: oh, did not know that. thanks for the info | 15:10 |
| tiagogomes | we also use a baserock/morph branch when we needed to have some delta against upstream | 15:13 |
| pedroalvarez | s/use/used/ .. because now we prefer something like baserock/{{ nearest-tag || branch-name || something-else}} | 15:20 |
| ssam2 | baserock/upstream-version+description-of-what-the-change-was (which is *infinitely* better than naming everything baserock/morph !!) | 15:21 |
| pedroalvarez | :) | 15:22 |
| pedroalvarez | that makes unpetrify-ref less useless | 15:22 |
| paulsherwood | pedroalvarez: actually, i believe mwilliams_ct and others are using it :) | 15:23 |
| pedroalvarez | good it's still present :) | 15:24 |
| paulsherwood | :-) | 15:25 |
| jjardon | not sure they should: I've been told several times that parameter is only there for information purposes; also, there is not any mechanism to check the "ref:" and "unpetrify-ref:" are in sync | 15:29 |
| paulsherwood | jjardon: agreed. i believe they are only experimenting at this point | 15:30 |
| mwilliams_ct | jjardon: I discussed this a bit earlier. The problem is that Concourse doesnt support SHAs (or tags). I've spoken to their upstream about this (not had a reply due to timezones) and benbrown_ is looking into changes. for now, unpetrify-ref as an approximation for branch does what I need | 15:31 |
| pedroalvarez | hmm... | 15:33 |
| mwilliams_ct | as an aside, my deployment of concourse is giving me 500s on every resource it tries to fetch. have I upset g.b.o by trying to pull too much too quickly? | 15:33 |
| pedroalvarez | what if the build tool could build from unpetrify-ref if ref not present, and we had a command that would generate the ref field? | 15:34 |
| radiofree | why do we need that | 15:35 |
| radiofree | it would be more useful to re-add "petrify-ref" | 15:35 |
| pedroalvarez | you mean the `morph petrify` subcommand, right? | 15:35 |
| radiofree | that's the one | 15:36 |
| jjardon | +1000 | 15:37 |
| pedroalvarez | well, haven't used it in a while, but the petrify command will need some improvements to make it look right | 15:38 |
| pedroalvarez | isn't it going to move ref fields to unpetrify-refs, and then resolve the commit of whatever is in ref and rewrite ref? | 15:39 |
| radiofree | yes | 15:39 |
| pedroalvarez | putting a lot of sha1s in unpetrify-refs... | 15:39 |
| pedroalvarez | has anyone used it recently? | 15:39 |
| radiofree | i will need to at some point | 15:40 |
| paulsherwood | radiofree: why? | 15:40 |
| radiofree | i'm not going to manually change all the qt5 components from "ref: v5.5.1" to "ref: sha" manually | 15:40 |
| * paulsherwood still, after all these years, is not clear about the need for petrify | 15:40 | |
| pedroalvarez | I normally try to use sha1s directly when doing integration work, but I understand some people prefer not to | 15:41 |
| paulsherwood | but why do you/we need to do that, at all? | 15:41 |
| pedroalvarez | I guess that: it's easier; tags are for humans, sha1s for machines | 15:41 |
| jjardon | paulsherwood: you write in your definitions: I want to build tag "5.5.1" morph petrify will convert that tag in the corresponding sha and put the result in the "ref:" parameter | 15:42 |
| radiofree | pedroalvarez: paulsherwood: if you'd like to do that for http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/strata/qt5-tools.morph to point the ref at the v5.5.1 please feel free to do so | 15:42 |
| radiofree | to upgrade to qt 5.5.1, i change the ref to the v5.5.1 tag, build, test, fix, build, test fix, to submit the patches i need to change that ref to a sha | 15:42 |
| paulsherwood | no, i mean why do we need to force the ref: to be a sha... don't builds log the refs, and stick them in the .meta files? isn't that enough? | 15:43 |
| jjardon | the reason why baserock doesn't use tags directly is because reproducibility problems | 15:43 |
| radiofree | i'll use sams branch to do that, which takes seconds instead of god-knows how long to manually go through a dozen repos to manually get the sha | 15:43 |
| jjardon | (I've been told) | 15:44 |
| radiofree | paulsherwood: i'd be happy to leave it as a tag | 15:45 |
| radiofree | but i'm unsure about the default behaviour of trove/lorry | 15:45 |
| radiofree | imo it should never replace branches/tags that have been force pushed | 15:45 |
| radiofree | it should put them in a queue somewhere for human input ("what would you like to do here?")... | 15:45 |
| *** gtristan has joined #baserock | 15:47 | |
| pedroalvarez | we are having a lot of performance issues with http://git.baserock.org :/ | 15:47 |
| pedroalvarez | for example, the lc queue (http://git.baserock.org/lc-status.html) shouldn't be full of "now"s | 15:48 |
| radiofree | :( | 15:48 |
| radiofree | it was really slow this morning | 15:48 |
| pedroalvarez | it's using 15G out of 15G of ram | 15:54 |
| pedroalvarez | lighttpd process on 100% of cpu | 15:55 |
| pedroalvarez | various cgit.cgi processes... :/ | 15:56 |
| *** ctbruce has joined #baserock | 15:58 | |
| pedroalvarez | I feel like this is going to be the host provider, but haven't seen any performance issues in other VMs | 15:59 |
| mwilliams_ct | benbrown_ and I have noticed something interesting on gbo. seems that using the --branch <tag> command over http doesn't work whereas calling using git:// works fine. not a massive deal, I'll just change protocols, but just wondered why | 16:15 |
| mwilliams_ct | (to clone a repository at a given tag) | 16:16 |
| franred | mwilliams_ct, and https? | 16:19 |
| pedroalvarez | franred: https needs authentication | 16:19 |
| mwilliams_ct | franred: not tested. not for the first time today, I think the bug is actually in my code anyway | 16:19 |
| mwilliams_ct | stand down sysadmins, just mwilliams messing up again | 16:20 |
| pedroalvarez | _o_ | 16:20 |
| CTtpollard | I don't think it's a specific g.b.o issue | 16:21 |
| *** ssam2 has quit IRC | 17:59 | |
| *** bashrc has quit IRC | 18:02 | |
| *** jonathanmaw has quit IRC | 18:14 | |
| *** edcragg has quit IRC | 18:18 | |
| *** Lachlan1975 has quit IRC | 18:35 | |
| *** ctbruce has quit IRC | 22:00 | |
| *** edcragg has joined #baserock | 22:41 | |
| *** toscalix has quit IRC | 22:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!