IRC logs for #buildstream for Thursday, 2018-11-15

*** rdale has quit IRC00:08
*** xjuan has quit IRC01:16
*** tristan has joined #buildstream03:08
*** tristan has quit IRC04:33
*** ctolentino has quit IRC05:58
*** cs-shadow has quit IRC05:58
*** lchlan has quit IRC05:58
*** ctolentino has joined #buildstream05:58
*** cs-shadow has joined #buildstream05:58
*** lchlan has joined #buildstream05:58
*** alatiera_ has joined #buildstream08:12
Kinnisonjuergbi: It ought to be pretty simple to spot and eliminate workspace path prefixes from scanning08:58
juergbithat's true08:58
juergbiprobably makes sense to do this08:59
*** rdale has joined #buildstream09:16
*** toscalix has joined #buildstream09:24
gitlab-br-botvalentindavid opened (was WIP) MR !862 (valentindavid/link_files_sort_resolved->master: Resolve paths before ordering them in link_files/copy_files) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/86209:31
*** jonathanmaw has joined #buildstream09:54
tiagogomestpollard wat09:59
tpollardtiagogomes: you were asking if bst could have the option to build without trying to use artifact servers10:10
tiagogomesaah, missed the MR, thanks for that10:11
tpollardnp :)10:12
*** raoul has joined #buildstream10:21
*** lachlan has joined #buildstream10:27
*** tristan has joined #buildstream10:37
jennistiagogomes are you still around?10:54
tiagogomesyes10:55
jennisYou did a commit a while back: plugin: bake API to get and validate a project path10:55
jenniswhich basically ensures that any `path:` we've declared in a bst will need to be contained within the project10:56
jenniswas the intent of this to also restrict local junctions to be hosted within the project? Because I don't think I agree with this10:56
*** toscalix has quit IRC10:57
tiagogomesIt was. I have committed a change to freedesktop-sdk for that10:58
jennisFor example, I have the freedesktop-junction.bst in my gnome-build-meta project, now instead of junctioning to the stable version on git, I want to junction locally to the version I have. And providing the absolute path to this project throws this exception10:58
jennisI also noticed the change preventing '../path/to/somewhere'10:58
jennisI don't see why we're restricting users to keep junctions (other BuildStream projects) inside of their current project?10:59
*** alatiera__ has joined #buildstream11:00
tiagogomesHow can you ensure reproducibility if you are using the content of random directories in your host file system11:00
*** alatiera_ has quit IRC11:01
*** toscalix has joined #buildstream11:01
KinnisonYou can't, but you may need to do the latter without care for the former11:01
tiagogomesjennis can't you still use a junction with a git source, but this file passing a filepath in "url:"?11:04
tiagogomesThat should work11:05
tiagogomesYou'd still need to have to commit the changes in your clone of freedesktop-sdk, but you don't need to push them11:06
jennisHeh, I could, but that's not the point11:07
jennisIt's more convenient for me as a user to just use my local version, and make changes as and when without having to push to a branch11:08
jennisI understand both arguments and it'd be nice if more people would weigh in with their opinion11:09
jenniss/would/could11:09
*** brlogger has joined #buildstream11:15
tiagogomesjennis tbc, you don't need to push them to a branch. You just need create a commit with the changes11:16
jennisAnd then specify a file url to the junction'd git project?11:19
jennisThis isn't the point I'm making though, the point is whether we should be this restrictive :p11:20
tiagogomesFile an issue then :)11:21
jennisWill do11:25
abderrahim[m]jennis I think you want to use a workspace11:25
jennisabderrahim[m], yes this also would work, thanks11:31
toscalixjennis: maybe the ML help you to get input to define the use case11:34
juergbitpollard: I'm a bit confused about the CLI option naming in !786. it's well possible that I missed a comment about this but in #494 I see --pull-buildtrees, not --pull-build-trees, and in !786 you comment that you're not completely happy with --pull-build-trees. what was the reason to use --pull-build-trees instead of --pull-buildtrees in the first place?12:15
WSalmontlater[m], ctolentino has just posted https://mail.gnome.org/archives/buildstream-list/2018-November/msg00038.html to the ML, the last paragraph talks about he cools ideas we were discussing about dealing with test and reusing build trees etc12:16
juergbitpollard: I'm starting to think --pull-buildtrees would make most sense. as treating buildtree as single word sounds sensible enough and we already use 'buildtree' e.g. as subdirectory name (that's not user-visible, though)12:16
juergbi(don't really want to start bikeshedding. main point is consistency, see my latest comment on the MR)12:16
*** gitlab-br-bot has joined #buildstream12:20
tpollardI got the impression that build-trees were preferred as separate words12:25
tpollardI can change it12:25
tpollardjuergbi: to match https://gitlab.com/BuildStream/buildstream/merge_requests/786#note_11376955912:27
juergbitpollard: however, before that, tristan proposed or at least mentioned --pull-buildtrees, though. I think in the MR comment it was more about using _ as separator where we want separation12:30
juergbitristan: if you happen to be around: quick question, --pull-buildtrees or --pull-build-trees as CLI option (and identical spelling for userconfig, of course)?12:31
tpollardmy preference is --pull-buildtrees12:35
*** alatiera__ has quit IRC12:41
juergbimine too. jonathanmaw also mentioned unhappiness with "pullbuildtrees"13:00
tpollardI've done a rework locally, will check everything still passes in the meantime13:01
juergbigreat13:01
tiagogomesI also prefer --pull-buildtrees13:01
tpollardI left a reply to your comment re setting a default value of something parsed from userconf data13:01
tpollardoh, got the reply just now...13:02
juergbiI'll make a quick branch for the quota thing to avoid future confusion13:03
tpollardcool, I'll drop the default13:05
*** lachlan has quit IRC13:13
*** ChanServ sets mode: +o tristan13:16
tristanjuergbi, tpollard --pull-buildtrees is fine with me... for user config I'm a bit concerned about grouping13:16
juergbiyes, I just commented about this13:16
juergbiof the existing groups, 'cache' might fit but not sure13:16
juergbior we could create a new group but don't have a concrete idea13:16
tristanmyeah, is it 'cache' or is it 'artifacts' ?13:17
tristanI guess it must be 'cache'13:17
*** alatiera__ has joined #buildstream13:20
juergbitristan: yes, artifacts is a list, not a group13:20
juergbiI think cache is a reasonable fit13:21
tristanjuergbi, sounds good to me13:21
tpollardok13:23
gitlab-br-botjuergbi opened MR !953 (juerg/context-default-values->master: _context.py: Drop duplicated default values for user configuration) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/95313:25
*** alatiera_ has joined #buildstream13:34
*** jonathanmaw has quit IRC14:00
*** jonathanmaw has joined #buildstream14:00
*** finn_ has joined #buildstream14:00
*** phildawson_ has quit IRC14:01
*** phildawson_ has joined #buildstream14:01
*** finn has quit IRC14:02
*** tristan has quit IRC14:06
*** alatiera_ has quit IRC14:12
*** tristan has joined #buildstream14:27
* paulsherwood wonders what could lead to "pipeline: failed" on main badges at https://gitlab.com/BuildStream/buildstream14:34
Kinnison    OSError: [Errno 30] Read-only file system: 'debugedit-disable-md2.patch'14:37
KinnisonLooks like the runner had a snafu14:37
KinnisonI think valentind and possibly jjardon are aware and on it14:38
juergbithere is also the issue that we have two different CI pipelines for the same commit (normal and overnight) and the badge simply shows the latest pipeline14:39
juergbithe overnight one tests bst against fdo sdk and that's the one that failed14:40
valentindSomething wrong with one builder.14:40
valentindAh, it is the filesystem of the builder that gets back in readonly.14:40
valentindIt happened already.14:40
valentindWhen operations fix crash usually it remounts read only.,14:40
valentindI will ask Dave.14:42
valentindRebooted the machine. Now it is mounted rw. We will see if it passes this night.14:44
*** lachlan has joined #buildstream14:47
jjardonvalentind: FYI, you can trigger the schelude jobs manually at any point (simply press "play" in the scheludes page). I have triggered https://gitlab.com/BuildStream/buildstream/pipelines/36738166 and https://gitlab.com/BuildStream/buildstream/pipelines/36738158 to check if it's fixed now14:55
valentindjjardon, There is only one builder for the moment. So I suppose one of them will just timeout.14:56
jjardonvalentind: I though buildstream used to have 2 for the overnight tests?14:57
valentindjjardon, Yes, but we are still waiting for operations to split the storage. So one is down because it does not have space.14:58
jjardonvalentind: let's setup something in DO, so we do not need to wait? It would take 10 min maximum14:59
valentindWould freedesktop sdk need more resource than a the builders provided by the bastion?15:04
jjardonvalentind: freedesktop-sdk doesnt share any infra with buildstream15:05
valentindjjardon, builstream has builders on DO.15:05
valentindI am talking about those.15:05
jjardonvalentind: ah sorry15:11
jjardonmisunderstood15:11
*** alatiera_ has joined #buildstream15:11
jjardonyeah, so we create a new bastion that will generate runners with the new spec15:11
jjardonvalentind: I have already done: see buildstream-bastion-overnight at https://gitlab.com/BuildStream/buildstream/settings/ci_cd#js-runners-settings (It's paused at the moment)15:12
*** alatiera_ has joined #buildstream15:12
jjardonvalentind: if you are ok, active it and give it a try15:12
valentindjjardon, let me look at the configuration.15:13
jjardonvalentind: It will generate machines with 4 CPU, 8GB of RAM and 160 GB of hard drive: that should be enough to build freedesktop-sdk15:14
gitlab-br-botcoldtom opened issue #767 (Yaml Dumper removes blank lines seemingly arbitrarily) on buildstream https://gitlab.com/BuildStream/buildstream/issues/76715:14
valentindjjardon, OK15:15
valentindWe need to tag it15:15
valentindOh it is.15:15
valentindGood15:15
valentindjjardon, You can start it.15:15
jjardoncool, sec15:16
valentindOh gitlab-runner does not clean correctly docker container. There are also failures on the old builder due to that.15:17
valentindI have a fix for the docker issues on the arm builders. I will apply it tomorrow.15:24
jjardonvalentind: new runner seems to work well: https://gitlab.com/BuildStream/buildstream/-/jobs/12105796815:30
jjardonvalentind: I have paused the codethink ones (codethink-x86_64-1 and codethink-x86_64-2), feel free to tell Codethink ops to remove them when you think new ones are tested enough15:31
valentindWhat do we do with them then?15:33
jjardonvalentind: with codethink-x86_64-1 and codethink-x86_64-2? nothing; they will take control back of them and use them for another stuff15:36
valentindok15:36
toscalixvalentind: please recod the deprecation in the tickets15:49
*** alatiera_ has quit IRC15:50
*** alatiera_ has joined #buildstream15:54
*** alatiera_ has quit IRC15:57
* jjardon updates https://wiki.gnome.org/Projects/BuildStream/Infrastructure15:57
jjardonvalentind: are  codethink-x86_64-1 and codethink-x86_64-2 gone already or you prefer to keept them for a while?15:58
gitlab-br-botjjardon closed issue #763 (Overnigth tests are failing) on buildstream https://gitlab.com/BuildStream/buildstream/issues/76316:01
jjardontoscalix: what ticket number, please?16:05
toscalixchecking16:05
toscalixjjardon: there is none16:09
toscalixI have on my todo to create the subgroup for infra and move this repo there: https://gitlab.com/BuildStream/sled-ansible-config16:10
toscalixwe will add the tickets there16:10
*** alatiera_ has joined #buildstream16:13
*** rdale has quit IRC16:19
*** rdale has joined #buildstream16:23
coldtomare there plans  in the works to upgrade the ruamel.yaml dependency? currently running `bst track` will "unfold" folded blocks (using >)16:31
*** alatiera_ has quit IRC16:36
*** alatiera_ has joined #buildstream16:36
*** jonathanmaw has quit IRC16:53
*** lachlan has quit IRC16:54
*** lachlan has joined #buildstream16:55
lachlanHi, I have a WIP branch for benchmarking that I would like to have reviewed for the broad approach being taken.16:55
gitlab-br-botcoldtom opened MR !955 (coldtom/bst-fmt->master: bst fmt: Add basic functionality) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/95516:55
gitlab-br-botjennis opened MR !956 (jennis/less_restrictive_pathways->master: We shouldn't fail on an absolute path to our project) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/95616:55
jennisA nice 'end of the day and I'm tired' patch up for review there ^^17:01
jennisWould recommend commit-by-commit as the total changes looks messy xD17:02
lachlanMR !9 (lachlanmackenzie/AddGraphingOfResults->master: WIP: Add runner to get and correlate results for plotting ) on buildstream/benchmarking https://gitlab.com/BuildStream/benchmarks/merge_requests/917:08
gitlab-br-botMR !9: Leeming/sandbox only https://gitlab.com/BuildStream/buildstream/merge_requests/917:08
*** toscalix has quit IRC17:10
tpollardjuergbi: I should have everything pushed now bar not having the directory logic in the pullqueue17:10
juergbigreat17:11
*** MOLLET6719 has joined #buildstream17:11
*** MOLLET6719 has left #buildstream17:11
juergbitpollard: does the comment about the directory logic move make sense to you?17:11
tpollardjuergbi: vaguely17:16
tpollardnow that it's in global context and not an option passed to the queue from stream, I should be able to reproduce the same logic within element?17:17
gitlab-br-botjjardon opened issue #768 (Overnigth tests are failing for bst-1.2 branch) on buildstream https://gitlab.com/BuildStream/buildstream/issues/76817:18
tpollardI'm trying to think if it will have ramifications in the future (one being an attempted partial pushed spawning a pull queue for the buildtree)17:22
juergbitpollard: yes. if the partial pull will go via pullqueue, Element should likely grow a method such as _pull_buildtree()17:26
jennisjuergbi, thanks for the review, I've replied17:30
*** abderrahim has joined #buildstream17:31
*** abderrahim4 has quit IRC17:32
*** finn_ has quit IRC17:53
*** alatiera_ has quit IRC18:00
juergbijennis: btw: doesn't this sound like you want a workspace for the junction? don't we support this?18:04
*** lachlan has quit IRC18:37
gitlab-br-botjuergbi approved MR !943 (aevri/unit_tests->master: contributing: add guidance on unit tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94319:48
*** ctolentino has quit IRC20:05
gitlab-br-botjuergbi merged MR !943 (aevri/unit_tests->master: contributing: add guidance on unit tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94320:25
*** tristan has quit IRC20:30

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!