IRC logs for #buildstream for Monday, 2019-02-11

gitlab-br-botjuergbi closed issue #662 (No warning produced for import element with configure-commands set) on buildstream https://gitlab.com/BuildStream/buildstream/issues/66205:10
gitlab-br-botjuergbi merged MR !1141 (juerg/import->master: import.py: Validate config node) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/114105:10
*** nimish has quit IRC06:39
*** alatiera has joined #buildstream07:01
gitlab-br-botjuergbi merged MR !1138 (juerg/symlinks->master: Symlink fixes) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/113807:13
gitlab-br-botjuergbi opened (was WIP) MR !1139 (juerg/list-all-directories->master: Return all directories in list_relative_paths()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/113907:17
*** nimish has joined #buildstream07:56
*** solid_black has joined #buildstream08:55
*** solid_black has quit IRC09:06
*** solid_black has joined #buildstream09:06
gitlab-br-botjuergbi opened MR !1142 (juerg/buffer-size->master: Increase read buffer size to improve performance) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/114209:12
gitlab-br-botjuergbi merged MR !1129 (snakeviz->master: contributing: snakeviz replaces pyflame+flamegraph) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/112909:20
*** jonathanmaw has joined #buildstream09:54
*** benschubert has joined #buildstream09:54
*** tpollard has joined #buildstream10:07
*** raoul has joined #buildstream10:09
*** rdale has joined #buildstream10:14
*** lachlan has joined #buildstream10:38
*** lachlan has quit IRC10:40
*** jonathanmaw_ has joined #buildstream10:41
*** jonathanmaw has quit IRC10:42
jjardonHi, does buildstream releases require anything else than create a tag, create the tarball and put it at https://download.gnome.org/sources/BuildStream/?10:42
juergbijjardon: if man-pages are in sync, that should be pretty much it10:47
juergbi(+ release mail)10:47
*** lachlan has joined #buildstream10:57
*** lachlan has quit IRC11:08
*** lachlan has joined #buildstream11:08
jjardonI wonder if we can help on release another 1.2.x; we would like to use the latest in that branch in GNOME11:31
jjardonwe are about to enter the hard code freeze soon so It would be nice to align if possible11:32
juergbiI see, are further backports pending that you'd want in that release, or is it really just about a release?11:34
juergbiI'm not sure when exactly tristan is travelling, and whether he had planned to include further commits or just spin a release11:35
lachlanHi, after some sleuthing from tlater[m] it seems that the buildstream/buildstream-fedora image and that it needs rebuilding - can this to be arranged?11:47
tlater[m]lachlan: I believe this has already happened11:48
tlater[m]Is there still trouble with the benchmarks?11:48
lachlanIt seems to be still broken, yes11:48
lachlanSane error11:48
lachlan* Same11:48
tlater[m]Hmm, I'll try to take a look this evening. We are installing the correct packages, so that shouldn't happen. Maybe the tag wasn't assigned correctly?11:49
jjardonjuergbi: just about a new release; the bst-1.2 branch has already several fixes interesting for us11:49
jjardonlachlan: what is the problem with the image?11:50
lachlanjjardon: gcc fails11:51
jjardonlachlan: can you paste the logs somewhere, please?11:51
jjardonCan I have reviews of https://gitlab.com/BuildStream/buildstream-docker-images/merge_requests/102 , please?11:52
lachlanI will post the link to the buildstream build (but I have just done it locally)11:52
lachlanhttps://gitlab.com/BuildStream/benchmarks/-/jobs/15919538511:52
tlater[m]jjardon (@_gimpnet_jjardon:matrix.org): It looks like the gcc-c++ and python-devel packages aren't installed.11:53
juergbiwell, but shouldn't pyroaring be pre-installed?11:54
jjardonlachlan: can you as a test, install those packages manually in the .gitlab-ci.yml, to be sure that is the problem?11:54
jjardonjuergbi: yeah, I was just thinking about that11:54
tlater[m]jjardon (@_gimpnet_jjardon:matrix.org): I have already tested exactly that :)11:54
jjardontlater[m]: ah ok, sorry :)11:54
jjardontlater[m]: builstream-docker-images has changed quite a bit; are we sure we have pyroaring there now?11:55
tlater[m]My guess originally was that the image failed to build and wasn't uploaded because the pipeline failed.11:55
jjardonyeah11:57
jjardonthe latest x86_64 available is old: https://hub.docker.com/r/buildstream/buildstream-fedora/tags11:57
jjardon(23 days old)11:57
tlater[m]Right11:57
tlater[m]Odd, I set it to rebuild and it had been rebuild yesterday.11:57
tlater[m]Well, maybe I didn't check properly11:57
jjardonsomehow is not being pushed11:58
* jjardon tries to check in the logs11:58
jjardonlachlan: what docker image version are you using?11:58
jjardonoh11:59
lachlansha of base image is - a7cc9c40301b8bedaefef2485ee5473495b2a161e4c06492871bd6aa7a36aa7512:00
jjardonseems we are not pusing to https://hub.docker.com anymore, but the gitlab registry? https://gitlab.com/BuildStream/buildstream-docker-images/-/jobs/15924588512:00
jjardonhere: https://gitlab.com/BuildStream/buildstream-docker-images/container_registry12:01
tlater[m]We were going to deprecate buildstream fedora12:01
tlater[m]Perhaps that's already happened and snuck by me.12:02
jjardonmm, or maybe no; let me check .gitlab-ci12:02
jjardontlater[m]: are we?12:02
jjardonwhat is the replacement?12:02
tlater[m]jjardon (@_gimpnet_jjardon:matrix.org): A plain "buildstream" image12:02
tlater[m]With tags for different distros12:02
jjardoncool12:02
tlater[m]And a "latest" tag that points to the 1.2 release.12:03
jjardonDoes that exist yet?12:03
tlater[m]I don't think it does12:03
tlater[m]But I've been a bit less attentive lately.12:03
tlater[m]<_gimpnet_lac "sha of base image is - a7cc9c403"> lachlan: On a side note, I think the benchmarks repo should start using a testsuite image anyway.12:04
tlater[m]Because the buildstream-fedora image comes with buildstream installed12:04
tlater[m]So we can't be 100% certain that we're running the correct package12:05
tlater[m]Pip is a little non-deterministic with that.12:05
jjardonlachlan: can you try with "latest" ?12:05
lachlantlater[m] It should be something to consider certainly.12:05
jjardonimage: buildstream/buildstream-fedora:latest12:05
lachlanjjardon: That is the tag that we use when we download the image12:06
jjardonlachlan: ugh, I'd avoid doing that: you can break your CI if you do that (as it seems already happened)12:07
tlater[m]Tbf, in this case the breakage occurred because of a new buildstream requirement. Not because the image changed.12:08
tlater[m]It broke because the image didn't change.12:08
jjardonsure, I think my recommendation still applies though12:10
jjardonso the strange thins is that "latest" is newer that the latest tag here: https://hub.docker.com/r/buildstream/buildstream-fedora/tags?page=112:10
jjardonso seems that the repo is not tagget anymore when we push a new image12:10
laurencedoes anyone have a link to the google docs of the notes from the gathering?12:22
juergbilaurence: https://bit.ly/2019-gathering-notes12:26
jjardonah, of course12:27
jjardontlater[m]: lachlan buildstream-fedora is for end users, so It installs buildstream 1.2 (that new dependency is for master only)12:28
laurencejjardon, thanks12:29
tlater[m]<_gimpnet_jja "ah, of course"> jjardon (@_gimpnet_jjardon:matrix.org): Right, that makes sense. So the change I expected half-happened.12:34
tlater[m]lachlan: That means the benchmarks repository must switch to a testsuite image.12:34
tlater[m]This one's probably closest to the current image: https://hub.docker.com/r/buildstream/testsuite-fedora12:35
* tlater[m] hurries to a different building for his next lecture12:35
lachlantlater[m]: Thanks will have a look later12:36
*** raoul has quit IRC12:36
gitlab-br-botLaurenceUrhegyi opened issue #901 (Remove BuildStream version 1.2 form Debian Buster packaging) on buildstream https://gitlab.com/BuildStream/buildstream/issues/90112:42
laurenceRegarding removing 1.2 from Debian packaging, what's the best approach? Is there anything in that ticket I've missed off?12:45
laurencejjardon, ?12:45
jjardonI think more that telling each maintainer of each distro to remove a package, the real fix is to fix buildstream docs to clearly state 1.2.x is not recommended for general consumption, and It's going to be deprecated soon12:46
jjardonthen is distros jobs to decide12:47
jjardonwe have quite a lot of distros packaging bst already: https://buildstream.build/install.html12:47
laurenceyeah, that's what made me think of this12:48
laurencejjardon, that makes sense to me, I'll update the ticket12:49
*** tristan has joined #buildstream12:51
*** nimish has quit IRC13:14
*** nimish has joined #buildstream13:14
*** tpollard has quit IRC13:26
*** tpollard has joined #buildstream13:28
*** tristan has quit IRC13:30
*** raoul has joined #buildstream13:33
benschubertBUG: Errno 31, too many links while doing a bst build debian-stack... does someone has a clue?13:41
KinnisonSounds like you're on a filesystem which limits hard links13:41
KinnisonCurrently we link into the CASCache for getting element metadata13:42
Kinnisonif you have too much cached, it'll splode with NTFS13:42
Kinnisonjonathanmaw_: You did some work to mitigate this, could you explain how to benschubert ?13:42
jonathanmaw_Kinnison: yep, I'll dig up the details13:43
jonathanmaw_benschubert: I encountered that when running the profiling suite that Kinnison provided, and outlined how I mitigated it in https://mail.gnome.org/archives/buildstream-list/2019-January/msg00041.html13:45
jonathanmaw_i.e. have a separate disk, formatted ext4; use ext2fsd to mount it in Windows; mount it into WSL; edit ~/.config/buildstream.conf to have builddir, artifactidr, etc. on the external disk13:46
jonathanmaw_s/external/separate/13:46
jonathanmaw_(with the added complication that I couldn't get it to properly use the ext4 disk as non-root, so I had to install buildstream as root)13:47
benschubertAh jonathanmaw_ , Kinnison ah, missing information: I'm running it on a RHEL machine :)13:48
KinnisonCould still be a similar issue13:48
KinnisonJust with larger numbers13:48
Kinnisone.g. debian-stack has over 75,000 elements13:49
KinnisonAnd it's possible max-links is 6553613:49
gitlab-br-botjuergbi merged MR !891 (aevri/include-error->master: More user-friendly reporting on include errors) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/89113:52
*** nielsdg has joined #buildstream13:56
nielsdgHi everyone, I'm using a fresh install of buildstream (using pip3) and am encountering the following error13:57
nielsdg> AttributeError: 'Repo' object has no attribute 'reload_config'13:57
nielsdgFull log for reference: https://paste.gnome.org/p0iwoqmgn13:58
benschubertKinnison: it's maximum for ext3 correct? not ext413:58
nielsdgAnyone who has seen something similar?13:58
nielsdgPython version is also 3.5.313:58
Kinnisonbenschubert: not sure13:58
*** raoul has quit IRC14:01
*** phildawson has quit IRC14:02
*** phildawson has joined #buildstream14:04
jjardonnielsdg: are you using buildstream master or the current release (1.2)?14:05
*** rdale has quit IRC14:05
jjardonnielsdg: If you are trying to build GNOME only 1.2 is supported14:05
nielsdgjjardon: Installed from pip, `bst --version` returns 1.2.314:06
nielsdgI assume 1.2.x is also supported?14:07
jjardonnielsdg: yep14:07
jjardonnielsdg: mmm, that should work; what SHA from gnome-build-meta are you trying to build? I will try to reproduce here14:08
nielsdglatest git master of gnome-build-meta, so 23c7e2d1d774f234411fa9452c009f8a239adc6414:08
jjardonnielsdg: do you have the bst-external plugins installed as well?14:10
Kinnisonbenschubert: try:14:11
nielsdgjjardon: yes, as outlined in its README (git clone and then pip install)14:11
Kinnisonbenschubert: find ~/.cache/buildstream/artifacts/extract -type f -links +1000014:11
Kinnisonbenschubert: does that show lots of things?  if so, boost the 10000 to say 6500014:11
Kinnisonbenschubert: if it still shows lots of things, then it's probably the extract hardlinking problem14:11
nielsdgjjardon: it gave the error `The 'buildstream-external' distribution was not found and is required by the application` otherwise14:11
jjardonnielsdg: do you have the ostree pygobject binding installed?14:12
jjardonnielsdg: gir1.2-ostree-1.0 if you use debian14:13
jjardonnielsdg: ostree and python3-gobject packages if you use Fedora14:14
nielsdgjjardon: Apparently: `gir1.2-ostree-1.0 is already the newest version (2016.15-5)`14:14
nielsdgjjardon: Don't know if the version might be too old (a14:14
nielsdg(I assume it might be that the specific property was only added in a later version)14:15
jjardonmmm, not sure if we have any minimum version for that, let me check14:16
alatieranielsdg: what version of bst-external do you have installed?14:16
gitlab-br-botjonathanmaw opened issue #902 (Element._update_state() does more stuff than it needs to when it's called) on buildstream https://gitlab.com/BuildStream/buildstream/issues/90214:16
jjardonnielsdg: in the meantime, can you install the one from stretch-backports ?14:16
jjardonnielsdg: ah14:17
jjardonlibostree >= v2017.8 with introspection data14:17
nielsdgalatiera: pip says 0.9.014:17
jjardonthat was a requirement that seems got lost14:17
nielsdgjjardon: Thanks! I'll get a newer version and check if it works14:18
jjardonnielsdg: see the instructions here: https://buildstream.build/source_install.html#stretch for Stretch14:19
jjardon(you need to activate backports)14:19
jjardonCan I have reviews of https://gitlab.com/BuildStream/website/merge_requests/112, please?14:27
benschubertKinnison: 65000 doesn't show anything, 64500 shows a lot of stuff :)14:29
Kinnisonbenschubert: sounds like a nlinks limit of some kind14:32
Kinnisonbenschubert: unless someone has changed it since, I googlefound: (found in same ext4.h): /* Maximal count of links to a file */ #define EXT4_LINK_MAX 6500014:33
benschubertlink _to_ a file? Why are we having 65k links to the same file Oo14:33
KinnisonBecause that's how we check stuff out of the CASCache14:34
Kinnisonwe just link() to it14:34
benschubertso the same file would be linked out 65k time? how?14:35
benschubertI mean, I understand we would check it out N times, with N the number of builders, but am I missing something?14:40
juergbiit can happen if tons of packages contain identical files14:40
juergbie.g., empty or dummy test files14:41
benschubertoh and the debian dummy project uses the same file all the time. Got it14:41
*** rdale has joined #buildstream14:41
*** phildawson has quit IRC14:42
*** raoul has joined #buildstream14:42
Kinnisonbenschubert: worse, we do the same for the metadata files14:42
KinnisonAnd every successful build has a build-result.yaml with the same content: "description: succeeded\nsuccess: true\n"14:42
Kinnisonthassit14:43
Kinnison/o\14:43
benschubertIs there an issue for this?14:43
KinnisonDunno, jonathanmaw_ did you make one?14:43
* Kinnison is hoping artifactasaproto will correct some of this (by lifting the duplicated metadata)14:43
juergbiI'm also planning on getting rid of extract directories, reducing hardlinks14:44
jonathanmaw_Kinnison: I don't think I did14:44
juergbiand with FUSE we shouldn't need any hardlinks anymore14:44
*** phildawson has joined #buildstream14:45
benschubertjuergbi: is there a time frame for this?14:46
juergbigetting rid of extract directories: I was planning on doing it shortly after switching to BuildBox, but I'm wondering whether maybe we should already do it before14:49
juergbiI'll likely be working a bit on CAS-optimized code paths soon, maybe I can do it as part of that14:49
gitlab-br-botjjardon opened MR !1143 (jjardon/distros->master: README.rst: Add table with distros with packaged buildstream) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/114314:49
jjardonnielsdg: any luck?14:50
*** jonathanmaw_ is now known as jonathanmaw14:57
nielsdgjjardon: Seems to work now. Thanks :-)15:04
jjardoncool :)15:05
gitlab-br-botjjardon opened issue #903 (buildstream should inmediatelly fail if ostree < v2017.8 is installed) on buildstream https://gitlab.com/BuildStream/buildstream/issues/90315:10
gitlab-br-botwillsalmon opened issue #904 (wrong sugestion) on buildstream https://gitlab.com/BuildStream/buildstream/issues/90415:20
*** johnward has left #buildstream15:41
jjardonwhat is test-wsl? Seems it can only handle 1 concurrent build, which can be a problem when several branches are running: https://gitlab.com/BuildStream/buildstream/pipelines?scope=running&page=116:04
*** lachlan has quit IRC16:04
jonathanmawjjardon: there should be two concurrent workers at the moment. I'm looking into provisioning some more runners.16:05
jonathanmawjjardon: test-wsl is a test job that checks that we're not breaking functionality for users that are running buildstream from WSL16:07
*** lachlan has joined #buildstream16:07
jjardonjonathanmaw: can you please add the details about this new runner at https://wiki.gnome.org/Projects/BuildStream/Infrastructure ?16:09
juergbijonathanmaw: I see only a single WSL runner online16:10
juergbithe 4 other WSL runners are offline16:10
jonathanmawjuergbi: aha, one of the runners is having network troubles16:12
*** lachlan has quit IRC16:25
*** lachlan has joined #buildstream16:30
*** raoul_ has joined #buildstream16:33
*** raoul has quit IRC16:34
*** raoul_ has quit IRC16:40
*** raoul_ has joined #buildstream17:07
tpollardadds68: looking forward to the MR to s/BuildStream/Buildstream ;)17:22
adds68Yes, we should do that!17:22
nielsdgIf I want to have access to systemd/systemctl inside a bst shell, how would I do that?17:23
nielsdgIs it possible?17:23
nielsdgRight now, I'm just getting "System has not been booted with systemd as init system (PID 1). Can't operate"17:25
juergbiadds68: I was about to mention this. inconsistent spelling of the project name is pretty bad, in my opinion. we need to change one or the other before using the logo anywhere17:27
juergbinielsdg: not sure whether we could easily enable this for non-isolated shell. what's your use case?17:28
juergbiif it's about privileged operations such as starting/stopping services, I don't think we should even attempt to support this17:29
adds68juergbi, yes i could raise an issue for people to discuss this, however i dislike BuildSream(BS)17:30
laurenceChanging the name of the project to Buildstream would require a lot of work17:32
laurenceneedless work17:32
laurenceIt should be kept as BuildStream, and used properly and consistently.17:33
adds68Lets open a ticket17:34
laurencedid anyone else get two emails from juergbi on the ML just now?17:37
*** lachlan has quit IRC17:38
gitlab-br-botadds68 opened issue #905 (Change the project logo away from CamelCase to Pascalcase) on buildstream https://gitlab.com/BuildStream/buildstream/issues/90517:39
jjardonlaurence: Seems there are not recent messages from juergbi : https://mail.gnome.org/archives/buildstream-list/2019-February/thread.html17:40
jjardonsorry: https://mail.gnome.org/archives/buildstream-list/2019-February/date.html17:40
*** raoul_ has quit IRC17:42
laurenceI am talking about 'Re: [BuildStream] Artifact as a Proto plan '17:42
juergbilaurence: I also got my own mail twice via ML, but I sent it only once and it's only once in the archive *shrug*17:42
laurencejuergbi, ah, ok, no probs17:42
valentindOn master there is import of "pyroaring".17:43
laurenceat least it's not just me...17:43
valentindWhat is this?17:43
juergbiroaring bitmap library, optimization to reduce memory usage17:43
valentindjuergbi, is not it missing from requirements?17:44
valentindI would expect tox can download it automatically.17:44
valentindBut it does not17:44
juergbino, it should be there17:44
juergbido you see the issue with tox?17:44
valentindStrange17:44
juergbiif tox issue, try: tox --recreate17:44
juergbipip3 install should also cover it for non-tox use17:45
valentindjuergbi, OK thank you. Did not know about it.17:45
valentindI thought tox was smart enough to not need that.17:46
*** lachlan has joined #buildstream17:47
*** raoul_ has joined #buildstream17:48
gitlab-br-botvalentindavid opened MR !1144 (valentindavid/pull-chmod-bug->master: buildstream/_artifactcache/cascache.py: Set 0644 rights to pulled files) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/114417:54
juergbijonathanmaw: do you know if the WSL runner is stuck? https://gitlab.com/BuildStream/buildstream/-/jobs/159524820 seems to be the runner's active job but still initializing after 2 hours according to the log18:14
juergbiother MRs in queue for nearly 4 hours18:15
juergbiwe may want to temporarily make the test non-gating until we sort out the runner situation18:15
jonathanmawjuergbi: looked to be a network problem with that runner18:25
jonathanmawI've taken it down since I don't have time to debug it18:25
jonathanmaw+1 on making the tests non-gating until we've sorted them out properly18:26
*** jonathanmaw has quit IRC18:27
*** raoul_ has quit IRC18:44
*** johnward has joined #buildstream18:51
gitlab-br-botjjardon opened MR !1145 (jjardon/allow_fail_WSL->master: .gitlab-ci.yml: allow "test-wsl" to fail until runner problems are sorted out) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/114519:14
*** lantw44 has quit IRC19:47
*** lantw44 has joined #buildstream20:00
*** lachlan has quit IRC20:17
gitlab-br-botjuergbi merged MR !1145 (jjardon/allow_fail_WSL->master: .gitlab-ci.yml: allow "test-wsl" to fail until runner problems are sorted out) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/114521:29
gitlab-br-botjjardon opened (was WIP) MR !1137 (jjardon/fedora_29->master: .gitlab-ci.yml: Test with current fedora release: 29) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/113721:32
*** jonathanmaw has joined #buildstream21:43
*** jonathanmaw has quit IRC21:43
*** nimish has quit IRC21:44
*** erle- has joined #buildstream22:02
*** alatiera has quit IRC23:22

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!