IRC logs for #buildstream for Thursday, 2018-11-08

*** xjuan has quit IRC01:27
*** Juan__ has quit IRC01:47
*** Juan__ has joined #buildstream01:47
*** Juan_ has joined #buildstream01:48
*** Juan__ has quit IRC01:51
*** Juan_ has quit IRC01:52
*** juanalday has joined #buildstream01:59
*** Juan_ has joined #buildstream02:00
*** Juan__ has joined #buildstream02:01
*** juanalday has quit IRC02:02
*** Juan_ has quit IRC02:03
*** Juan__ has quit IRC02:04
*** ntz has joined #buildstream02:25
*** dabdine has joined #buildstream02:40
*** evilgohan2 has joined #buildstream06:02
*** alatiera_ has joined #buildstream07:27
*** Elazar has joined #buildstream07:58
gitlab-br-botjjardon opened issue #756 (overnigth tests are failing on building base/locales.bst: "TypeError: 'NoneType' object is not subscriptable") on buildstream https://gitlab.com/BuildStream/buildstream/issues/75608:00
jjardonHi, has anyone seen this compilation error with grpcio before?: https://gitlab.com/BuildStream/buildstream-docker-images/-/jobs/11776430208:12
juergbijjardon: is this the first time trying this with aarch64?08:22
jjardonjuergbi: yup08:22
juergbijjardon: grpc is not pure python, so it needs to be built. for x86-64 a prebuilt version is on pypi, maybe there is still none for aarch6408:23
juergbiwhich means that it needs a toolchain as build dep08:23
jjardonjuergbi: it only fails for F2908:23
juergbimaybe g++ is already installed on Debian?08:23
jjardonDebian and F28 seems to be fine08:23
jjardonah08:24
juergbior it depends on libc version whether the prebuilt version is used, or something like that08:24
jjardonlet me see08:24
juergbias it's C++ where the ABI is compiler version dependent, maybe it wants to do a rebuild because of gcc808:24
juergbiI don't see any prebuilds for aarch64: https://pypi.org/project/grpcio/#files08:25
jjardonjuergbi: compiler version is the same for F28 and F2908:27
jjardonglibc versions are different though (2.27 vs 2.28)08:28
juergbijjardon: might need a patch like this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=86754108:34
juergbidon't know why it works on F28, though08:34
juergbithe issue is that grpc sources are taken from pypi, so we can't trivially patch that08:34
*** alatiera__ has joined #buildstream08:34
*** alatiera_ has quit IRC08:36
jjardonyeah :/08:40
* jjardon opened https://github.com/grpc/grpc/issues/1715608:40
jjardonnot a lot of rush as seems several tests are failing when running on aarch64 anyway: https://gitlab.com/BuildStream/buildstream/issues/75508:47
*** bochecha has joined #buildstream08:57
gitlab-br-botjjardon closed issue #370 (Setup aarch64 runners) on buildstream https://gitlab.com/BuildStream/buildstream/issues/37009:34
gitlab-br-botvalentindavid opened MR !940 (valentindavid/script-artifact-corruption-1.2->bst-1.2: [backport] Fix cache corruption by scripts when layout and integration commands are used) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94009:43
jjardonvalentind: have you seen https://gitlab.com/BuildStream/buildstream/issues/756 ; not sure is related with the latest changes to fix the cache corruption09:46
*** toscalix has joined #buildstream09:46
valentindjjardon, it might have to do with my fix of scriptelement.09:46
valentindI am looking at it.09:46
*** alatiera__ is now known as alatiera_09:51
*** jonathanmaw has joined #buildstream09:58
*** benschubert has joined #buildstream10:00
valentindjjardon, I try to build locally, but I get: AttributeError: 'GitTagSource' object has no attribute 'tracked'10:00
valentindOn gnu-config.bst10:00
gitlab-br-botvalentindavid closed issue #645 (Default stripping rules cause seemingly random failures) on buildstream https://gitlab.com/BuildStream/buildstream/issues/64510:01
gitlab-br-botvalentindavid merged MR !918 (valentindavid/fix-recursive-strip-debug->master: Fix infinite recursion in default strip debug command) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/91810:01
gitlab-br-botvalentindavid reopened issue #645 (Default stripping rules cause seemingly random failures) on buildstream https://gitlab.com/BuildStream/buildstream/issues/64510:02
jjardonvalentind: are you building 18.08 HEAD or the specific commit specify in buildstream .gitlab-ci.yml?10:03
*** raoul has joined #buildstream10:03
valentindjjardon, 18.08 head10:03
jjardonvalentind: do you have installed the latest bst-external? (for the git_tag plugin)10:04
valentindjjardon, yes10:04
valentindjjardon, I tried bst_external with both master and the commit in fd.o-sdk's .gitlab-ci hash for it.10:05
jjardonstrange, did you try to build fdsk 88d7c22c2281b987faa02edd57df80d430eecf1f ?10:06
valentindjjardon, It looks to me the way git_tag is written, it will not work with newer versions of buildstream.10:06
valentindjjardon, Now I am trying with 88d7c22c2281b987faa02edd57df80d430eecf1f and it seems to work better.10:06
jjardonvalentind: does that mean buildstream master is somehow breaking API with the plugins or something? Maybe we should document that then10:07
valentindjjardon, no, it is not API. It is just changing the git plugin.10:08
valentindjjardon, what the git_tag plugin does is to inherit from the git plugin from buildstream.10:08
valentindwhat it should do is just copy the version from 1.2 and inherit from that.10:08
jjardonvalentind: maybe I'm wrong, but if a plugin works with 1.2.x, but it doesn't with master, then somehow buildstream is breaking API?10:10
valentindjjardon, It uses non API stuff.10:10
valentindjjardon, the git plugin is not API.10:10
valentindAnd git_tag inherits from the git plugin.10:11
valentindThe git plugin however uses API. So you could copy it.10:11
valentindI mean you could copy the git plugin from 1.2 and make it run on 1.4.10:12
jjardonright, I guess this would be solved when we fix https://gitlab.com/BuildStream/buildstream/issues/739 ?10:12
valentindjjardon, If they are stable yes.10:12
valentindjjardon, for the time being, just copy the plugin from 1.210:13
jjardonok, coldtom: ^^ this can be of interest10:14
*** tiagogomes has quit IRC10:17
*** tiagogomes has joined #buildstream10:18
*** lachlan has joined #buildstream10:30
gitlab-br-botcoldtom opened issue #757 (`bst workspace open` dislikes git sources sometimes) on buildstream https://gitlab.com/BuildStream/buildstream/issues/75710:41
gitlab-br-bottoscalix closed issue #743 (Document incompatibilities of the 1.4 release) on buildstream https://gitlab.com/BuildStream/buildstream/issues/74310:44
*** alatiera_ has quit IRC10:45
toscalixdear bot, I did not close it. I moved it to the website repo.10:45
*** lachlan has quit IRC10:46
*** lachlan has joined #buildstream10:56
*** jonathanmaw_ has joined #buildstream10:59
*** jonathanmaw has quit IRC11:00
*** lachlan has quit IRC11:01
jennisjjardon, regarding my fix for #733, I've not actually fixed this11:01
gitlab-br-botIssue #733: quota config parameter should be the maximum amount of cache allowed, not the minimum https://gitlab.com/BuildStream/buildstream/issues/73311:01
jennisAll it was was the error message being misleading...11:02
jennisFor example, if you specify a cache quota of 90% disk space, and the cache is on a file system where we've already used  20% BuildStream fails11:03
*** ctolentino has joined #buildstream11:03
jennisIt's just it was failing and printing:11:03
jennisError instantiating artifact cache: Your system does not have enough available space to support the cache quota specified.11:03
jennisYou currently have:11:03
jennis- 8G of cache in use at /home/jennis/.cache/buildstream/artifacts11:03
jennis- 363G of available system storage11:03
jennisWhich was misleading11:03
jjardonah, ok. Well at least improves the confusion in that area11:04
gitlab-br-botBenjaminSchubert closed issue #696 (Missing tests for Linux-non-bwrap-non-FUSE situations) on buildstream https://gitlab.com/BuildStream/buildstream/issues/69611:07
gitlab-br-botBenjaminSchubert closed issue #736 (Integration tests are leaking into each others) on buildstream https://gitlab.com/BuildStream/buildstream/issues/73611:07
gitlab-br-botBenjaminSchubert merged MR !921 (bschubert/tests-no-chroot-linux->master: test behavior when there is no sandbox on Linux) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/92111:07
gitlab-br-botvalentindavid reopened issue #749 (Script element corrupts cache when using layout and enables integration commands) on buildstream https://gitlab.com/BuildStream/buildstream/issues/74911:09
gitlab-br-botvalentindavid opened MR !941 (valentindavid/fix-script-install-root->master: Fix bug with root mounted as non-artifact in script plugin.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94111:10
gitlab-br-botBenjaminSchubert opened issue #758 (the pip source plugin incorreclty search for a valid version) on buildstream https://gitlab.com/BuildStream/buildstream/issues/75811:10
valentindjjardon, !941 should fix the issue during the orvernight build.11:11
gitlab-br-botMR !941: Fix bug with root mounted as non-artifact in script plugin. https://gitlab.com/BuildStream/buildstream/merge_requests/94111:11
jjardonvalentind: cool! thanks11:11
valentindThat was stupid of me. I "inlined" the value of a variable in the case I was testing.11:13
jjardonvalentind: :) Is the fix included in the backport at https://gitlab.com/BuildStream/buildstream/merge_requests/940/ as well?11:14
valentindjjardon, not yet. I will squash it now.11:14
juergbivalentind: hm, regarding #749, shouldn't this prevented by BuildStream core? i.e., I think we should make it impossible for (reasonable) element plugins to corrupt the cache11:16
gitlab-br-botIssue #749: Script element corrupts cache when using layout and enables integration commands https://gitlab.com/BuildStream/buildstream/issues/74911:16
juergbiI've only glanced over the issue, so I might be missing something11:16
jjardonvalentind: cool. Sorry I ask again but I still have a problems with  https://gitlab.com/BuildStream/buildstream/merge_requests/875/ : the build fail with "bwrap: execvp sh: No such file or directory" when I try to use the SDK: https://gitlab.com/BuildStream/buildstream/-/jobs/117927066 any idea what is missing?11:17
valentindjuergbi, Yes, we should redesign thing so that plugins cannot mess up.11:17
juergbiok, in that case we should either keep that issue open with a suitable comment or file a new issue, otherwise we forget about it11:17
gitlab-br-botBenjaminSchubert opened MR !942 (bschubert/fix-pip-python->master: plugins/sources/pip.py: also look for python version named "python") on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94211:17
juergbi(with BuildBox this is anyway prevented, though)11:18
valentindjuergbi, toscalix told me to open an issue about that. I forgot, but I will do it.11:18
juergbita11:18
valentindjjardon, !940 (backport) has the fix now11:19
gitlab-br-botMR !940: [backport] Fix cache corruption by scripts when layout and integration commands are used https://gitlab.com/BuildStream/buildstream/merge_requests/94011:19
*** lachlan has joined #buildstream11:21
*** lachlan has quit IRC11:24
gitlab-br-botvalentindavid opened issue #759 (Plugins can easily corrupt cache) on buildstream https://gitlab.com/BuildStream/buildstream/issues/75911:28
valentindjuergbi, #75911:28
gitlab-br-botIssue #759: Plugins can easily corrupt cache https://gitlab.com/BuildStream/buildstream/issues/75911:28
valentindI marked it urgent so we will discuss it with Tristan.11:29
valentindIf someone could review !941 which is a fix to my f**k up in !934. That would be nice.11:37
gitlab-br-botMR !941: Fix bug with root mounted as non-artifact in script plugin. https://gitlab.com/BuildStream/buildstream/merge_requests/94111:37
gitlab-br-botMR !934: Fix cache corruption by scripts when layout and integration commands are used https://gitlab.com/BuildStream/buildstream/merge_requests/93411:37
Kinnisonvalentind: While I can't see anything wrong with !941, I'm not sure I understand how it fixes things, even given the test, so I won't say anything on-issue11:39
valentindKinnison, ok I can git some more explanations.11:40
Kinnisonvalentind: an explanation of what the actual error is in the issue would be sufficient11:40
Kinnisonvalentind: as it is, the issue is just about a typeerror11:41
valentindKinnison, maybe there is a problem in the exception handling. But the real issue is: OSError: [Errno 39] Directory not empty: '/root/.cache/buildstream/build/base-locales-2adia6f7/root/tmp'11:54
valentindKinnison, I have fixed the title of #75611:55
gitlab-br-botIssue #756: overnigth tests are failing on building base/locales.bst: OSError: [Errno 39] Directory not empty: '/root/.cache/buildstream/build/base-locales-2adia6f7/root/tmp' https://gitlab.com/BuildStream/buildstream/issues/75611:55
KinnisonRight, but that doesn't tell me *WHY* that exception was happening11:55
Kinnisoni.e. what is the fix doing?11:55
KinnisonIs it non-empty because it was mounting the *real* /tmp in?11:55
KinnisonIf so, then the regression test MUST ensure that the real /tmp doesn't leak in11:55
valentindKinnison, sure, I can try to see if it leaked.11:56
KinnisonIn generally I like regression tests to have explanations of what they're preventing a regression of, though I'm very happy for that to be via the issue closed as part of the commit message :-)11:57
KinnisonIn generally?  In general!11:57
* Kinnison sighs and clearly needs his lunch11:57
valentindAh, yes, but it fails, so I cannot checkout and verify it leaked. I will add another test using another directory than tmp12:00
* Kinnison wonders how none of the other integration tests failed12:01
Kinnisoni.e. what about the nightlies caused the issue to be exposed?12:01
valentindKinnison, ok, so the reason it does not leak into the cache is because directories are not hard links.12:04
valentindSo adding files is actually safe.12:04
valentindI can try to find another test that modifies a file in root.12:05
valentindKinnison, sorry, I do not find any cache corruption issue to test.12:09
jennisjjardon just to let you know 939 has bene updated12:11
jennisbeen12:11
Kinnisonvalentind: Aah so this is quite complex to verify :(12:30
valentindKinnison, I am still on it.12:30
valentindWell, it is easy to verify, the test I have crashes before the fix.12:31
Kinnisonvalentind: If you can't find a nice simple solution, then just extra explanation on the issue would change my "I can't see anything wrong" into an "I approve" (as long as it makes sense to me :-)12:31
valentindBut if you want a beautiful test, it is more complex.12:31
Kinnisonvalentind: To that end, an explanation of *why* it crashes without the fix is quite sufficient12:31
valentindI will still try to add a test, at least to understand why it is complex.12:32
Kinnisontext is just as good as code for that :-D12:32
KinnisonOK cool12:32
valentindMaybe I do not manage, but at least I should know why.12:32
* Kinnison apologises if he's being overly whiny, but regression tests really worry me if they're not clear :-)12:32
valentindKinnison, found another test easier to understand. Will still keep the other one though. But it helps to understand what is happening12:40
Kinnisonvalentind: cool, let me know when to re-look at the MR12:41
valentindKinnison, it is pushed.12:43
* Kinnison has a look-see12:43
valentindBasically, when you do not override 'install-root' , then12:43
valentindYou can end up with corruption.12:44
Kinnisonis script/corruption-image.bst missing from the commit, or am I being daft?12:45
valentindKinnison, it came from the previous MR.12:46
valentindIt is just an artifact with a file called canary and content 'alive\n'12:46
valentindSee https://gitlab.com/BuildStream/buildstream/merge_requests/934/diffs12:46
Kinnisonaah okay12:46
gitlab-br-botdanielsilverstone-ct approved MR !941 (valentindavid/fix-script-install-root->master: Fix bug with root mounted as non-artifact in script plugin.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94112:47
valentindthanks12:47
Kinnisonvalentind: with the extra explanation in the MR, and the additional test, that looks good to me -- approved12:47
Kinnisoncaches are hard :-)12:47
gitlab-br-bottlater approved MR !939 (jennis/quota_declaration_fix->master: Add local cache expiry documentation and fix misleading error message when specifying a percentage cache quota) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/93912:53
*** alatiera_ has joined #buildstream13:10
gitlab-br-botvalentindavid closed issue #756 (overnigth tests are failing on building base/locales.bst: OSError: [Errno 39] Directory not empty: '/root/.cache/buildstream/build/base-locales-2adia6f7/root/tmp') on buildstream https://gitlab.com/BuildStream/buildstream/issues/75613:11
gitlab-br-botvalentindavid closed issue #749 (Script element corrupts cache when using layout and enables integration commands) on buildstream https://gitlab.com/BuildStream/buildstream/issues/74913:11
gitlab-br-botvalentindavid merged MR !941 (valentindavid/fix-script-install-root->master: Fix bug with root mounted as non-artifact in script plugin.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94113:11
*** lachlan has joined #buildstream13:58
KinnisonAre there any tests which verify that the cache_key algorithm hasn't changed?14:17
Kinnisoni.e. so we can tell if we've made a change which incorrectly affects cache keys?14:17
valentindKinnison, yes.14:17
Kinnisonexcellent14:17
Kinnisonthank you valentind14:17
Kinnison(I'm fiddling a potential optimisation which touches how _element.__calculate_cache_key() works and was worried :-D14:17
Kinnison)14:18
valentindtests/cachekey/cachekey.py14:18
Kinnisonbrilliant, thank you14:19
* Kinnison has a quick'n'dirty just to see if it improves performance. If it does, I'll look at tidying and installing what I need to run that test14:20
*** lachlan has quit IRC14:23
*** lachlan has joined #buildstream14:24
gitlab-br-botBenjaminSchubert closed issue #732 (Linting always fails when missing FUSE) on buildstream https://gitlab.com/BuildStream/buildstream/issues/73214:25
KinnisonWoot, my change passes those tests, thanks14:27
gitlab-br-botaevri opened MR !943 (aevri/unit_tests->master: WIP: contributing: add guidance on unit tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94314:34
*** lachlan has quit IRC14:42
gitlab-br-bottlater approved MR !935 (aevri/contributing_gitlab->master: Add more to GitLab-relevant parts of contributing) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/93514:57
*** jonathanmaw_ has quit IRC15:12
*** jonathanmaw has joined #buildstream15:12
gitlab-br-botLaurenceUrhegyi approved MR !935 (aevri/contributing_gitlab->master: Add more to GitLab-relevant parts of contributing) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/93515:14
laurencetpollard, tlater[m] can you merge https://gitlab.com/BuildStream/buildstream/merge_requests/935 ?15:15
laurencei think you have the power15:15
*** lachlan has joined #buildstream15:25
*** lachlan has quit IRC15:28
NexusI can't seem to find any generic bst build tests, are there no tests that only test bst build and not also track/checkout?15:36
* Kinnison wonders if anyone might know why https://gitlab.com/BuildStream/buildstream/-/jobs/118114137 happened?15:40
KinnisonIt seems odd that there's a bunch of DEBUG output there15:40
Nexusi've been getting a lot of output too, but different15:41
Nexusapparently jmac fixed it locally previosuly, but can't remember how15:41
Nexusi had so much output that i needed to expand by scroll buffer  to 50k lines15:42
Kinnisonjmac: If you have a moment, would you mind looking at the test failure I linked above and letting me know if you have any ideas?  (That's the only fail in the branch, all other workers passed)15:45
* Kinnison isn't in a rush though15:45
jmacAlready did. Never seen that one before, sorry.15:54
jmacA lot of things like that depend on your pytest version, so it might be worth seeing if updating that with pip3 changes anything15:54
jmacOr alter the version in dev-requirements.txt15:55
Kinnison:( I thought the versions were stable across all our dockers :(15:59
jmacHang on, are you worried about the DEBUG strings or the DeprecationWarnings?16:01
Kinnisonthe DEBUG strings16:03
Kinnisoni.e. why the assert fails16:03
jmacAh, OK, I was looking into DeprecationWarnings for Neux16:04
jmacNexus16:04
jmacNo idea where the DEBUG message comes from16:04
Kinnisonokay, sorry to have invoked you unnecessarily16:08
gitlab-br-botaevri merged MR !935 (aevri/contributing_gitlab->master: Add more to GitLab-relevant parts of contributing) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/93516:17
gitlab-br-botdanielsilverstone-ct opened issue #760 (Plugin.__del__ writes DEBUG messages to stdout) on buildstream https://gitlab.com/BuildStream/buildstream/issues/76016:45
Kinnisontlater[m]: I've filed https://gitlab.com/BuildStream/buildstream/issues/760 which I'd appreciate your input on if you ahve anything to add16:45
Kinnisonjmac: tlater[m] and I have isolated the issue in that pipeline to ^^ if you're interested16:45
*** toscalix has quit IRC16:53
Nexusjuergbi: i've replied/reacted to your comments on https://gitlab.com/BuildStream/buildstream/merge_requests/93716:57
*** lachlan has joined #buildstream16:58
*** abderrahim4 has joined #buildstream17:27
*** abderrahim3 has quit IRC17:29
*** lachlan has quit IRC17:51
*** finn_ has quit IRC18:08
coldtomdoes BuildStream automatically track git submodules?18:09
*** toscalix has joined #buildstream18:10
*** toscalix has quit IRC18:10
gitlab-br-botjjardon closed issue #700 (Document how to cleanup the bst cache: locally and in the server side) on buildstream https://gitlab.com/BuildStream/buildstream/issues/70018:22
gitlab-br-botjjardon merged MR !939 (jennis/quota_declaration_fix->master: Add local cache expiry documentation and fix misleading error message when specifying a percentage cache quota) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/93918:22
gitlab-br-botjjardon reopened issue #700 (Document how to cleanup the bst cache: locally and in the server side) on buildstream https://gitlab.com/BuildStream/buildstream/issues/70018:27
*** raoul has quit IRC18:28
*** lachlan has joined #buildstream18:35
*** lachlan has quit IRC18:39
*** neBu has joined #buildstream18:45
*** lachlan has joined #buildstream18:46
*** lachlan has quit IRC18:48
*** c0ffeemachine has joined #buildstream18:49
*** c0ffeemachine has quit IRC18:51
*** finn_ has joined #buildstream18:52
*** catonano has joined #buildstream18:52
*** jonathanmaw has quit IRC18:54
*** alatiera_ has quit IRC19:12
*** xjuan has joined #buildstream20:09
*** ctolentino has quit IRC20:31
*** _cmd_ has joined #buildstream21:36
*** catonano has quit IRC22:53
*** bochecha has quit IRC23:27
*** mohan43u has quit IRC23:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!