| *** tristan has quit IRC | 07:13 | |
| *** tristan has joined #buildstream | 07:32 | |
| *** bochecha has joined #buildstream | 07:53 | |
| *** alexandrufazakas has joined #buildstream | 08:04 | |
| *** tiagogomes has joined #buildstream | 08:13 | |
| *** benschubert has joined #buildstream | 08:18 | |
| *** raoul has joined #buildstream | 08:41 | |
| *** phil has joined #buildstream | 09:07 | |
| *** jonathanmaw has joined #buildstream | 09:08 | |
| ironfoot | Hello! o/ I'm currently playing with buildstream, and one of my builds failed. In the build log file, the last thing you can see is "failed with exitcode 1". Is there a way to see the actuall error? | 09:10 |
|---|---|---|
| juergbi | ironfoot: this means that the external command failed and its stdout/stderr messages should be in the build log file as well | 09:15 |
| ironfoot | yeah, I can see some of the stderr output in this log, I was surprised it wasn't clear when I looked at it | 09:18 |
| ironfoot | ah right, at the end of the log there is the command that failed, repeated | 09:18 |
| ironfoot | context to ilustrate why it was a bit confusing: https://paste.gnome.org/phw5wmpxy | 09:21 |
| juergbi | ah, due to the huge command line | 09:22 |
| ironfoot | yeah | 09:22 |
| ironfoot | full error: https://paste.gnome.org/pbmeyqlx4 | 09:22 |
| *** ChanServ sets mode: +o tristan | 09:24 | |
| tristan | valentind, ^^^^^^^^^^^^^^^ Does that error building the cross compiler mean anything to you ? | 09:24 |
| valentind | Yes | 09:24 |
| valentind | There are binaries with hardlinks. | 09:25 |
| valentind | The problem is that for freedesktop-sdk we use the original path for the path to debug file. We do not use build-id. | 09:25 |
| valentind | But if there are hardlinks, since find follows filesystem order which is not reproducible, we might get a different path as first one. | 09:26 |
| tristan | valentind, Orthogonal question... do you know why the build log does not tell you more clearly ? | 09:26 |
| tristan | Oh | 09:26 |
| valentind | So what we do in the install-commands, is to replace hardlinks by symlinks. | 09:26 |
| tristan | "Stripping files with multiple links can be problematic for reproducibility." | 09:26 |
| valentind | Stripping files with multiple links can be problematic for reproducibility. | 09:27 |
| valentind | File /buildstream-install/usr/lib/sdk/toolchain-x86_64/bin/gcc-ar has multiple links | 09:27 |
| valentind | This was in the build logs. | 09:27 |
| tristan | Maybe that message from the toolchain is missing some more context | 09:27 |
| valentind | How more clearly do you want it? | 09:27 |
| tristan | I.e. it could be "ERROR: <insert above message>, failed" | 09:27 |
| valentind | Sure, we could do that. | 09:28 |
| valentind | Next time we change all the artifact keys, we can do that. | 09:28 |
| valentind | I think we should move to using debug-id path for debug files instead anyway. | 09:29 |
| valentind | So that we do not care about hardlinks. | 09:29 |
| benschubert | Hey tristan, when you have time, could you look if you see other things I might have missed when testing my changes for https://gitlab.com/BuildStream/buildstream/merge_requests/1438#note_187177300 ? | 09:32 |
| alexandrufazakas | I could also really use a review on !1430 if anyone has some time for it :) | 09:34 |
| gitlab-br-bot | MR !1430: Add bst init argument https://gitlab.com/BuildStream/buildstream/merge_requests/1430 | 09:34 |
| *** raoul_ has joined #buildstream | 09:39 | |
| *** raoul has quit IRC | 09:41 | |
| *** lachlan has joined #buildstream | 09:46 | |
| alexandrufazakas | tpollard: thanks for having a look at my MR :) | 09:57 |
| tristan | benschubert, looks great :) | 10:02 |
| benschubert | tristan: thanks a lot! | 10:03 |
| *** tiagogomes has quit IRC | 10:21 | |
| gitlab-br-bot | marge-bot123 merged MR !1419 (aevri/set_resource_limits->master: platform: re-scope set_resource_limits) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1419 | 10:23 |
| *** tiagogomes has joined #buildstream | 10:25 | |
| tpollard | alexandrufazakas: np, I think you're pretty close to getting approval. As it's a breaking change to the cli we just have to ensure we're getting it *right* | 10:30 |
| alexandrufazakas | tpollard: Yeah, I understand that :) | 10:30 |
| *** raoul_ has quit IRC | 10:38 | |
| *** raoul_ has joined #buildstream | 10:38 | |
| gitlab-br-bot | BenjaminSchubert merged MR !1438 (bschubert/node-api-nosanitize->bschubert/new-node-api: Remove the need for 'node_sanitize') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1438 | 10:58 |
| *** raoul__ has joined #buildstream | 11:01 | |
| *** raoul_ has quit IRC | 11:01 | |
| *** bochecha_ has joined #buildstream | 11:06 | |
| *** bochecha has quit IRC | 11:06 | |
| *** bochecha_ is now known as bochecha | 11:06 | |
| ironfoot | so, I re-ran everything manually on the bst shell, and it worked? https://paste.gnome.org/puzycqzt0 | 11:20 |
| *** traveltissues has joined #buildstream | 11:21 | |
| tristan | ironfoot, and you typed "echo $?" after every command ? | 11:23 |
| tristan | I am not clear on the result of the last conversation with valentind about this... the thing I dont understand is... He seems to clearly state that "yes... this is correct, it does not work"... on the other hand... this looks like it is build instructions from the freedesktop-sdk cross-compiler build... which "clearly work" | 11:25 |
| tristan | So which is it ? Does it work or doesn't it ? | 11:25 |
| ironfoot | bst build thing.bst, doesn't work, then from within the shell gives me after failure seems to work. Doing a clean build/failure to have a shell at the point of the failure now to retry | 11:31 |
| ironfoot | I'll run things inside of a subshell with `set -ex` to see if any of the commands fail | 11:32 |
| juergbi | ironfoot: '...has multiple links' is a fatal error as it executes `false` | 11:32 |
| juergbi | https://paste.gnome.org/pbmeyqlx4#line-166 | 11:32 |
| ironfoot | right | 11:33 |
| juergbi | i.e., this is intentional in this script | 11:33 |
| juergbi | either avoid multiple links or change the script if this is not considered an issue in your environment | 11:33 |
| tristan | Ahh | 11:35 |
| tristan | valentind, I didn't realize it was our script printing that unclear error message | 11:35 |
| tristan | valentind, I thought it was something from objdump and friends | 11:35 |
| tristan | Error message looks more like just a remark / warning to me :) | 11:35 |
| ironfoot | yeah, I was %100 sure that was a warning :/ | 11:37 |
| valentind | It is not a warning. | 11:37 |
| lachlan | Hi, I'm trying to do some benchmark analysis using formatted logfiles as part of overall benchmarking. As part of this process I'm trying to establish the time deltas between different runs, but I've come across mismatches in logged entries - more specifically I have entries in the binutils baserock build log with messages such as "fetch:gnu-toolchain/stage1-binutils.bst-0,START ..." which mismatch to other logs (from different runs) through the intro | 11:40 |
| lachlan | duction of the "-0" at the end of the bst file entry. As far as I can tell there should be no persist-ency in the build system (docker based) with each build run being carried out in a clean environment, although this is still being examined. Can anyone provide any initial ideas as to whether buildstream would cause this number to be appended to bst files and why? | 11:40 |
| tpollard | I've had issues with that whilst poking around with the frontend subprocessing | 11:44 |
| lachlan | So not unique, hmm | 11:46 |
| tpollard | It's expected though | 11:47 |
| tristan | Can we stop calling this "frontend subprocessing" or "running the UI in a subprocess" ? | 11:48 |
| tristan | I thought we had clearly/firmly agreed and decided that that was precisely the opposite of what we were going to do | 11:49 |
| tpollard | tristan: I would prefer to stop calling it that yes | 11:49 |
| *** traveltissues has quit IRC | 11:49 | |
| tristan | It reads quite badly if we have mailing list threads which still hold on to that title (which jumps out at you like something not to do), and then have to read into it to find out that people are in fact talking about doing the opposite | 11:50 |
| tpollard | lachlan: when a source related job is happening (in this case fetching the element's source) the front end prints the name of the own element with the index of source appended to it | 11:50 |
| tpollard | which in most cases is the 0 index | 11:51 |
| tristan | tpollard, In fact... that is not really true - although I can see how that may have become true in a side branch | 11:52 |
| tristan | tpollard, it did do that at one point, and stopped | 11:52 |
| tristan | tpollard, the reason for this is essentially this line: https://gitlab.com/BuildStream/buildstream/blob/master/src/buildstream/_frontend/widget.py#L192 | 11:53 |
| tristan | tpollard, it's rather a tiny unimportant detail and we could easily pivot, and make a UX decision about it later on, just pointing that out | 11:54 |
| tristan | right now we prefer printing the *task* element ID | 11:54 |
| tristan | not the unique_id of the specific element doing something | 11:54 |
| tristan | however, I should point out that this is rather important in another sense | 11:55 |
| tristan | tpollard, The reason we originally did that... is because you have situations where you are "Building foo.bst"... and then you are going to stage and run integration commands and such from a bunch of dependencies... | 11:55 |
| tristan | tpollard, in the process of building "foo.bst", you end up calling functions on dependencies which might issue log messages | 11:56 |
| tristan | tpollard, If you print the actual element names of the dependencies which are doing things on behalf of that build... the logging is completely unintelligible | 11:56 |
| tristan | I think the main reason why we have "task_id or element_id" on that line... is because in some contexts (like when you run `bst checkout` or `bst shell`), you don't currently *have* an associated task_id | 11:57 |
| tristan | because things are being run directly from the main process instead | 11:58 |
| tristan | that will probably change with jonathanmaw's recent branch and with subprocessing of the scheduler, ensuring that "things always happen as tasks" | 11:58 |
| tristan | I would expect it to probably change, anyway | 11:58 |
| tristan | tpollard, Sorry for the wall of text, I hope it is enlightening though :) | 11:59 |
| lachlan | The main problem I have is that this formatting does not seem to be consistently applied, but it can be worked around - although I fear that things might be stumbling into a soup of regex mangling to create comparable build trees. | 12:00 |
| tristan | Essentially... whether we print the "-0" or not for source related things is unimportant... but it *is* important that we don't print the element names of dependencies as the task element in the logging, as that makes things really unintelligible | 12:01 |
| tristan | lachlan, Ok so... what *you* are saying is that this effects a background story which is *parsing* the log lines for benchmarking purposes ? | 12:02 |
| tristan | Might I infer ? | 12:02 |
| tristan | lachlan, I would say that if that is the case, then it becomes important to ensure consistency on this level yes - without regressing https://gitlab.com/BuildStream/buildstream/blob/master/src/buildstream/_frontend/widget.py#L192 | 12:02 |
| lachlan | Yes, this is as per the plan to introduce structured logging and perform benchmarking logging analysis to provide deeper performance statistics | 12:03 |
| tristan | lachlan, The answer might be that we should *never* use message.element_id, and always have messages issued by elements grouped by *task_id* | 12:03 |
| tristan | OR | 12:03 |
| tpollard | tristan: well then there might be a bug somewhere, because I'm pretty sure that behaviour is existing on master | 12:04 |
| tristan | Perhaps we should have %{task-id} and %{element-id} as separate configrations in the logging format | 12:04 |
| tristan | tpollard, I suspect you are seeing it at checkout or shell time | 12:04 |
| lachlan | I have already introduced so regex work to circumvent some file stamping, so this can be introduced as another rule since it is now clear that this is an expected buildstream effect. | 12:04 |
| lachlan | *some | 12:05 |
| tristan | tpollard, But it's certainly a bug... imagine trying to visually parse a log where you have many ongoing builds... and then you have dependencies issuing log lines and you have no idea which START/SUCCESS task those lines are related to | 12:05 |
| tristan | tpollard, it's not technically a bug if it's not happening in scheduled builds | 12:05 |
| tristan | tpollard, because when you type `bst checkout foo.bst`, you really know you are only doing one thing | 12:06 |
| tristan | ergo it doesnt appear to be a problem | 12:06 |
| tristan | lachlan, I would imagine being able to set [%{task-id}][%{element-id}] in `logging: format:` in the user config would be very useful to you though... as it would allow you to both (A) Identify which START/STOP are related to which *task*, and (B) Also identify specific granular things at the "what each element did" level, while doing so on behalf of the task | 12:08 |
| tristan | lachlan, but that does sound like an elaborate thing which is probably unneeded in the short term :) | 12:08 |
| tpollard | tristan: looking at the commits in my working branch... looks like I was having the problem with Source() _track generating the info message | 12:08 |
| tristan | tpollard, So I just looked at a `bst track` output and didnt have any bugs (it did *not* print any trailing "-0" in that context), but not with master | 12:09 |
| tristan | So I think that is an important regression to fix if it happened | 12:09 |
| *** raoul__ has quit IRC | 12:10 | |
| *** bochecha_ has joined #buildstream | 12:12 | |
| tpollard | agreed | 12:13 |
| tpollard | and changes quite a bit with out approach of not doing plugin lookups for the multiprocessing | 12:14 |
| *** bochecha has quit IRC | 12:14 | |
| *** bochecha_ is now known as bochecha | 12:14 | |
| tristan | Right | 12:15 |
| tristan | tpollard, we'll want to decide what info is relevant to send over the wire... but I would think it's small enough that it is worth (A) Sending both task related element info AND element specific info over the wire... (B) Let the frontend decide what to do with that | 12:16 |
| tpollard | and I'll ensure that when the WIP goes up for the foundations of the work that it's not referred to as the UI being subprocessed! | 12:16 |
| tristan | tpollard, I'm off to dinner... thanks for policing the naming of the task for me :) | 12:20 |
| tristan | tpollard, I worry sometimes with so many people working on different things, small headlines like this being corrected much improves overall group understanding of what is going on :D | 12:20 |
| *** tristan has quit IRC | 12:24 | |
| *** traveltissues has joined #buildstream | 12:26 | |
| *** traveltissues has quit IRC | 12:38 | |
| *** raoul__ has joined #buildstream | 12:48 | |
| jonathanmaw | Hi tristan, I've finished my changes to !1409 to rearrange where State is created, make the frontend read state directly, and use callbacks for notifying the frontend when something's changed | 13:03 |
| gitlab-br-bot | MR !1409: Separate frontend state handling from core state https://gitlab.com/BuildStream/buildstream/merge_requests/1409 | 13:03 |
| *** phildawson_ has joined #buildstream | 13:04 | |
| *** phil has quit IRC | 13:05 | |
| jonathanmaw | though I did end up with vestigial callbacks for changes to TaskGroups, since the StatusHeader widget doesn't store anything that needs to be recalculated | 13:05 |
| *** lachlan has quit IRC | 13:25 | |
| *** phildawson_ has quit IRC | 13:51 | |
| *** lachlan has joined #buildstream | 13:51 | |
| *** lachlan has quit IRC | 13:57 | |
| *** phildawson_ has joined #buildstream | 14:02 | |
| *** lachlan has joined #buildstream | 14:04 | |
| *** lachlan has quit IRC | 14:07 | |
| *** traveltissues has joined #buildstream | 14:08 | |
| *** lachlan has joined #buildstream | 14:10 | |
| tlater[m] | alexandrufazakas: Do tell when your branch is ready | 14:17 |
| tlater[m] | I think we might be looking at a merge today :) | 14:17 |
| alexandrufazakas | hey tlater. I think it's ready right now tbf | 14:18 |
| tlater[m] | Right, I'll give it a quick final review | 14:19 |
| alexandrufazakas | Awesome, thank you :D | 14:19 |
| tlater[m] | (And get marge to merge my MR first ;p) | 14:20 |
| *** CTtpollard has joined #buildstream | 14:22 | |
| tlater[m] | alexandrufazakas: Good work so far, but have you amended the documentation in doc/examples/first_project.rst or whatever that file was? | 14:23 |
| *** tpollard has quit IRC | 14:23 | |
| tlater[m] | Hm, I suppose it might not need updating since it doesn't use `-C`? | 14:23 |
| *** lachlan has quit IRC | 14:24 | |
| tlater[m] | Yeah, that looks alright | 14:25 |
| alexandrufazakas | tlater: Nothing there uses -C or --directory I think | 14:25 |
| tlater[m] | Yup, I just checked | 14:26 |
| alexandrufazakas | Sorry, I was playing with grep to see if I missed something there | 14:26 |
| tlater[m] | alexandrufazakas: In that case, could you do me a favor and squash all but the first and last commits? | 14:27 |
| tlater[m] | It's a well engineered history, but i don't think the tests will continuously succeed without an atomic change :) | 14:27 |
| alexandrufazakas | tlater: So all commits in between should be squashed into `Make bst init take an argument`? | 14:28 |
| *** raoul__ is now known as raoul | 14:28 | |
| alexandrufazakas | tlater: I think they worked with every commit, last time I checked they did | 14:28 |
| alexandrufazakas | If that's what you menat | 14:28 |
| tlater[m] | Other than that, you have my approval | 14:28 |
| alexandrufazakas | s/menat/meant | 14:28 |
| tlater[m] | Hm, you're right, they should actually pass | 14:30 |
| tlater[m] | Sure, that's good. Thanks for putting up with my pedantry :) | 14:30 |
| gitlab-br-bot | tlater approved MR !1430 (AlexFazakas/add-bst-init-argument->master: Add bst init argument) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1430 | 14:30 |
| alexandrufazakas | Haha, no worries. You helped me out a bunch with all the work on this MR | 14:30 |
| tlater[m] | alexandrufazakas: Rebase, and then assign marge-bot123 (top right, the "assign" button) | 14:31 |
| tlater[m] | Marge should then just grab your MR and eventually merge it (whenever she's done merging whoever's comes first). | 14:31 |
| alexandrufazakas | tlater: Aight, I just did that | 14:32 |
| alexandrufazakas | Love the bot's name btw, whoever came up with that hah | 14:32 |
| tlater[m] | It is rather cool, but I've not had the opportunity to tell someone to "marge" their branch yet :| | 14:32 |
| raoul | If I try and build freedesktop-sdk or buildbox-integration I get "[synthetic node]: Value of 'submodules' is not of the expected type 'Mapping'" Anyone got any idea what's causing this? it's not a particularly useful error message and all actual occurrences of submodules are dictionaries/mappings | 14:32 |
| tlater[m] | raoul: I think that's fallout from 1.2 -> 2.0 | 14:33 |
| tlater[m] | You'll want to build a specific branch of freedesktop-sdk | 14:33 |
| tlater[m] | (search for -bst2 among the branches) | 14:33 |
| raoul | tlater[m], get the same thing on "freedesktop-sdk-18.08.31.1-bst2" which seems to be the only bst2 branch | 14:35 |
| tlater[m] | raoul: bst-plugins-experimental? | 14:36 |
| tlater[m] | Or bst-external? | 14:36 |
| raoul | oh do I need to change which plugins are referred to? | 14:36 |
| tlater[m] | raoul: You shouldn't need to if you're on the correct branch | 14:38 |
| tlater[m] | But that's the error you get if you have the wrong plugins running, so you probably aren't on the right branch :D | 14:38 |
| *** lachlan has joined #buildstream | 14:40 | |
| raoul | oh looking at the overnight tests maybe I want to be on a specific version of bst-plugins-experimental | 14:42 |
| tlater[m] | raoul: I've not had to do that, latest should work | 14:42 |
| tlater[m] | Unless this broke over the past 4 days or so | 14:42 |
| raoul | well it works now after I installed some missing packages so I guess you do now? | 14:45 |
| tlater[m] | Hrm, that would be bad | 14:46 |
| tlater[m] | raoul: Open an issue against bst-plugins-experimental? | 14:47 |
| *** lachlan has quit IRC | 14:47 | |
| raoul | tlater[m], can do, could you check it doesn't work for you either? | 14:49 |
| * tlater[m] stashes his tree again... ;p | 14:49 | |
| raoul | oh wait maybe I just didn't pull on bst-plugins-experimental | 14:51 |
| raoul | swear I did | 14:51 |
| tlater[m] | raoul: Working fine for me | 14:53 |
| raoul | yeah seems to work now, must have been on an older version of bst-plugins-experimental that didn't work at some point | 14:54 |
| tlater[m] | That would do it, yes | 14:54 |
| tlater[m] | It's a fairly recent change, ~2 weeks I believe | 14:55 |
| *** lachlan has joined #buildstream | 15:00 | |
| gitlab-br-bot | marge-bot123 closed issue #1052 (BuildStream chokes when staging tarfiles with hardlinks that point outside of a chosen base directory) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1052 | 15:03 |
| gitlab-br-bot | marge-bot123 merged MR !1431 (tar-target-renaming->master: tar.py: Make link target renaming work between base-dirs) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1431 | 15:03 |
| *** lachlan has quit IRC | 15:07 | |
| *** lachlan has joined #buildstream | 15:15 | |
| *** lachlan has quit IRC | 15:19 | |
| gitlab-br-bot | BenjaminSchubert opened MR !1442 (bschubert/new-node-compose->bschubert/new-node-api: Rework node compose API) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1442 | 15:22 |
| *** lachlan has joined #buildstream | 15:38 | |
| gitlab-br-bot | marge-bot123 closed issue #702 (Make `bst init` take an argument) on buildstream https://gitlab.com/BuildStream/buildstream/issues/702 | 15:43 |
| gitlab-br-bot | marge-bot123 merged MR !1430 (AlexFazakas/add-bst-init-argument->master: Add bst init argument) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1430 | 15:43 |
| tlater[m] | Congrats alexandrufazakas :) | 15:43 |
| *** bochecha has quit IRC | 15:48 | |
| alexandrufazakas | Thanks tlater :^) | 15:52 |
| gitlab-br-bot | BenjaminSchubert opened MR !1444 (bschubert/node-api-noassertions->bschubert/new-node-api: _yaml: Move 'node_final_assertions' to 'Node._assert_fully_composited') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1444 | 15:53 |
| *** lachlan has quit IRC | 15:58 | |
| *** lachlan has joined #buildstream | 16:20 | |
| *** phil has joined #buildstream | 16:24 | |
| *** phildawson_ has quit IRC | 16:26 | |
| *** alexandrufazakas has quit IRC | 16:30 | |
| *** traveltissues has quit IRC | 16:36 | |
| gitlab-br-bot | aevri opened (was WIP) MR !1443 (aevri/context_nitpicks->master: Minor fixups in _context.py) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1443 | 16:41 |
| *** tristan has joined #buildstream | 16:42 | |
| *** CTtpollard is now known as tpollard | 16:50 | |
| tpollard | sorry for stopping the marge aevri :) | 16:50 |
| aevri | Not at all, my fault. Thanks for reviewing so quick :) | 16:51 |
| gitlab-br-bot | tpollard approved MR !1443 (aevri/context_nitpicks->master: Minor fixups in _context.py) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1443 | 16:57 |
| *** jonathanmaw has quit IRC | 17:02 | |
| *** phil has quit IRC | 17:05 | |
| *** lachlan has quit IRC | 17:31 | |
| *** tiagogomes has quit IRC | 17:38 | |
| *** traveltissues has joined #buildstream | 17:40 | |
| *** traveltissues has quit IRC | 18:07 | |
| gitlab-br-bot | marge-bot123 merged MR !1443 (aevri/context_nitpicks->master: Minor fixups in _context.py) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1443 | 19:33 |
| *** benschubert has quit IRC | 21:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!