| *** bochecha has quit IRC | 01:30 | |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 03:54 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 03:54 | |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 04:27 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 04:27 | |
| *** semanticdesign has joined #buildstream | 08:34 | |
| *** semanticdesign has left #buildstream | 08:35 | |
| *** semanticdesign has joined #buildstream | 08:40 | |
| *** valentind has joined #buildstream | 08:50 | |
| *** starmad has joined #buildstream | 09:18 | |
| *** adds68 has joined #buildstream | 09:25 | |
| *** mathieu has joined #buildstream | 09:28 | |
| *** valentind has quit IRC | 09:30 | |
| *** jude has joined #buildstream | 09:39 | |
| *** jonathanmaw has joined #buildstream | 09:40 | |
| *** tiago has quit IRC | 10:15 | |
| *** tiago has joined #buildstream | 10:18 | |
| gitlab-br-bot | buildstream: merge request (flakes-refactor->master: Remove unused imports) #178 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/178 | 10:27 |
|---|---|---|
| *** ssam2 has joined #buildstream | 10:28 | |
| *** jude has quit IRC | 10:32 | |
| *** jude has joined #buildstream | 10:50 | |
| *** bethw has joined #buildstream | 10:57 | |
| gitlab-br-bot | buildstream: merge request (unused-vars-refactor->master: Refactor, removing unused variables) #177 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/177 | 11:18 |
| ssam2 | tlater, this is the glibc/musl bug : https://sourceware.org/bugzilla/show_bug.cgi?id=21604 | 11:22 |
| tlater | ta ssam2 | 11:22 |
| ssam2 | and here's a rather hacky script to scrape a whole mailman archive https://github.com/ssssam/dotfiles/blob/master/bin/fetch-list | 11:22 |
| ssam2 | edit the BASE_URL line to point to the correct place and it should just work | 11:22 |
| ssam2 | unless it doesn't | 11:22 |
| tlater | Hmm... The linux kernel config script has #!/bin/bash, which doesn't work in a busybox system | 12:05 |
| tlater | I find this works fine with /bin/ash | 12:06 |
| gitlab-br-bot | buildstream: merge request (sam/savefile-utility->master: utils.py: Add SaveFile utility class) #179 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/179 | 12:06 |
| tlater | But I'm not sure where I should put the sed line to fix that | 12:06 |
| tlater | Is there anywhere in the autoconf plugin that's more suitable than 'configure-commands'? | 12:06 |
| ssam2 | no | 12:06 |
| ssam2 | that's the best place | 12:07 |
| * tlater wonders if building linux always took this long | 12:25 | |
| paulsherwood | tlater: the first time i tried baserock, my machine was still running after 24 hours :) | 12:25 |
| tlater | Heh, baserock isn't just the linux kernel though ;) | 12:27 |
| paulsherwood | oh, well no then | 12:29 |
| paulsherwood | how long is it takign you? | 12:30 |
| tlater | 20 minutes so far | 12:30 |
| paulsherwood | ok, let me guess... | 12:30 |
| paulsherwood | are you using a VM? | 12:30 |
| tlater | Actually, yes I am | 12:30 |
| tlater | Good point | 12:30 |
| * tlater is also used to building on a far faster machine than this one | 12:30 | |
| tlater | Ah, done, finally! | 12:31 |
| paulsherwood | alright... next... how many cores does your VM have? | 12:31 |
| paulsherwood | and is your kernel build set to use them all? (ie -j) | 12:31 |
| tlater | Looks like 4, and yes | 12:31 |
| paulsherwood | and plenty of memory? | 12:33 |
| tlater | Yeah :) | 12:33 |
| paulsherwood | http://wiki.baserock.org/build-times/ | 12:33 |
| tlater | Seems reasonable: 00:18:45 [linux-x86-64-generic] | 12:34 |
| paulsherwood | i think that was an outlier | 12:34 |
| * paulsherwood hasn't done this for a while | 12:34 | |
| tlater | In that case, it's possible that buildstream is being slow... ssam2 has been running into issues with this as of late | 12:35 |
| ssam2 | linux can take a long time depending on config | 12:37 |
| ssam2 | if you copied the config Baserock uses, it enables loads of stuff and probably does take about 20 mins to build | 12:37 |
| ssam2 | you can get a way with a much more minimal config if you just want to boot a vm though ... maybe try just doing `make qemu_x86_defconfig` and just use that | 12:38 |
| paulsherwood | looking through https://github.com/devcurmudgeon/build-logs I think it was down to approx 4 minutes on aws, 10 mins on my macbook | 12:38 |
| tlater | I'll try with a more minimal config and see how long that takes | 12:40 |
| tlater | Ack, ssam2's config isn't in the upstream repo :| | 12:42 |
| ssam2 | ah, i just found it from googling | 12:42 |
| ssam2 | try a more detailed search for 'linux minimal config x86' and see what you dig up :-) | 12:42 |
| ssam2 | for some reason the bst-external plugin docs include a list of functions at the bottom | 13:05 |
| ssam2 | e.g. https://buildstream.gitlab.io/bst-external/bst_external.elements.dpkg_deploy.html#module-bst_external.elements.dpkg_deploy | 13:06 |
| ssam2 | where the buildstream plugins don't: https://buildstream.gitlab.io/buildstream/sources/git.html#module-sources.git | 13:06 |
| ssam2 | jonathanmaw, any idea why ? | 13:06 |
| ssam2 | i notice a bit of a difference in the doc/Makefile files between the two repos, but not sure if that's the cause | 13:06 |
| jonathanmaw | ssam2: hrm, not sure | 13:07 |
| ssam2 | seems the autogenerated .rst files differ ... in bst-external they get :members: and :undoc-members: and :show-inheritence: | 13:08 |
| ssam2 | where in buildstream they get none of that | 13:08 |
| jonathanmaw | ssam2: looks like the buildstream plugins are generated using plugin.rsttemplate, but the bst-external ones aren't | 13:10 |
| ssam2 | aha | 13:10 |
| ssam2 | seems create-element-skeletons function isn't actually being called | 13:11 |
| jonathanmaw | I think it might be being called incorrectly | 13:11 |
| jonathanmaw | I think the globbing faisl | 13:12 |
| jonathanmaw | fails | 13:12 |
| ssam2 | makes sense | 13:12 |
| ssam2 | i think the plugins should also be being ignored by sphinx-apidoc, but aren't being | 13:12 |
| ssam2 | which is why it works anyway | 13:12 |
| ssam2 | the buildstream/plugins dir doesn't have an __init__.py file and isn't a python package, so that's why sphinx-apidoc ignores them in the buildstream.git docs build | 13:13 |
| ssam2 | we should probably not be calling sphinx-apidoc at all for this module, since it doesn't have any public API other than the plugins | 13:14 |
| *** xjuan has joined #buildstream | 13:29 | |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 13:53 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 13:53 | |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 14:04 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 14:05 | |
| tlater | ^ Pretty creative way to spam | 14:05 |
| *** mcatanzaro has joined #buildstream | 14:06 | |
| ssam2 | sphinx never fails to confuse me | 14:48 |
| ssam2 | i have a link in index.rst to <bst_external.elements.dpkg_build> | 14:48 |
| ssam2 | and sphinx generates build/html/bst_external/elements/dpkg_build.html | 14:48 |
| ssam2 | which has a section with an id of "module-bst_external.elements.dpkg_build" | 14:49 |
| ssam2 | and yet ... | 14:49 |
| ssam2 | I get bst_external/elements/dpkg_build.rst: WARNING: document isn't included in any toctree | 14:49 |
| ssam2 | somehow that link in the index.rst isn't working, but no clue why | 14:50 |
| ssam2 | this is pretty much the same thing that works in buildstream.git's docs, but isn't working in bst-external | 14:50 |
| ssam2 | wait, it seems to actually be working now despite the warnings | 14:51 |
| ssam2 | well, that makes less sense than before, but at least it works | 14:52 |
| mathieu | is there a way to run a command inside a `bst shell --build element.bst` shell and have its result (e.g files created) exported out of that shell? | 15:09 |
| *** mathieu is now known as bochecha | 15:09 | |
| bochecha | (changed nick, not sure why Polari decided not to use the usual one) | 15:09 |
| tlater | bochecha: I just wanted the exact same functionality ;) - pretty sure there's nothing | 15:11 |
| ssam2 | i thought we added that already... | 15:15 |
| ssam2 | yeah, you can do `bst shell COMMAND` | 15:15 |
| ssam2 | or do you want some kind of automatic diff of files that were changed ? | 15:15 |
| bochecha | I'm getting to the point where my app (well, a PoC at the moment) can just be cloned by anybody, and there's no need to install a compiler, library headers, a virtualenv, ... just BuildStream then `bst build` / `bst shell`; the only thing I'm missing is for those commands like collecting the translation strings into a .pot file, which I can run in a bst shell, but then I can't get the | 15:16 |
| bochecha | generated pot file out of the sandbox | 15:16 |
| ssam2 | that's pretty cool | 15:16 |
| ssam2 | i don't do much with gettext to be honest; but is it not possible to have the translation step as an element ? | 15:17 |
| tlater | Hmm, are shell sandboxes stored to ~/.config/buildstream/build ? | 15:17 |
| bochecha | ssam2: it is, it has the potential to make first contributions immensely easier :) (in addition to not having to maintain virtualenvs and deps long term, rebuilding them on each developer workstation at each new Python, etc...) | 15:17 |
| ssam2 | yeah | 15:18 |
| ssam2 | i feel like in future we should have a higher level tool to enable this kind of thing ... | 15:18 |
| bochecha | ssam2: maybe, I didn't think about that.... but then that element will generate a pot file which will get pushed into buildstream's cache, how do I take it out and put it in my app source code? | 15:18 |
| ssam2 | well, you build your source code in an element too, right ? | 15:18 |
| bochecha | I do | 15:18 |
| ssam2 | elements can have multiple dependencies staged in different places, I thik | 15:18 |
| ssam2 | *think | 15:18 |
| ssam2 | so maybe you could stage the .pot on top of the source tree at buildtime somehow | 15:19 |
| bochecha | but the pot file needs to be put back into the source tree (the git repo) | 15:19 |
| ssam2 | for what purpose ? | 15:19 |
| ssam2 | i mean, i know it needs to be there once you run 'make' | 15:20 |
| bochecha | so that translators can find it and translate into their languages :) | 15:20 |
| ssam2 | aha, I see | 15:20 |
| ssam2 | that's a tricky one then, yeah | 15:20 |
| bochecha | but I don't want to run "make", because that requires having the deps installed on my laptop | 15:20 |
| ssam2 | self-modifying projects! | 15:20 |
| bochecha | which I'm trying to avoid | 15:20 |
| ssam2 | sure, you want to run make inside the sandbox | 15:20 |
| bochecha | exactly | 15:20 |
| ssam2 | but there must be a way to make the .pot file appear in the sandbox at the correct place before you run make | 15:20 |
| bochecha | having it appear in the sandbox is not a problem | 15:21 |
| bochecha | it's in the source git repo, so it gets taken in the sandbox like every other file | 15:21 |
| bochecha | the problem is updating that pot file, putting the new version in the git source tree | 15:21 |
| bochecha | that requires running some commands, which only exist in the build sandbox ): | 15:21 |
| bochecha | hence, generating the pot file in the build sandbox, but getting it back out to the source tree | 15:22 |
| ssam2 | right, ok | 15:22 |
| bochecha | maybe I should open an issue to discuss this better? | 15:23 |
| ssam2 | would `bst shell sh -c "generate Makefile.pot; cat Makefile.pot"` > Makefile.pot (or whatever) be enough ? | 15:23 |
| bochecha | afaik the output of the `cat` command won't be displayed by bst | 15:23 |
| ssam2 | hmm, that seems wrong | 15:24 |
| bochecha | bst displays its own output (which I wouldn't want) but only displays the commands it runs, not their output | 15:24 |
| bochecha | the output of the commands go to the log file | 15:24 |
| ssam2 | that's a bit useless in my opinion | 15:24 |
| ssam2 | there should definitely be a way to see the output | 15:24 |
| bochecha | yeah | 15:25 |
| bochecha | I often fail my builds on purpose, so that bst will ask me what to do and I can type "log" to see the outputs :P | 15:25 |
| ssam2 | i'm not so sure about showing build logs on stdout in general, just because they happen in parallel so it would all get mixed up together | 15:25 |
| ssam2 | but `bst shell` should definitely be scriptable | 15:26 |
| tlater | ssam2: Wasn't there a quiet mode specifically for this | 15:26 |
| tlater | *? | 15:26 |
| * tlater can't remember the exact incantations to run `bst shell` that way | 15:26 | |
| ssam2 | yes i thought we already added this | 15:27 |
| tlater | --no-interactive maybe? | 15:27 |
| tlater | `bst shell --no-interactive sh -c "generate Makefile.pot; cat Makefile.pot"` > Makefile.pot | 15:28 |
| ssam2 | yeah, when I run a command with `bst shell` I definitely see the output | 15:30 |
| * tlater wonders if this changes when output is redirected and/or --no-interactive is specified | 15:31 | |
| ssam2 | no way to hide the status output though | 15:31 |
| ssam2 | --no-interactive doesn't help | 15:31 |
| ssam2 | I think what we lack is the --quiet flag | 15:31 |
| ssam2 | for bochecha's use case | 15:31 |
| bochecha | the --no-interactive option was added recently? | 15:32 |
| bochecha | bst complains it doesn't exist, here :-/ | 15:32 |
| ssam2 | maybe | 15:32 |
| ssam2 | it doesn't help here anyway. I guess it's about prompting on failure, rather than hiding status messages | 15:32 |
| bochecha | ah, `bst --no-interactive shell ...` | 15:32 |
| bochecha | wrong order :) | 15:32 |
| bochecha | right, that works, I but I get the bst output added to the generated file | 15:36 |
| bochecha | should I send a MR adding a --quiet flag? | 15:37 |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 15:37 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 15:37 | |
| ssam2 | bochecha, if you could, that would be great ! | 15:47 |
| bochecha | or maybe sending all buildstream output to stderr, so that a simple `>` redirection doesn't catch it? | 15:47 |
| ssam2 | possible, yeah | 15:48 |
| ssam2 | personally i'd be in favour of all our status output going on stderr, not sure what others thing | 15:48 |
| ssam2 | *think | 15:48 |
| gitlab-br-bot | buildstream: merge request (optional-yaml-for-source-plugins->master: plugincontext: Let plugins not have YAML defaults) #180 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/180 | 15:48 |
| ssam2 | juergbi, what do you think about putting all bst status output onto stderr ? | 15:49 |
| gitlab-br-bot | buildstream: merge request (project-name->master: project: Export the name as a variable) #182 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/182 | 15:49 |
| bochecha | (just realized I had never opened the pull request on exporting the project name as a variable, even though I had pushed the code to a branch a while ago) | 15:50 |
| ssam2 | jonathanmaw, does https://gitlab.com/BuildStream/bst-external/merge_requests/12 look OK to fix the bst-external documentation ? | 15:52 |
| bochecha | ssam2: the patch to move everything to stderr is a one-liner :) | 15:55 |
| bochecha | so the only question is whether or not it's a good idea | 15:55 |
| ssam2 | bah, my docs branch was wrong | 15:55 |
| ssam2 | now actually working | 15:56 |
| ssam2 | bochecha, agreed ... if it's so simple, send an MR and we can discuss there :-) | 15:56 |
| * tlater dislikes the idea of throwing *all* output to the output labeled 'error' | 15:56 | |
| bochecha | writing the commit message right now :P | 15:56 |
| ssam2 | i consider the label "error" to mean "diagnostic/status messages" | 15:57 |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 15:57 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 15:57 | |
| tlater | What is "out" then? | 15:57 |
| ssam2 | and "output" to mean "actual output of the command", which is nothing for most buildstream commands | 15:57 |
| ssam2 | except for `bst show`, `bst shell` and maybe others | 15:57 |
| ssam2 | actually that's a point; `show` should be writing to stdout | 15:58 |
| ssam2 | the whole idea being that when running commands from scripts, you don't ever get status messages mixed up with the actual content | 15:59 |
| tlater | I think it would be better to send output from commands run inside a shell/build to a different file descriptor, rather than deciding that everything buildstream says should be considered a status message | 15:59 |
| ssam2 | not sure I follow, but I agree that not everything is a status message | 15:59 |
| tlater | Could we not just output messages from commands to one of the other 14 file descriptors? Or is this only a feature for bash scripts | 16:00 |
| ssam2 | however, can you think of stuff other than the output of `bst show` and `bst shell` that shouldn't be considered a status message ? | 16:00 |
| ssam2 | what unix programs send output to random file descriptors ? | 16:01 |
| tlater | I don't think I like your definition, to be frank. Most CLI applications I know use 'out' for status messages, and 'err' for status messages that indicate problems. | 16:01 |
| ssam2 | what's a "CLI application" ? | 16:01 |
| tlater | docker for example | 16:02 |
| ssam2 | right | 16:02 |
| ssam2 | but the point we're discussing is making bst commands usable from scripts | 16:02 |
| ssam2 | to do that, i'm saying we should behave more like normal programs from /usr/bin do | 16:02 |
| tlater | You wouldn't achieve that this way anyway, because build output would be random anyway | 16:02 |
| ssam2 | build output is totally offtopic here | 16:03 |
| tlater | What output is supposed to go to stderr then? | 16:03 |
| ssam2 | everything that we write to the console, apart from the output of shell commands run by `bst shell`, and text produced by `bst show` | 16:04 |
| ssam2 | build output goes to log files on disk, as before ... | 16:04 |
| tlater | So this is *only* up for discussion to ensure we can isolate `bst shell` output? | 16:04 |
| ssam2 | yes | 16:05 |
| tlater | I feel moving all output to a different descriptor just for that is a bit too heavy handed, especially because it breaks consistency with other applications IMO | 16:05 |
| ssam2 | ok | 16:05 |
| ssam2 | but can you think of a use case that would actually be broken by that change ? | 16:05 |
| tlater | Not really, no, but anyone trying to parse buildstream output will have a moment of wondering where their output goes. | 16:06 |
| tlater | I think moving `bst shell` command output to stderr is better | 16:07 |
| ssam2 | if anyone is currently trying to parse the mess of ANSI escape sequences that we output on stdout, i feel sorry for them | 16:07 |
| tlater | Those can be disabled with --no-color | 16:07 |
| juergbi | there shouldn't escape sequences by default for non-terminal output, right? | 16:07 |
| ssam2 | true | 16:07 |
| ssam2 | but also we could completely change how we output status messages and break their scripts | 16:07 |
| ssam2 | we don't declare our status output to be any point of stability | 16:08 |
| tlater | Why would we do this over a `--quiet` flag? | 16:08 |
| ssam2 | i'd be happy with a --quiet flag | 16:09 |
| bochecha | me too, it was merely a suggestion :) | 16:09 |
| juergbi | common practice is to output warnings and other log messages to stderr if also non-log output is produced (which then uses stdout). if there is no real output, stdout is commonly used for everything, i.e., the log messages, iirc | 16:09 |
| tristan | bst shell is an interesting case because it can output both stderr/stdout, and it's nicer to be able to provide both to a caller separately | 16:10 |
| juergbi | for bst shell stderr for bst logs makes sense to me. similar to ssh, we should aim to have a transparent shell | 16:10 |
| tristan | I think we discussed this before, and I'm in favor of putting everything logging related in stderr, except for a few things we want to be parsible output, like bst show output (not startup status), and `workspace list` output and such | 16:11 |
| tristan | but that doesnt solve the pretty particular case of `bst shell` | 16:12 |
| ssam2 | yeah, i can see you might want to capture stderr | 16:13 |
| juergbi | quiet by default makes sense to me for bst shell. and passing through stdout and stderr separately from the child process | 16:13 |
| juergbi | i.e., like ssh | 16:13 |
| ssam2 | although, the only reason you should want to capture it is to pass on errors to the user; so it might not be the end of the world if there was bst log output in there too | 16:13 |
| tristan | arguably, if things were to be well scripted *around* bst shell, the caller would want to have all three; --quiet might be undesirable | 16:13 |
| tristan | and, the caller would want to be able to inform buildstream what code the script wants buildstream to exit on, in the case of a buildstream runtime error | 16:14 |
| tristan | otherwise how do you distinguish programatically between a parse error, an error where the sandbox did not provide the exec or it's runtime deps, and an exit code reported by the target exec ? | 16:15 |
| * tristan thinks valgrind has weird opts like this | 16:15 | |
| jonathanmaw | ssam2: that fix looks good to me, and looks like it works when I build the documentation locally | 16:19 |
| * jonathanmaw hits the merge button | 16:19 | |
| ssam2 | thanks! | 16:19 |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 16:29 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 16:29 | |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 16:41 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 16:41 | |
| *** bochecha has quit IRC | 16:55 | |
| *** jonathanmaw has quit IRC | 17:12 | |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 17:17 | |
| *** Ya_ALLAH_Ya_Muhmd has left #buildstream | 17:17 | |
| *** noah has joined #buildstream | 17:29 | |
| *** jude has quit IRC | 17:33 | |
| ssam2 | another bst-external docs build breakage ... https://gitlab.com/BuildStream/bst-external/-/jobs/44044371 | 17:47 |
| ssam2 | seems 'make' vanished from the docker images | 17:58 |
| ssam2 | no idea why | 17:58 |
| ssam2 | oh well, we can fix the version for now | 17:59 |
| * tlater is getting kernel panics now ;-; | 18:00 | |
| tlater | Shouldn't have messed with the baserock config x) | 18:00 |
| ssam2 | it's very easy to build a broken linux | 18:01 |
| tlater | Yeah, I think I'm lacking ext4 drivers | 18:01 |
| * tlater might just go back, but is very happy with his 50MB image | 18:02 | |
| *** bethw has quit IRC | 18:03 | |
| *** ssam2 has quit IRC | 18:03 | |
| *** noah has quit IRC | 18:04 | |
| tlater | Yep, ext4 is built as a module but this kernel doesn't support modules | 18:08 |
| * tlater o\ | 18:08 | |
| *** valentind has joined #buildstream | 18:23 | |
| *** semanticdesign has quit IRC | 18:33 | |
| *** starmad has quit IRC | 20:29 | |
| *** Ya_ALLAH_Ya_Muhmd has joined #buildstream | 20:37 | |
| *** Ya_ALLAH_Ya_Muhmd has quit IRC | 20:42 | |
| *** xjuan has quit IRC | 20:47 | |
| *** jude has joined #buildstream | 21:19 | |
| *** starmad has joined #buildstream | 22:31 | |
| *** valentind has quit IRC | 23:12 | |
| *** jude has quit IRC | 23:18 | |
| *** jude has joined #buildstream | 23:24 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!