| *** tristan has quit IRC | 04:41 | |
| *** tristan has joined #buildstream | 05:15 | |
| *** ChanServ sets mode: +o tristan | 06:17 | |
| *** jonathanmaw has joined #buildstream | 08:03 | |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 10 commits (last: element.py: Renamed Element.fetch() to Element.pull()) https://gitlab.com/BuildStream/buildstream/commit/2a2f37f75a524c9e1accf3a152569d70c04be174 | 08:06 |
|---|---|---|
| *** tlater has joined #buildstream | 08:17 | |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 3 commits (last: _artifactcache/pushreceive.py: Our local copy of ostree-push) https://gitlab.com/BuildStream/buildstream/commit/1e619cd5683b89b1e83beae01022bccc3ff0768d | 08:18 |
| tlater | tristan: I'm not sure where to tie in the workspaces into the build process | 08:24 |
| tristan | :) | 08:26 |
| tristan | tlater, that means you're getting somewhere ! | 08:27 |
| tlater | Also stuck now ^^ | 08:27 |
| tristan | so there are a few details to work out | 08:27 |
| tristan | one of them is, choosing an active workspace instead of the regular source when building | 08:27 |
| tristan | I guess that comes first | 08:28 |
| tristan | Also, the cache key has to be calculated based on the workspace checksum instead of the real source checksum | 08:28 |
| tlater | Those things are mostly handled by the plugins, so it seems hard to add | 08:29 |
| tristan | tlater, lets start with that part, afterwards I think we'll want to encode some more metadata in the artifacts and taint the ones we've built with workspaces active, and any reverse dependency elements should be tainted as well if building in strong cache key calculation mode | 08:29 |
| tristan | Nobody said it was gonna be easy :) | 08:30 |
| tristan | tlater, I expect this will mostly be handled by understanding whether workspaces are active at the element.py level | 08:30 |
| tristan | Perhaps also at the source.py level, i.e. the base classes should take care of both issues above | 08:31 |
| tristan | So I guess it starts with "A source should know at all times if it's workspaced" | 08:31 |
| tristan | Then the source will derive it's cache key in a different way | 08:32 |
| tristan | and stage in a different way | 08:32 |
| tristan | if it's workspaced | 08:32 |
| tristan | The element may have some things to do if one or more of it's sources are workspaced | 08:32 |
| tlater | Why? | 08:33 |
| tristan | but I suspect that is unneeded until later | 08:33 |
| tristan | i.e. probably unneeded for routing of staging, and calculation of cache keys | 08:33 |
| tlater | Ah, alright | 08:33 |
| tristan | But needed for appropriately tainting of local artifacts (which we'll handle later) | 08:33 |
| tlater | So most of it can probably be done on the source level. I suppose I could add this to wherever the sources are parsed? | 08:34 |
| tristan | Ummm | 08:42 |
| tristan | tlater, Probably best to not bypass source paring I think | 08:43 |
| tlater | Right, then I won't have to fiddle with plugin implementations | 08:43 |
| tristan | rather, once we load up, the project knows what sources are workspaced | 08:43 |
| tristan | however interestingly yeah... the element.py will have to be the one to ask the workspace | 08:44 |
| tristan | because the source cannot know by itself how to ask the project if it's workspaced or not | 08:44 |
| tlater | I guess I can add that functionality to the get_sources() call, and simply return a workspace whenever we hit a source that's supposed to be a workspace. | 08:45 |
| tristan | tlater, so there are a couple of approaches to fixing that part... one might be to give sources internal unique identifier strings, like "<element-name>-<source-index>" | 08:45 |
| tristan | That is another idea indeed... but I wonder if we lose something by not handling it in Source directly | 08:47 |
| tristan | tlater, for instance we could be looking at later enhancements like; lets say that the project changes and there are active workspaces | 08:47 |
| tristan | Maybe we'll want to record in our internal state; what branch or url a workspace was activated for, and then tell the user that there might be an issue with an active workspace (i.e. project data no longer matches recorded workspace state) | 08:48 |
| tristan | Another rude approach would be to add some loop to the Pipeline after loading everything; telling each Source what their workspace is | 08:48 |
| tristan | not that bad either | 08:49 |
| tlater | The problem with that is that plugins will suddenly have to know about workspaces | 08:49 |
| tristan | I dont see how that is true | 08:49 |
| tristan | source.py would have to know | 08:49 |
| tristan | But then again, the abstract nature of source parameters in yaml make such a feature difficult yes | 08:50 |
| tristan | tlater, anyway; I think I feel more comfortable handling these things in Source object directly, the way the Source knows how to act can be either way | 08:55 |
| tristan | Can be Source._set_workspace(directory) API, called by the pipeline after loading | 08:56 |
| tristan | Or could be by implementing a unique key for sources to check themselves, but perhaps the former explicit codepath is easier to follow | 08:56 |
| tlater | Yeah, that makes sense. | 08:57 |
| tristan | I dont like the idea of having a "Something else" which handles source tasks completely separately from the sources | 08:57 |
| *** tiagogomes has joined #buildstream | 08:57 | |
| tlater | I also just realized that sources *do* have a private _stage method, so this doesn't have to leak into plugins | 08:57 |
| tristan | yeah | 09:06 |
| tristan | we have mostly wrappers around things that call into plugins | 09:06 |
| tristan | tlater, there was another approach I considered, which was to have "Controller" objects to interface with plugins | 09:07 |
| tristan | That would buy us some additional safety and clear up some additional private namespace for plugins | 09:07 |
| tristan | but | 09:07 |
| tristan | I thought that was just going too far | 09:07 |
| tlater | I prefer this, that much abstraction tends to just get confusing :) | 09:08 |
| tristan | yeah, and makes you look like that fanatic who spent way too much time reading Design Patterns, pulling out buzzwords every chance you get | 09:08 |
| tristan | so, I was too shy to be that guy, ya know ;-) | 09:09 |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 3 commits (last: _artifactcache/pushreceive.py: Our local copy of ostree-push) https://gitlab.com/BuildStream/buildstream/commit/b6d1a00ac088620b9314040f4c289234107f1d38 | 09:14 |
| *** violeta has quit IRC | 09:17 | |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 1 commit (last: Testing) https://gitlab.com/BuildStream/buildstream/commit/342a42328d69a834bfd4123e947c1257b0b492cf | 09:26 |
| *** ssam2 has joined #buildstream | 09:43 | |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 1 commit (last: Fixup) https://gitlab.com/BuildStream/buildstream/commit/f564a1c6145652c34722579d1002062f024e3682 | 09:47 |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 4 commits (last: _artifactcache/pushreceive.py: Our local copy of ostree-push) https://gitlab.com/BuildStream/buildstream/commit/60e65d4a830631dd561b8b8e1430327968763dce | 10:15 |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 3 commits (last: _artifactcache/pushreceive.py: Our local copy of ostree-push) https://gitlab.com/BuildStream/buildstream/commit/d30feff94c012c0319c01b4c0300add7fe2c8a59 | 10:17 |
| *** jonathanmaw has quit IRC | 10:32 | |
| *** jonathanmaw has joined #buildstream | 10:35 | |
| tristan | Ok so, maybe someone can help me figure this out | 11:03 |
| tristan | I had previously, concurrent pushes working well; but no way to gracefully terminate the remote ssh process | 11:03 |
| tristan | Adding `ssh -tt` fixes this; and when killing the local ssh process with SIGTERM, the remote process launched by ssh receives SIGHUP, handling SIGHUP I can do some cleanup and exit gracefully | 11:04 |
| tristan | The -tt option; forces a terminal | 11:05 |
| *** jonathanmaw has quit IRC | 11:05 | |
| *** jonathanmaw has joined #buildstream | 11:05 | |
| juergbi | what exactly happens with concurrent pushes and -tt? | 11:05 |
| tristan | So, maybe that in itself is an issue; because I'm communicating with the foreign process via it's stdin/stdout | 11:05 |
| tlater | Building with workspaces looks like it's working, outside of caching, btw \o/ | 11:06 |
| tristan | With concurrent pushes, and actually I'm not sure if it's only concurrent; I'm starting to suspect it has to do with data being sent... | 11:06 |
| tristan | I get this: ^Z escape not available to multiplexed sessions | 11:06 |
| tristan | Or sometimes: ~& escape not available to multiplexed sessions | 11:06 |
| juergbi | ah, it's no longer a transparent shell? | 11:06 |
| tristan | Or: ~C escape not available to multiplexed sessions | 11:07 |
| juergbi | i.e., binary data could mess things up as they are interpreted as escape codes | 11:07 |
| tristan | thats exactly what I'm starting to suspect yes | 11:07 |
| tristan | binary data over stdin, being treated as a terminal: boom | 11:07 |
| juergbi | https://www.cyberciti.biz/faq/openssh-linux-unix-osx-kill-hung-ssh-session/ | 11:07 |
| juergbi | "When a pseudo-terminal has been requested, ssh supports a number of functions through the use of an escape character." | 11:08 |
| tristan | Mhmmm | 11:08 |
| tristan | So I guess there are some options | 11:08 |
| tristan | An annoying one, would be to have some stream translator which escapes the binary data ? | 11:09 |
| tristan | So that if those control chars are encountered, they get escaped and then properly interpreted by bst-artifact-receive ? | 11:09 |
| tristan | Ideally, we dont use pseudo-terminal, and find a "proper" way to send sighup | 11:10 |
| tristan | I would think, closing the remote's stdin might do such a thing | 11:10 |
| juergbi | if it doesn't attempt to read anything, i don't think anything will happen | 11:12 |
| tristan | Well, it will try to read something | 11:12 |
| tristan | the bst-artifact-receive() will read until end of input | 11:13 |
| juergbi | right, i guess in this paticular case, it will never take long until the next read | 11:13 |
| juergbi | shouldn't the remote process actually receive SIGPIPE when closing ssh connection without -tt? | 11:14 |
| juergbi | or might python disable SIGPIPE? | 11:14 |
| tristan | Maybe yeah | 11:14 |
| tristan | broken pipe yeah | 11:15 |
| juergbi | but even if it's disabled, next read should return EPIPE | 11:15 |
| juergbi | the receive process just stayed around forever or what happened exactly? | 11:15 |
| juergbi | (possibly worth checking with strace if it's not obvious) | 11:15 |
| tristan | That will raise a python BrokenPipeError | 11:15 |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 1 commit (last: Testing) https://gitlab.com/BuildStream/buildstream/commit/84422bb168c483ccaba9ac12c71b36a74d4f91b5 | 11:19 |
| juergbi | tristan: alternatively, you could try ssh -tt -o "EscapeChar none" | 11:20 |
| juergbi | there is even a shortcut. ssh -tt -e none | 11:21 |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 1 commit (last: Testing) https://gitlab.com/BuildStream/buildstream/commit/4f28c251616b518f16bf2cebb5700a6aad985040 | 11:22 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 1 commit (last: Tie workspaces into sources) https://gitlab.com/BuildStream/buildstream/commit/c714211da6262d12f40abedf73c6e9e4b9d319e9 | 11:31 |
| gitlab-br-bot | push on buildstream@artifacts-ostree-push (by Tristan Van Berkom): 1 commit (last: Testing) https://gitlab.com/BuildStream/buildstream/commit/44fcce430a44f9f213328df5c191de8c37e094b5 | 11:31 |
| tristan | Ohhhh | 11:32 |
| tristan | Ok maybe I'll do that instead | 11:32 |
| tristan | but this way seems a bit cleaner; I still have the old -tt in place so lets see | 11:32 |
| tristan | It could be the control characters are what is used to send the SIGHUP ? but I'm not seeing it | 11:33 |
| tristan | I certainly dont understand this thoroughly haha | 11:33 |
| tristan | Nice | 11:36 |
| tristan | Ok that works | 11:36 |
| tristan | just close the connection | 11:36 |
| tristan | And in this case I've used just a blind "except:" in the bst-artifact-receive code, if any exception at all bubbles up while receiving objects; we cleanup and re-raise the exception | 11:37 |
| tristan | it gets cleaned up anyway | 11:37 |
| juergbi | ok | 11:39 |
| * tristan is letting it run and push some artifacts concurrently before merging | 11:41 | |
| tristan | but, I'm sure it will work... | 11:41 |
| tristan | I did a full build of ~170 artifacts yesterday | 11:41 |
| tristan | also tried aborting half way and retry build, so I know the pulled stuff is correct too | 11:41 |
| juergbi | sounds good | 11:42 |
| tristan | Unfortunately downloading the 1.2GB artifact over http is still slow ;-) | 11:43 |
| tristan | but meh, not too bad, it takes around an hour | 11:43 |
| tristan | (and probably much quicker from EU) | 11:43 |
| tristan | mirroring artifact caches at least for readonly is something we can/should eventually consider | 11:44 |
| ssam2 | that's about how long the base SDK import seems to take | 11:44 |
| tristan | ssam2, right, you are talking about the base freedesktop SDK ? | 11:44 |
| ssam2 | yeah | 11:44 |
| tristan | I'm not sure where sdk.gnome.org is hosted, but if it's US, then latency is also playing a role I suspect | 11:45 |
| tristan | ssam2, maybe you can try building https://gnome7.codethink.co.uk/gnome-modulesets.git :) | 11:45 |
| tristan | see if the artifact cache is faster for you | 11:45 |
| tristan | when downloading from manchester | 11:46 |
| tristan | anyway give me some time to land the branch first | 11:46 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 15 commits (last: context.py & userconfig.yaml: New syntax for remote artifact caches) https://gitlab.com/BuildStream/buildstream/commit/66fa3a11db6faff0620f3dc62c8d05c9f71f8734 | 12:00 |
| tristan | hilarious, most any artifact takes seconds to push | 12:00 |
| tristan | except base-system.bst and base-configure.bst | 12:01 |
| tristan | alright... I've landed it | 12:01 |
| tristan | juergbi, this is possibly going to conflict anything you may have changed in artifactcache | 12:01 |
| tristan | s/possibly/definitely | 12:01 |
| tristan | I've moved _artifactcache.py -> _artifactcache/artifactcache.py | 12:02 |
| tristan | And added the push/pull stuff in there | 12:02 |
| tlater | Alright, looks like workspaces are functional | 13:05 |
| tristan | :) | 13:05 |
| tristan | Sweeeet ! | 13:05 |
| tristan | ok time to spare ! | 13:05 |
| tlater | I've put up a MR for my testing branch | 13:05 |
| tlater | https://gitlab.com/BuildStream/buildstream-tests/merge_requests/10 | 13:05 |
| tristan | looks like the runners are under pressure | 13:06 |
| tlater | yeah | 13:06 |
| tlater | That test will probably fail anyway, both file permissions and buildstream master will have trouble. But I can only fix that if the actual feature is merged. | 13:06 |
| tristan | tlater, I dont think that is the right url you pasted | 13:06 |
| tristan | is there a buildstream branch ? | 13:06 |
| tlater | Oh, yeah, I need to put up that MR too | 13:07 |
| tlater | https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 13:07 |
| * tristan just pushed documentation for artifact caches, but is waiting for the runners to update the website | 13:07 | |
| tlater | Forgot to fetch... | 13:10 |
| gitlab-br-bot | buildstream: merge request (jonathan/dpkg-build->master: WIP: Jonathan/dpkg build) #37 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/37 | 13:24 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Adding documentation about setting up artifact caches.) https://gitlab.com/BuildStream/buildstream/commit/8764d1ceacae942c4208807847aaad9d3ef42a23 | 13:27 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 13:27 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 20 commits (last: utils.py: Adding _tempdir() utility context manager) https://gitlab.com/BuildStream/buildstream/commit/e5b21704e9ed68f29590f428ca70b7c97ae7925a | 13:32 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 13:32 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 13:34 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 6 commits (last: main.py: Add workspace commands) https://gitlab.com/BuildStream/buildstream/commit/6f61c9824a60d121df89f72ff578fbf5c20e722e | 13:34 |
| gitlab-br-bot | push on buildstream@testing (by Tristan Van Berkom): 1 commit (last: bst-artifact-receive: Use click instead of argparse) https://gitlab.com/BuildStream/buildstream/commit/b8fe52f72afc0b7973f3d207f6aebb0f427246d5 | 13:34 |
| tlater | I wonder if it would make sense for the ci repo to checkout a branch with the same name from the main repo | 13:36 |
| tlater | That way there's no awkward conversion test failure | 13:36 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 4 commits (last: main.py: Add workspace commands) https://gitlab.com/BuildStream/buildstream/commit/355b9a31c45fb0588930ee88680084d126267176 | 13:36 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 13:36 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: bst-artifact-receive: Use click instead of argparse) https://gitlab.com/BuildStream/buildstream/commit/19855f4361aadfdfa9106d5f5c8be5ee17ee7e49 | 13:36 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 5 commits (last: bst-artifact-receive: Use click instead of argparse) https://gitlab.com/BuildStream/buildstream/commit/19855f4361aadfdfa9106d5f5c8be5ee17ee7e49 | 13:36 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 13:36 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch testing | 13:36 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch artifacts-ostree-push | 13:36 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Regenerating man pages) https://gitlab.com/BuildStream/buildstream/commit/20679de73d345c53b9a5466a63082db917b8445a | 13:37 |
| tristan | tlater, did you break pages ? | 13:43 |
| tristan | with the .gitlab-ci.yml ? | 13:43 |
| tlater | I don't think so, perhaps dependencies are missing with the new image? | 13:43 |
| tristan | I dont think so either | 13:44 |
| tristan | maybe gitlab is going nuts | 13:44 |
| tlater | Possible, there's a huge amount of lag on the bot, too | 13:44 |
| tristan | "missing pages artifacts" | 13:45 |
| tristan | weird | 13:46 |
| tristan | I really want pages updated now :-/ | 13:46 |
| tlater | Why are pages in the test stage? | 13:46 |
| tristan | I dont know about that | 13:47 |
| tristan | I never added "stages" | 13:48 |
| tlater | I suppose the default is just test then | 13:48 |
| tristan | I dont know if they run concurrently against the same backing filesys either | 13:48 |
| tristan | I have absolutely no clue, about any of that | 13:49 |
| tlater | They don't | 13:49 |
| tristan | it just works | 13:49 |
| tristan | until now | 13:49 |
| tlater | If they run concurrently they use independent systems | 13:49 |
| tlater | But the test should just create documentation normally as far as I can tell. | 13:49 |
| tristan | /bin/sh: find: command not found | 13:50 |
| tlater | Ah | 13:50 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/jobs/21854674 | 13:50 |
| tristan | wtf ? | 13:50 |
| tlater | Yeah, it's a different image | 13:50 |
| tlater | find needs to be installed manually... | 13:50 |
| tristan | tlater, can you immediately fix that ? | 13:51 |
| tlater | Yeah, will do | 13:51 |
| gitlab-br-bot | push on buildstream@patch-1 (by Tristan Maat): 1 commit (last: Fix missing find command in pages script) https://gitlab.com/BuildStream/buildstream/commit/fd01d32dfb6dd412b7eb883cf25464a7c3e41a6c | 13:52 |
| tlater | That should hopefully do it | 13:52 |
| tristan | Any chance I could get pages published within 30 minutes so that I can send a link to the new docs with an email to baserock-dev ? | 13:53 |
| tristan | it's 11pm here :-/ | 13:53 |
| tlater | If you merge that, yes | 13:53 |
| gitlab-br-bot | buildstream: merge request (patch-1->master: Fix missing find command in pages script) #51 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/51 | 13:54 |
| tlater | Sorry, merge request didn't work. | 13:54 |
| gitlab-br-bot | buildstream: merge request (patch-1->master: Fix missing find command in pages script) #51 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/51 | 13:55 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Fix missing find command in pages script) https://gitlab.com/BuildStream/buildstream/commit/fd01d32dfb6dd412b7eb883cf25464a7c3e41a6c | 13:55 |
| tristan | merged | 13:55 |
| tristan | ahead of pipeline | 13:55 |
| tlater | I wonder why the missing find command didn't cause the pipeline to fail | 13:56 |
| tristan | we probably are not using it ? | 13:58 |
| tristan | Why would BuildStream need find ? | 13:59 |
| tristan | Maybe it's something you might want to use in the test cases, though | 13:59 |
| tristan | Ok well that *helped* | 13:59 |
| tristan | But still pages is not deploying | 13:59 |
| tlater | Python is missing? | 13:59 |
| tlater | /bin/sh: python: command not found | 13:59 |
| tlater | But that doesn't make any sense | 14:00 |
| tlater | python3 problem? | 14:00 |
| tristan | tlater, ok so... it needs python | 14:02 |
| tristan | how do I do that | 14:02 |
| tlater | Python is installed | 14:02 |
| tristan | does it have python ? | 14:02 |
| tlater | Yup | 14:02 |
| tristan | or only python3 ? | 14:02 |
| tlater | Not sure, that may be it | 14:02 |
| tristan | It needs /usr/bin/python | 14:02 |
| tlater | ln -s? | 14:02 |
| tristan | there is some ugly hack in doc/ | 14:03 |
| tristan | nah | 14:03 |
| tristan | need python | 14:03 |
| tristan | real | 14:03 |
| tristan | gimme python | 14:03 |
| tristan | We can fix it later | 14:03 |
| tristan | if it's possible | 14:03 |
| tlater | ssam2, how does this image's python thing work? | 14:03 |
| tlater | Best I can think of is quickly adding ln -s /usr/bin/python3 /usr/bin/python | 14:03 |
| tristan | sigh | 14:03 |
| tristan | wont be correct I think | 14:04 |
| tristan | hack is weird | 14:04 |
| tristan | it wants to ask /bin/python about it's version | 14:04 |
| tristan | and explicitly run a sphinx-build3 script that we have | 14:04 |
| tristan | freaking weird | 14:04 |
| tristan | python | 14:04 |
| tristan | I think it comes from andrew leeming | 14:05 |
| tristan | maybe he had a weird system | 14:05 |
| tlater | Fedora apparently just has python | 14:05 |
| tlater | So this should work | 14:05 |
| tristan | why did we change the image ? | 14:07 |
| tristan | I dont remember any pull request for this, it was working before wasnt it ? | 14:07 |
| tlater | Because of the strange tempdir images, this was part of adding the CI | 14:07 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Weird ln -s python try to fix damn docs build) https://gitlab.com/BuildStream/buildstream/commit/d8f27901d95b3dcb509ec8567e64cf1a10f464ee | 14:07 |
| tlater | I didn't notice the pages were failing | 14:07 |
| tristan | I mean, they worked recently enough | 14:08 |
| tristan | after your test cases merge | 14:08 |
| tristan | did the image change after that ?? | 14:08 |
| tlater | Not since the CI changes | 14:08 |
| tristan | maybe a test cases fix ? | 14:08 |
| tlater | Nope, not at all | 14:08 |
| tristan | that ln -s is... reaaaly bad | 14:10 |
| tristan | But pages still wont deploy https://gitlab.com/BuildStream/buildstream/pipelines/9731692 | 14:10 |
| tlater | There are a bunch of warnings about .rst files not being included | 14:10 |
| ssam2 | i can't quite figure out the context here | 14:11 |
| jonathanmaw | tristan: I'm having an issue fetching sources which seems to be related to https://gitlab.com/BuildStream/buildstream/commit/e5b21704e9ed68f29590f428ca70b7c97ae7925a#e3712df8ffb834c44cde88704818e3e4da3918c3_662_682 traceback at https://pastebin.com/UesadLuZ | 14:11 |
| ssam2 | oh I see, .gitlab-ci.yml is now running the docs build inside the Docker image I created for running BuildStream | 14:11 |
| tlater | Yup | 14:12 |
| ssam2 | which I guess is unavoidable if we want to run the integration tests from that too | 14:12 |
| tlater | Actually, maybe not | 14:12 |
| tlater | I think we can specify individual images per job | 14:12 |
| ssam2 | ah cool | 14:12 |
| ssam2 | that should be a simple fix then | 14:12 |
| tristan | :'( | 14:12 |
| ssam2 | although the image should be more or less stock Fedora 25 | 14:12 |
| tlater | Lemme just check docs... | 14:12 |
| ssam2 | but that probably doesn't have python2, and may also not have /usr/bin/python (As the python2 package provides that) | 14:13 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Revert "Weird ln -s python try to fix damn docs build") https://gitlab.com/BuildStream/buildstream/commit/85a3bde44710c3c8973ad0fd07b2c71cd834dba8 | 14:13 |
| * tristan revers the horrible ln -s | 14:13 | |
| tlater | Alright, yeah, apparently that's possible | 14:14 |
| tlater | What was the old image? | 14:14 |
| tlater | Oh, it didn't have one :/ | 14:15 |
| jonathanmaw | Is anyone else able to fetch git sources? | 14:15 |
| jonathanmaw | is the problem caused by me having old versions of stuff again? | 14:16 |
| tlater | tristan: The only fix I can think of is installing python2 along with findutils, just to do what that hack wants us to do. | 14:17 |
| tristan | tlater, yeah python2 is /usr/bin/python | 14:18 |
| tristan | I thought you said fedora only has 'python' | 14:18 |
| tristan | How do I do it ? | 14:18 |
| tristan | anyway it seems it's not the problem | 14:18 |
| tlater | add python2 next to findutils | 14:18 |
| tristan | just 'python2' ? | 14:18 |
| tristan | not 'fancy-weird-package-name-python2' ? | 14:18 |
| tristan | heh | 14:18 |
| tlater | I think so | 14:18 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Adding real python2 for pages build) https://gitlab.com/BuildStream/buildstream/commit/5be75163ab3aa59c1adcb7ad39751e1da6cf918b | 14:19 |
| tristan | hmm | 14:19 |
| tlater | If that doesn't work, apparently the default tag is ruby:2.1, but that will require adding a few of the old hacks to get it to run | 14:20 |
| tlater | :/ | 14:22 |
| tlater | What do the warnings mean? | 14:23 |
| tlater | Perhaps the sphinx version changed, and that is causing issues | 14:23 |
| tristan | didnt work | 14:23 |
| tristan | No | 14:23 |
| tristan | the pages are getting generated fine | 14:23 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/jobs/21859238 | 14:24 |
| tristan | But: https://gitlab.com/BuildStream/buildstream/pipelines/9732165 | 14:24 |
| tristan | What the helll | 14:24 |
| tlater | Not a CI issue then? | 14:24 |
| tristan | Uploading artifacts to coordinator... ok | 14:24 |
| tristan | "missing pages artifacts" | 14:24 |
| tristan | :-/ | 14:24 |
| tristan | Maybe gitlab is just broken | 14:24 |
| tristan | ugh | 14:25 |
| tlater | Should I try restoring that job to what the default stuff was before? | 14:26 |
| tristan | tlater, try in a branch, and in that branch you can remove the 'only:\n-master' bit | 14:29 |
| tristan | but I doubt that is what's happening, I suspect rather it's gitlab that is fucked | 14:30 |
| tlater | If anything, it's the deploy server. I assume that's managed by codethink? | 14:30 |
| tristan | wait | 14:30 |
| tristan | https://gitlab.com/BuildStream/buildstream/pipelines/9727228 | 14:30 |
| tristan | This one succeeded | 14:31 |
| tristan | it was from 2 hours ago master | 14:31 |
| tristan | before I added the artifacts docs | 14:31 |
| tristan | think it's something as stupid as... the output now contains 'artifacts.html' ? | 14:31 |
| tlater | Hm, well, at least it means the current setup isn't the problem | 14:34 |
| tristan | Deploy passed 2 hours ago means exactly that | 14:34 |
| gitlab-br-bot | push on buildstream@pages-test (by Tristan Van Berkom): 2 commits (last: Docs: Moving artifacts.rst -> artifactshare.rst) https://gitlab.com/BuildStream/buildstream/commit/d43490b33102d13ab0c24d3729f1312633fba581 | 14:34 |
| tristan | ugh | 14:38 |
| tristan | ok well | 14:38 |
| tristan | that didnt fix it | 14:38 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch pages-test | 14:38 |
| tristan | Anyway the findutils was an important fix | 14:40 |
| tristan | As you can see at the last deployment: https://buildstream.gitlab.io/buildstream/ | 14:40 |
| tristan | All of the plugin links are broken | 14:40 |
| tristan | Because `find` was missing when we tried to generate their templates | 14:40 |
| tlater | Oh, darn | 14:40 |
| tristan | docs remain broken until... I dont know | 14:40 |
| gitlab-br-bot | push on buildstream@pages-test-before (by Tristan Van Berkom): 1 commit (last: Test) https://gitlab.com/BuildStream/buildstream/commit/4801f74f6a710875a9674df297d1d5390b63ad5b | 14:42 |
| gitlab-br-bot | push on buildstream@pages-test-after (by Tristan Van Berkom): 1 commit (last: Testing) https://gitlab.com/BuildStream/buildstream/commit/5ecd978ff70bb75f0d3d5ac6dfc01b21470a096e | 14:42 |
| tristan | I'm retrying the last commit which passed | 14:43 |
| tristan | and then retrying the following commit, which literally is the one which adds the docs I want to add | 14:43 |
| tristan | nope | 14:45 |
| tristan | Exactly the same commits | 14:45 |
| tristan | Dont deploy anymore | 14:45 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch pages-test-after | 14:46 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch pages-test-before | 14:47 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch patch-1 | 14:47 |
| tlater | That's a really strange issue... | 14:48 |
| jonathanmaw | aha! mkdtemp is misbehaving for me because python3.5 changed the default values for prefix and suffix. | 14:48 |
| jonathanmaw | tristan: Do we want to keep support for python3.4? If so, I think utils._tempdir will need to be changed :/ | 14:49 |
| tristan | Oh ? | 14:55 |
| tristan | tempfile.mkdtemp() ? | 14:56 |
| tristan | Some of the options are from 3.5 ? | 14:56 |
| jonathanmaw | tristan: yep | 14:56 |
| tristan | jonathanmaw, we want prefix and dir, dont care for suffix really... what's new ? | 14:56 |
| jonathanmaw | in 3.4 suffix defaults to "", and prefix defaults to "tmp" | 14:56 |
| jonathanmaw | in 3.5 they both default to None, iiuc | 14:56 |
| tristan | Ummm | 14:56 |
| tristan | in which case, we dont really care | 14:56 |
| tristan | jonathanmaw, does this cause anything to break ? | 14:57 |
| jonathanmaw | tristan: yes. I can't fetch git sources (and probably other stuff). | 14:57 |
| tristan | jonathanmaw, ok so then... force default prefix to "tmp" ? | 14:57 |
| tristan | should fix it | 14:57 |
| tristan | jonathanmaw, if that works, feel free to push it directly to master | 14:58 |
| tlater | tristan: Anything I can do, considering you will probably only manage to look at my MR tomorrow? | 14:58 |
| jonathanmaw | tristan: fetching is happening now, no sign of further crashes yet. | 14:59 |
| tristan | tlater, I didnt have time to look today, was quite stressful, leaving now | 15:02 |
| tlater | I figured :) | 15:03 |
| tristan | tlater, if you *really* think your branch is ready (without metadata considerations) and fault tolerant (handles cases where you try to open workspaces on things that are not fetched yet, _elegantly_ and with no user frustration)... | 15:03 |
| tristan | tlater, then you can try to nail the rest of your work on https://gitlab.com/BuildStream/buildstream/issues/24 | 15:03 |
| * tristan hits the road for the night | 15:03 | |
| tlater | o/ | 15:03 |
| tristan | \o | 15:04 |
| jonathanmaw | o/ | 15:04 |
| *** tristan has quit IRC | 15:07 | |
| gitlab-br-bot | push on buildstream@master (by Jonathan Maw): 1 commit (last: utils.py: Fix _tempdir for python3.4) https://gitlab.com/BuildStream/buildstream/commit/eecec59a93fba04fd9b84e474cf4237aea35ff16 | 15:11 |
| *** jude has quit IRC | 15:13 | |
| *** tristan has joined #buildstream | 15:19 | |
| *** jude has joined #buildstream | 15:31 | |
| jonathanmaw | hrm, the fix to _tempdir seems to come with a failure in CI on pages:deploy, but I don't have permission to view the problem. | 15:33 |
| jonathanmaw | I trust it's benign, because my build is still continuing | 15:33 |
| tlater | That's the issue we were trying to fix earlier, it's nothing to do with your changes | 15:37 |
| jonathanmaw | ok | 15:37 |
| *** jude_ has joined #buildstream | 15:45 | |
| *** jude has quit IRC | 15:45 | |
| *** jonathanmaw has quit IRC | 16:08 | |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 12 commits (last: Regenerating man pages) https://gitlab.com/BuildStream/buildstream/commit/20679de73d345c53b9a5466a63082db917b8445a | 16:19 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 16:19 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 2 commits (last: main.py: Deal with arch changes) https://gitlab.com/BuildStream/buildstream/commit/1e67ef33a12fe92cf36314088719cf05868f46c5 | 16:19 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 16:19 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 1 commit (last: _pipeline.py: Deal with tracking and source consistency) https://gitlab.com/BuildStream/buildstream/commit/c4f8c5a6bebae5474b72dbd752260e6200570d6b | 16:23 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 16:23 |
| *** jude_ has quit IRC | 16:28 | |
| *** jude__ has joined #buildstream | 16:28 | |
| *** tlater has quit IRC | 16:49 | |
| *** jude__ has quit IRC | 17:34 | |
| tristan | fwiw: https://gitlab.com/gitlab-org/gitlab-ce/issues/34899 | 17:47 |
| *** ChanServ sets mode: +o tristan | 17:50 | |
| *** ssam2 has quit IRC | 18:57 | |
| *** jude has joined #buildstream | 19:22 | |
| *** tristan has quit IRC | 20:49 | |
| *** jude has quit IRC | 21:07 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!