| *** xjuan has joined #buildstream | 00:34 | |
| *** xjuan has quit IRC | 03:11 | |
| *** tristan has quit IRC | 05:01 | |
| *** tristan has joined #buildstream | 06:14 | |
| *** tristan has quit IRC | 06:18 | |
| *** tristan has joined #buildstream | 06:22 | |
| *** tristan has quit IRC | 06:26 | |
| *** brejoc has joined #buildstream | 07:21 | |
| *** tristan has joined #buildstream | 07:35 | |
| *** tristan has quit IRC | 07:39 | |
| *** franred has joined #buildstream | 07:56 | |
| *** tristan has joined #buildstream | 08:18 | |
| *** tristan has quit IRC | 08:20 | |
| *** tristan has joined #buildstream | 08:20 | |
| *** ChanServ sets mode: +o tristan | 08:20 | |
| Chris | Oops, apologies tristan, I was afk by that stage. The output contained gaps, nothing missing there. | 08:39 |
|---|---|---|
| Chris | I'll try the build again and let you know how it gets on. | 08:39 |
| *** jonathanmaw has joined #buildstream | 08:40 | |
| tristan | Chris, no worries, I think I need to fix that line as I described, ENOTEMPTY doesnt look right | 08:51 |
| Chris | Hm, it's failed again, but when trying to extract a glibc artifact. | 09:15 |
| Chris | http://paste.baserock.org/owonecehun | 09:15 |
| tristan | Chris, this is interesting that you are encountering this consistently and I have never | 09:16 |
| tristan | Chris, can you try modifying the line as I suggested ? | 09:17 |
| Chris | Sure, give me a sec. | 09:17 |
| tristan | Chris, after that... one more thing before relaunch | 09:17 |
| tristan | Chris, rm -rf ~/.cache/buildstream/artifacts/extract | 09:18 |
| tristan | That will ensure the extracts happen again | 09:18 |
| tristan | to help provoke the problem happening again | 09:18 |
| tristan | (although it wont provoke rebuilds of what you've already built) | 09:18 |
| Chris | Sure. You're suggesting changing ENOTEMPTY to EEXISTS, yes? | 09:19 |
| tristan | Exactly | 09:20 |
| tristan | os.rename is trying to place the extracted artifact to a location that exists | 09:20 |
| tristan | if it already exists, it means another process beat that one to the punch | 09:21 |
| tristan | which is fine | 09:21 |
| Chris | Ok, will keep you informed. | 09:22 |
| tristan | Chris, thanks :) | 09:23 |
| Chris | It's thrown this almost immediately, just reading through it now. No build failure, but it's been at this point for about 4 and a half mins thus far. http://paste.baserock.org/doluqinege | 09:27 |
| tristan | Chris, ok the frontend thing I will be fixing today... | 09:30 |
| tristan | But that's a symptom that shouldnt happen anyway | 09:31 |
| tristan | Now that is a bit weird, about the exception | 09:31 |
| tristan | Oh | 09:31 |
| tristan | Chris, there is no S in EEXIST actually | 09:32 |
| tristan | my bad | 09:32 |
| Chris | Haha no problem. | 09:32 |
| tristan | well, only one S :) | 09:32 |
| tristan | no trailing one | 09:32 |
| *** ssam2 has joined #buildstream | 09:32 | |
| Chris | 4th time's a charm. | 09:33 |
| tristan | Chris, Ok great, lets apply that change then | 09:34 |
| jonathanmaw | tristan: have you had a chance to reivew my new merge request? | 09:34 |
| Chris | It's building now, will keep you posted. | 09:34 |
| tristan | jonathanmaw, sorry, I should have by now, have been trying to wrap up and do something with all my lost time swimming in pip/venv yesterday :-S | 09:35 |
| tristan | jonathanmaw, give me one hour to wrap up and most probably merge your branch by then | 09:35 |
| tristan | jonathanmaw, in the meantime... please look into the bzr source :D | 09:36 |
| *** ssam2 has quit IRC | 09:36 | |
| tristan | with that, I think we'll have all sources needed to auto-convert flatpak json | 09:36 |
| *** ssam2 has joined #buildstream | 09:38 | |
| tristan | Chris, would you like to file an MR for that or should I just commit / push the change ? | 09:46 |
| Chris | If you have it ready to go, fire away, otherwise I can do it now | 09:47 |
| tristan | Chris, sure | 09:48 |
| tristan | just testing that the change doesnt break anything here for any reason and pushing that | 09:49 |
| Chris | No probs. It's building ok here thus far. | 09:49 |
| tristan | yeah, I wonder if here it never triggered because lightning fast SSD | 09:50 |
| tristan | but still this is curious | 09:50 |
| tristan | Chris, according to `man 2 rename`: | 09:51 |
| tristan | ENOTEMPTY or EEXIST | 09:52 |
| tristan | newpath is a nonempty directory, that is, contains entries other than "." and "..". | 09:52 |
| tristan | Chris, so I suppose the error chosen may be OS / condition dependent | 09:52 |
| tristan | We should handle both in this case | 09:52 |
| * jonathanmaw starts reading how bzr works | 09:53 | |
| Chris | It seems like it. I'll have a read into it. | 09:55 |
| tristan | yeah, I suspect my OS/libc (and juergbi's) has been reporting ENOTEMPTY, while yours decided on EEXIST for some reason | 09:56 |
| tristan | But its documented at the libc level that both can be raised | 09:57 |
| tristan | Chris, indeed, I just got the same error you did, but with [Errno 39] Directory not empty:... | 09:58 |
| tristan | So /me changes that to 'if e.errno not in [os.errno.ENOTEMPTY, os.errno.EEXIST]' and calls it fixed | 09:58 |
| Chris | Neat. | 09:59 |
| tristan | Ok, pushed that fix | 10:01 |
| tristan | And along with it, we now haz man pages | 10:02 |
| tristan | But these wont appear with `pip install --user -e .` | 10:02 |
| tristan | They *will* appear with `pip install --user .` | 10:02 |
| tristan | Which installs directly into your ~/.local | 10:02 |
| tristan | Only difference is that changes to your buildstream checkout source code will not take effect on the bst command | 10:03 |
| tristan | without `pip uninstall && pip install --user .` | 10:03 |
| tristan | err, `pip uninstall buildstream && pip install --user .` | 10:03 |
| * tristan now checks/merges jonathanmaw's code and then sets out to include an "Installing and Running" section of the docs with ironfoot's docker guide included | 10:04 | |
| tristan | Oh, and I shouldnt forget also, more graceful handling of uncaught exceptions in plugins | 10:04 |
| tristan | I'm thinking, a BUG: message in the frontend | 10:04 |
| tristan | instead of a double stack trace | 10:05 |
| ironfoot | think about people sending infromation about something not working | 10:06 |
| ironfoot | might be good to show the stack trace | 10:06 |
| tristan | ironfoot, of course, but only the one that is important :) | 10:13 |
| tristan | ironfoot, currently the frontend chokes and fires another exception when a child process has a bug | 10:13 |
| ironfoot | aha, i see | 10:14 |
| tristan | jonathanmaw, merged your tar source :) | 10:35 |
| tristan | jonathanmaw, I added to that a couple commits | 10:35 |
| tristan | One to just add a link in the main documentation face page (we list all the plugins so they are easy to lookup) | 10:35 |
| tristan | And notably this one which makes a couple minor changes: https://gitlab.com/BuildStream/buildstream/commit/de09acfd87f6c97b6e877fc0d4b10340844ee6c0 | 10:36 |
| * tristan has been pondering making some change in how plugins report errors, perhaps changing it so that they just call self.error(), and have Plugin.error() raise an ElementError/SourceError | 10:37 | |
| tristan | that would only mean that errors would automatically always have an str(self) formatted into the string | 10:38 |
| tristan | but meh | 10:38 |
| * tristan files https://gitlab.com/BuildStream/buildstream/issues/8 regarding man page auto-generation | 10:56 | |
| tristan | If anyone feels like getting a pip/setuptools/distutils headache, feel free to fix that :D | 10:57 |
| *** brejoc has quit IRC | 11:00 | |
| *** brejoc has joined #buildstream | 11:36 | |
| jonathanmaw | tristan: AIUI, bzr is written with python, using bzrlib. Does that give me free rein to use bzrlib for the bzr source plugin? | 11:49 |
| ssam2 | my only concern would be whether it's installed by default in the Python search path, but on my system seems like it is | 11:55 |
| ssam2 | does it work with python3 ? | 11:56 |
| ssam2 | seems to be installed for python2 only on my system in fact | 11:56 |
| jonathanmaw | hrm, yep, bzrlib doesn't get installed for python3 | 11:59 |
| jonathanmaw | so it looks like I'm using the binary :-/ | 11:59 |
| tristan | I am skeptical of using bzrlib | 12:18 |
| tristan | Right now as it stands, using ostree via python bindings is a bit of a pain and hurdle for installing, for one | 12:19 |
| tristan | For another, it's causing some unexplained bugs with suspend/resume, possibly because we're running it directly in process and maybe it has it's own signal handling | 12:20 |
| tristan | Installing bzr is one thing, installing bzrlib may be another | 12:20 |
| tristan | Anyway yes, it's python 2, lets just invoke it :) | 12:21 |
| jonathanmaw | tristan: I haven't used bzr before, what does a typical use-case look like? (i.e. do people refer to a specific commit in a bzr branch?) | 12:33 |
| tristan | jonathanmaw, I think its relatively similar to svn ? | 12:36 |
| tristan | I havent used it in a long time, and only in passing to build something someone else maintains | 12:36 |
| tristan | The key of course is "Find out how to address an entirely specific set of sources" (this is 'ref' and need not necessarily be a single string/checksum, 'ref' can be a dict if the VCS demands it) | 12:37 |
| ssam2 | i believe in bzr you get a revision number | 12:38 |
| tristan | And "Find out how to determine what is the specific set of sources for a symbolic name" (this is 'track', most if not all VCSs have the concept of branch or tag) | 12:38 |
| brejoc | Hi I'm the author of this comment: https://blogs.gnome.org/tvb/2017/02/06/introducing-buildstream-2/#comment-3940 | 12:42 |
| brejoc | I'm really interested in exploring the possibilities of BuildStream for web applications. But I'm lacking overview… a lot. I understand how I'm supposted to "build" my application, but I have no clue where to start from. Usually I'm just picking the current Debian release as a Docker image to start building. Where would I get a Debian base system here? Additionaly is is it already possible to package everything in a neat Debia | 12:42 |
| brejoc | n package at the end? Sorry for the huge amount of text… | 12:42 |
| tristan | Welcome brejoc :) | 12:45 |
| tristan | No worry, thanks for your interesting comments ! | 12:45 |
| tristan | Ok so I'll try to reply some of this hear | 12:46 |
| tristan | We should really start using the mailing list though for more serious/archived discussion :D | 12:46 |
| tristan | brejoc, So there are a few cruxes to the issue | 12:46 |
| tristan | First, if you want to build something with BuildStream, well... you have to build it :) | 12:47 |
| tristan | So, the last demo in the latest blog post, builds a "damn everything" | 12:47 |
| tristan | brejoc, but that wont be what you want | 12:47 |
| tristan | Because building everything, means maintaining everything, so that's a very specific case | 12:48 |
| tristan | brejoc, what I have been having in mind, is to implement Source plugins which can interact with distro packaging mechanisms | 12:48 |
| tristan | brejoc, please stop me if/when you have any questions :) | 12:49 |
| tristan | So basically, in BuildStream you have a pipeline if Elements, and each Element can have one or more Sources | 12:49 |
| brejoc | No, no, please go on! :) | 12:49 |
| tristan | Sources are there to obtain anything external, usually to obtain source code | 12:49 |
| tristan | But, for instance we have the ostree source plugin, which can be used to pull in pre-built flatpak runtimes | 12:50 |
| tristan | In the last example /me finds link | 12:50 |
| brejoc | I've seen it, I know what you mean. | 12:50 |
| tristan | for instance we have: https://gitlab.com/BuildStream/buildstream-tests/blob/build-gnome/elements/gnu-toolchain/base-platform.bst | 12:51 |
| tristan | And https://gitlab.com/BuildStream/buildstream-tests/blob/build-gnome/elements/gnu-toolchain/base-sdk.bst | 12:51 |
| tristan | Those are "import" elements which, we use to stage a deterministic runtime from which we bootstrap our own | 12:51 |
| tristan | just as an example | 12:51 |
| tristan | So "import" just takes what is in a Source and stages it verbatim | 12:52 |
| tristan | Now, a Source plugin has some constraints | 12:52 |
| brejoc | So I would have to find an ostree of a Debian release. Are there any? | 12:52 |
| tristan | Because we want A.) To always refer to something exact, we want to always build something deterministic | 12:52 |
| tristan | And B.) We want to be able to "track" something symbolic | 12:52 |
| tristan | So that we can always "try the latest stable branch of foo" | 12:53 |
| tristan | brejoc, I think not, I am thinking bigger than that actually :D | 12:53 |
| tristan | or, I think it's a bit more ambitious but possible | 12:53 |
| brejoc | Okay :) | 12:53 |
| tristan | Rather than enforcing that an ostree repository be created for certain versions of Debian/Fedora/FooDistro | 12:54 |
| tristan | I would really like to see, say, 'dnf' and 'dpkg' Source implementations emerge | 12:54 |
| tristan | Which actually cater to those constraints | 12:54 |
| tristan | So this way, we could have a Source which is able to track "jessie" | 12:54 |
| tristan | And able to checkout a "very specific version of jessie" | 12:55 |
| brejoc | Oh, I see. That is indeed very interesting! | 12:55 |
| tristan | This would open the door, both for the kind of use case you describe, but for other things as well | 12:55 |
| tristan | Say, imagine we had the GNOME release modulesets, and we could run CI on variants of distributions | 12:55 |
| tristan | "Lets try building the whole GNOME Desktop on the latest development release of Debian sid, or Fedora, or whatever" | 12:56 |
| tristan | I did some reading last week I think on the subject, and it doesnt seem altogether obvious | 12:56 |
| tristan | But I think it can be done | 12:56 |
| tristan | Because it's a typical case for sysadmins | 12:57 |
| tristan | I.e. they have produces a prestine install of something that works | 12:57 |
| tristan | And they want to dump some metadata into a file, and reuse that metadata to reinstall another machine with "Exactly this RHEL version, _exactly_" | 12:57 |
| tristan | or suchlike | 12:57 |
| tristan | So that would be a great direction forward, for the one half of the pipeline | 12:58 |
| brejoc | You are absolutely right. Usually not every server is update to date. | 12:58 |
| tristan | For the other part, I have in other projects been able to fairly easily generate binary rpms from binary artifacts I've built _without_ rpm | 12:59 |
| tristan | and think that is also entirely possible with dpkg as well | 12:59 |
| tristan | This of course will work much better when; you have built and are deploying from a sysroot/SDK that is actually the target version of Debian you want to distribute to | 12:59 |
| tristan | So, you could have support for staging various versions of Debian -> Building on that -> Deploying for _that_ version of debian | 13:00 |
| tristan | And it would theoretically be no different from doing the same thing manually, on a real debian host of that version | 13:00 |
| tristan | Except of course, you could do the whole thing on gentoo or whatever :) | 13:01 |
| brejoc | You are describing an utopian future of building software. I live it! :D | 13:01 |
| brejoc | live==love | 13:02 |
| tristan | For the deploying Docker container cases, I think thats even more easily achieved, you would just have a pipeline which takes the "everything you staged + what you've built = > deploy it all into a container image" | 13:02 |
| tristan | brejoc, I'm more and more excited about this project as it materializes too yes :) | 13:03 |
| brejoc | Wow, I'm actually pretty gobsmacked at the moment! | 13:04 |
| tristan | Enough to look into writing up a "dpkg" Source implementation ? | 13:05 |
| tristan | hehe | 13:05 |
| brejoc | I can try. But you'd have to point me into the right direction. | 13:06 |
| brejoc | Let's start with: which is the best disto to install BuildStream. Ubuntu seems to be no perfect fit. | 13:07 |
| tristan | Well, jonathanmaw has just whipped up 'tar', the ever typical tarball source, and now looking into 'bzr' (which wraps up all the sources supported by flatpak, so soon we try to convert flatpak json to BuildStream projects) | 13:07 |
| tristan | brejoc, I use debian stretch, but since recently it works with debian jessie | 13:08 |
| tristan | On jessie you need bubblewrap and ostree + the ostree gir support from jessie backports | 13:08 |
| tristan | It needs python 3.4 (it may work or older pythons but that's untested) | 13:09 |
| brejoc | Are there packages for bubblewrap and ostree? | 13:09 |
| tristan | Yes, both in stretch and jessie-backports | 13:09 |
| brejoc | Awesome. So my first two steps would be the installation of BuildStream and looking into the tar source from jonathanmaw? | 13:10 |
| tristan | So I'm in the process of writing up some user docs for installing | 13:11 |
| brejoc | The tar source can already be found in master? | 13:11 |
| tristan | but the instructions in https://gitlab.com/BuildStream/buildstream/blob/master/HACKING.rst should work | 13:11 |
| tristan | Yes | 13:11 |
| tristan | Umm, yes it's in master, probably you dont want it as a template though | 13:11 |
| tristan | brejoc, the Source implementation is pretty straight forward though | 13:12 |
| brejoc | Okay. I'll be back soon! :p | 13:13 |
| tristan | So a Source derives from Plugin (as does Element), which provides some helpers you'll need to use | 13:13 |
| tristan | Alright :) | 13:13 |
| * tristan was going to point at docs | 13:13 | |
| tristan | brejoc, there are a few sources already though, the only thing about tar is that it doesnt really 'track' (cause it's just a tarball) | 13:13 |
| tristan | but all of them show pretty much what is needed to implement | 13:14 |
| tristan | git is well done, but harder to read / follow because it supports submodules | 13:15 |
| * tristan is falling behind schedule for GUADEC but notices he still has another full week before CFP deadline... good | 13:17 | |
| brejoc | Okay, I'll take a look and see if I understand it. | 13:17 |
| tristan | brejoc, I'm on east asia time so I might disappear, but I do pop in late night every now and then so I'm not "gone" | 13:18 |
| tristan | next week I'll be europe time for a week | 13:18 |
| brejoc | Nice. Then we'll be in the same time zone. | 13:18 |
| * tristan should probably writeup better plugin author facing docs | 13:19 | |
| tristan | But https://buildstream.gitlab.io/buildstream/buildstream.plugin.html#module-buildstream.plugin | 13:19 |
| brejoc | I'm in France right now, but should be back in Germany at the end of the month. | 13:19 |
| tristan | And https://buildstream.gitlab.io/buildstream/buildstream.source.html#module-buildstream.source | 13:19 |
| tristan | Should be helpful | 13:19 |
| brejoc | Thanks! Haven't seen those yet. | 13:20 |
| * tristan very much enjoyed his short stay in Berlin, especially the hacker spaces were very cool | 13:20 | |
| tristan | but I've heard that "Berlin is not really Germany" :) | 13:20 |
| tristan | yeah I think the docs are pretty nice | 13:20 |
| tristan | check em out :) | 13:20 |
| tristan | Starting from https://buildstream.gitlab.io/buildstream/ | 13:20 |
| brejoc | Oh well, which part of Germany is Germany then? Saarland, where I'm coming from, isn't really Germany too. :) | 13:21 |
| brejoc | Perhaps Bayern. But nobody likes them! :p | 13:21 |
| tristan | Heh, it's just what I've been told, that Berlin is something completely different :) | 13:21 |
| brejoc | It is. But Berlin is nice. I've been there three times. The nerd scene is awesome there! | 13:22 |
| *** franred has quit IRC | 13:24 | |
| *** brejoc has quit IRC | 14:25 | |
| *** tristan has quit IRC | 14:39 | |
| *** tristan has joined #buildstream | 14:43 | |
| *** ChanServ sets mode: +o tristan | 14:43 | |
| *** tristan has joined #buildstream | 15:04 | |
| *** ChanServ sets mode: +o tristan | 15:31 | |
| *** tiagogomes_ has quit IRC | 15:54 | |
| *** brejoc has joined #buildstream | 15:58 | |
| *** brejoc has quit IRC | 16:03 | |
| *** jonathanmaw has quit IRC | 16:24 | |
| *** brejoc has joined #buildstream | 18:25 | |
| *** ssam2 has quit IRC | 18:41 | |
| *** brejoc has quit IRC | 20:16 | |
| *** xjuan has joined #buildstream | 21:39 | |
| *** xjuan has quit IRC | 22:26 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!