| *** sambishop has quit IRC | 05:07 | |
| *** mike has joined #baserock | 05:18 | |
| *** zoli__ has joined #baserock | 05:19 | |
| *** mike is now known as Guest27260 | 05:19 | |
| *** zoli__ has quit IRC | 05:43 | |
| *** Guest27260 has quit IRC | 05:45 | |
| *** Guest27260 has joined #baserock | 06:00 | |
| *** zoli__ has joined #baserock | 06:14 | |
| *** sambishop has joined #baserock | 06:45 | |
| *** fay_ has joined #baserock | 07:04 | |
| *** a1exhughe5 has joined #baserock | 07:20 | |
| *** mariaderidder has joined #baserock | 07:40 | |
| *** sambishop has quit IRC | 07:58 | |
| *** Gest4651 has joined #baserock | 07:58 | |
| *** Gest4651 has quit IRC | 08:01 | |
| *** bashrc_ has joined #baserock | 08:03 | |
| SotK | morning! | 08:13 |
|---|---|---|
| richard_maw | morning | 08:13 |
| * SotK likes the new changes to Mason | 08:13 | |
| petefoth | richard_maw: is either wearing his cloak of invisibility, or WFH | 08:14 |
| pedroalvarez | SotK: :D | 08:16 |
| pedroalvarez | I missed a few things, but I'm going to put the new version soon | 08:16 |
| *** mdunford has joined #baserock | 08:17 | |
| *** ssam2 has joined #baserock | 08:30 | |
| *** ChanServ sets mode: +v ssam2 | 08:30 | |
| *** gary_perkins has joined #baserock | 08:35 | |
| *** CTtpollard has joined #baserock | 08:51 | |
| *** jonathanmaw has joined #baserock | 08:58 | |
| *** edcragg has joined #baserock | 09:07 | |
| *** mdunford has quit IRC | 09:31 | |
| * pedroalvarez notices that now distbuild shows the real number of artifacts | 09:43 | |
| pedroalvarez | but now it doesn't count how many has it has already built | 09:43 |
| pedroalvarez | e.g. "Need to build 260/260 artifacts" | 09:43 |
| pedroalvarez | but it didn't start from the beggining | 09:44 |
| pedroalvarez | minour problem compared to the previous one, though :) | 09:45 |
| *** mdunford has joined #baserock | 09:48 | |
| persia | Rearranging the string might solve that. | 09:54 |
| persia | e.g. "Need to build artifact 260 of 260". | 09:54 |
| * SotK isn't sure how that is any better | 09:55 | |
| ssam2 | pedroalvarez: that will be a bug in the patches I sent.. | 09:56 |
| pedroalvarez | persia: the thing is that it doesnt need to build 260 | 09:58 |
| ssam2 | not immediately obvious where the bug is though. the message is sent here: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/tree/distbuild/build_controller.py#n568 | 09:58 |
| ssam2 | and it pretty clearly sets artifact.state to BUILT or UNBUILT based on the cache response, /then/ sends the CacheStatus message based on those values.. | 09:58 |
| ssam2 | oh, actually the bug is obvious | 09:59 |
| tiagogomes_ | is there any particular software that someone would like to see in baserock? | 10:03 |
| rjek | Exim, SpamAssassin, ClamAV, Pyzor. | 10:03 |
| tiagogomes_ | is there any particular software that someone would like to see in baserock that is not related with email? | 10:04 |
| rjek | :( | 10:05 |
| pedroalvarez | :) | 10:05 |
| richard_maw | sshfs | 10:05 |
| rjek | ScummVM is all I have that is not email-related, sorry. | 10:05 |
| Kinnison | bind9, dnssec-tools, | 10:06 |
| persia | tiagogomes_: Pascal, Go, and Java compilers, bootstrapped without blobs. No, the gcc implementations for these languages aren't enough. | 10:06 |
| persia | bind would indeed be lovely. | 10:06 |
| Kinnison | And haskell | 10:06 |
| Kinnison | please | 10:06 |
| ssam2 | tiagogomes_: for infrastructure, Cherokee, HAProxy, and exim would all be useful | 10:06 |
| pedroalvarez | openstack! | 10:06 |
| persia | For Haskell, my preference is ghc. | 10:06 |
| rjek | Cherokee++ | 10:06 |
| * Kinnison hides | 10:07 | |
| tiagogomes_ | Go looks interesting, but doesn't need/will need go to compile? | 10:07 |
| persia | Kinnison: Why hide? Do you have another Haskell compiler you'd prefer? | 10:07 |
| Kinnison | persia: I'm hiding from Cherokee | 10:07 |
| Kinnison | persia: for haskell, ghc is definitely my preference | 10:07 |
| *** zoli__ has quit IRC | 10:07 | |
| * rjek wishes people would stop writing self-hosted compilers. | 10:08 | |
| * Kinnison doesn't mind self-hosted compilers, so long as they can cross-prep with a C backend | 10:08 | |
| Kinnison | Sadly that set is pretty much limited to valac | 10:08 |
| persia | tiagogomes_: That is the fundamental issue with all the languages Kinnison and I named. There may be ancient versions of codebases that can compile in other languages and be used to compile early versions of interpreters, which can then be used to compile more recent ones, etc. | 10:08 |
| tiagogomes_ | righto | 10:09 |
| persia | For fpc, I know that not to be true, as the early development compiler was a proprietary one. For Java, we haven't been able to replicate a pure from-gcc build without a blob. | 10:10 |
| persia | I don't know if anyone has recently investigated ghc or golang | 10:10 |
| Kinnison | I investigated ghc about a year ago | 10:10 |
| Kinnison | sadly its from-C stuff had atrophied and its cross-compilation support was a tad grindy | 10:10 |
| rjek | Free Pascal is very good, but it has *always* been written in Pascal. | 10:11 |
| rjek | (Originally built using Borland Turbo Pascal) | 10:11 |
| persia | rjek: Yes, but it used to be possible to compile fpc with non-fpc. I do not believe this to still be true. | 10:11 |
| rjek | And I don't think gcc's Pascal compiler can build stuff written in Borland's dialect to let you bootstrap. | 10:11 |
| Kinnison | Anyway, Cherokee is a nice httpd and its inclusion in baserock would be good | 10:12 |
| Kinnison | Just don't expect upstream to respond nicely | 10:12 |
| * Kinnison is grumpy today | 10:12 | |
| Kinnison | Then again, Stefan likely will respond -- he's much nicer than I am | 10:13 |
| persia | If we restricted lorries only to upstream projects where upstream had expressed a willingness to resolve problems we report, we'd have a much smaller set of software available. | 10:13 |
| ssam2 | i've redeployed the frontend (haproxy) system at baserock.org to be Fedora 21 instead of Fedora 20. Let me know if any infrastructure is suddenly broken | 10:15 |
| ssam2 | about compilers, it seems pretty clear we're going to have to make the bootstrapping requirements for Baserock systems more complex. e.g. 'the build-system can be built anywhere you have a C compiler plus these tools'; 'the java-system can be built anywhere you have everything needed for the build-system plus a Java compiler', etc | 10:17 |
| ssam2 | seems ugly, but the alternative of making everything bootstrappable from C seems impossibles | 10:17 |
| pedroalvarez | then we will end up releasing a java system, and a pascal system, and so on | 10:21 |
| persia | This makes integration hard. | 10:21 |
| persia | How large would we need to make a development system to include C, Pascal, Java, Haskell, and Go? | 10:22 |
| rjek | Contribute C backends to all the languages :) | 10:22 |
| *** lachlanmackenzie has joined #baserock | 10:22 | |
| persia | Some upstreams are actively unsupportive of that (fpc, golang), others don't want to maintain it (ghc, Java) | 10:23 |
| * rjek is left with rewriting all software in C. | 10:23 | |
| rjek | Which isn't ideal either | 10:23 |
| persia | Right, hence my query about size. | 10:23 |
| ssam2 | persia: we could just provide instructions of how to use the upstream binaries that projects provide in order to bootstrap them | 10:24 |
| persia | I don't like blobs, but if we do the bootstrap from blobs in a repeatable and declarative manner, then publishing a system supporting those bootstraps is a useful service. | 10:24 |
| ssam2 | if the upstream project is unbootstrappable but doesn't provide binaries either, I think they are too unhelpful for us to care about | 10:24 |
| persia | ssam2: The problem is new / different architectures. | 10:25 |
| ssam2 | I think that's their problem, again | 10:25 |
| ssam2 | as long as our inclusion of such flawed compilers doesn't affect the portability of the rest of the software we built in Baserock, I don't think we should worry about it | 10:26 |
| persia | It's an adoption problem. If the upstreams are supported by distros, and developers expect it to work and write useful software in those languages, then people who use those distros may not be able to use Baserock. | 10:26 |
| ssam2 | if it's supported by a distro for a given architecture, we have a blob we can use to bootstrap | 10:27 |
| ssam2 | this discussion feels like the exact inverse of one that we had about this time last year :-) | 10:27 |
| persia | Distro architecture support changes over time. | 10:27 |
| persia | A recent example being BE POWER, where it used to be that everyone supported it, and now very few support it. | 10:28 |
| persia | As a result, if we aren't maintaining the bootstrapped blob, it becomes unavailable to users over time. | 10:28 |
| ssam2 | I guess... but again, if people want to use Java on BE POWER and we can't built it for BE POWER because of the way Java is written, I think that's a problem for the Java folk to solve | 10:28 |
| persia | A less recent example is that Java used to be bootstrappable, but the code that was used to bootstrap has bitrotted, so it is no longer bootstrappable from C, even with the old code. | 10:28 |
| ssam2 | not that we can't help them solve it, but we shouldn't try to move the problem into Baserock | 10:29 |
| ssam2 | s/built/build/ | 10:29 |
| persia | My points are 1) that we *can* often build any given self-hosting compiler for any of our architectures today (but that it is hard to promise users they can continue to do so in the future, complicating long-term maintenance), and 2) that if the process for getting working self-hosting compilers requires users to maintain their own development system forks, we can't expect anyone to use our reference development system (currently an assumption for a | 10:30 |
| persia | number of our transitions). | 10:30 |
| persia | I don't understand why these aren't Baserock problems, or why the relevant upstream projects should care about our specialised needs. | 10:31 |
| tiagogomes_ | re Java. Can't you build openjdk with GCJ? | 10:34 |
| persia | tiagogomes_: no. You could build ecj with gcj, and then use that to get to OpenJDK, but that doesn't work anymore. | 10:42 |
| ssam2 | persia: the bit that confuses me is your saying "our specialised needs." I don't have any specialised needs, myself | 10:45 |
| ssam2 | i'm quite happy with x86 | 10:45 |
| ssam2 | well, ARM is nice too. but you get my point -- i've little interest in spending time making sure things work on obscure architectures, although I like that it's possible | 10:46 |
| ssam2 | and it's clearly decisions by the upstream project (deciding not to have their implementation be bootstrappable from C, and/or deciding not to care about cross-compiling to those architectures) that cause problems for people who want to use those projects on unusual architectures | 10:48 |
| *** zoli__ has joined #baserock | 10:49 | |
| persia | ssam2: "Our specialised needs" being an apparent unwillingness to host blessed blobs that are not trivally bootstrappable. | 10:49 |
| persia | There are good and valid reasons for this, but I'm not convinced that a random upstream project is likely to care what our project wants, when compared to other consumers of their code. | 10:50 |
| ssam2 | I'm proposing that we host blessed blobs for cases like these | 10:51 |
| ssam2 | at arm's length. | 10:51 |
| ssam2 | but, it will be nice to not be going against the will of the projects which think they are a good idea, for however long it takes them to realise that they aren't a good idea | 10:52 |
| richard_maw | 'they' being the blobs, rather than the project | 10:52 |
| ssam2 | either ;) | 10:52 |
| persia | Just as a matter of record, all of baserock depends on such a blob model. We can't start without a C compiler. | 10:57 |
| persia | That we chose C is arguably a choice that directly discriminates against the other projects. | 10:57 |
| persia | And for C, we do host the blessed blobs, and distribute them in our development system. | 10:58 |
| persia | I think we've a bit of work to justify why we don't want to do that for other environments. | 10:58 |
| persia | And as such, I wonder if it isn't easier to just include all of them in our development systems. | 10:59 |
| ssam2 | the 'blessed blobs' being the 'build' system images at http://download.baserock.org/ ? | 11:01 |
| persia | That contains a blessed binary C compiler, yes. | 11:02 |
| persia | Well, to be fair, an entire C-based toolchain. | 11:02 |
| persia | Most of the rest of it was built from source, so avoids the need to be blessed. | 11:03 |
| richard_maw | though morph isn't picky about who blessed the C toolchain, so long as someone has | 11:03 |
| ssam2 | is there a usable free software kernel and userland written in a language that we don't treat in the same way we treat C? | 11:03 |
| ssam2 | I think treating C specially is justifiable because Linux and GNU are written in it | 11:03 |
| persia | Right. I don't assert morph should be picky about who blessed fpc, ghc, etc. Just that we should expect someone to have blessed it, and provide blessed versions. | 11:04 |
| persia | Not all of GNU is in C, but I can see that argument. There are kernels and runtimes that are written in other languages, but last I checked morph expects to be building linux systems in ways that are hard to detangle. | 11:05 |
| persia | Where I'm more uncertain is why e.g. ghc upstream cares that we're only treating C specially. | 11:06 |
| persia | Or, for that matter, why they should treat C as special when they are Haskell folk, and may not be comfortable (or even knowledgeable) in C. | 11:07 |
| ssam2 | persia: I think that if we want GHC we should treat it specially too | 11:08 |
| ssam2 | persia: but I think we should treat it separately, so that people who don't use GHC don't get limited by its portability constraints | 11:09 |
| persia | ssam2: OK. Can you think of any reason not to treat Java, Python, Go, and Haskell as special and include them in the reference build systems? | 11:09 |
| ssam2 | persia: I think we should do that. But I don't think we should put them in 'build-system-x86_64.morph' | 11:10 |
| persia | Hrm. How do we avoid end-users needing to fork their own build systems then? | 11:10 |
| ssam2 | 'java-system-x86_64', 'go-system-x86_64', 'haskell-system-x86_64', ... | 11:10 |
| persia | What if I want Java and Go? | 11:10 |
| ssam2 | the composability of strata theoretically makes this easy | 11:11 |
| ssam2 | the reference systems are examples, after all | 11:11 |
| ssam2 | to be honest, putting them all in 'devel-system-x86_64' would be fine, as long as we continue not to release that | 11:11 |
| persia | So I need to maintain my own reference system? | 11:11 |
| persia | How can I build it, if I don't have a build system with the blessed blobs? | 11:11 |
| ssam2 | someone needs to maintain it, I'm not going to do it for you | 11:11 |
| ssam2 | unless I'm hired to of course :) | 11:12 |
| persia | That part goes without saying :) | 11:12 |
| ssam2 | as someone who makes releases, i would stop doing them if I suddenly had to care about GHC, Java and Go just to make releases | 11:12 |
| ssam2 | unless someone had made it really easy to make releases with all those systems in | 11:13 |
| SotK | ssam2: +1 | 11:13 |
| ssam2 | s/systems/compilers/ | 11:13 |
| persia | If there were blessed blobs in the default reference build system, such that you had the blobs to build each release, would you consider that being easy? | 11:14 |
| SotK | it took me more than a day to do 15.25, I wouldn't want to suddenly have vastly more to do on top of what I did | 11:14 |
| ratmice___ | well, there is concern that given some binary blob for one arch/libc variation, or it introduces some disjointedness | 11:14 |
| ssam2 | persia: depends how often each of them broke | 11:14 |
| ssam2 | but I think I see your point better suddenly | 11:15 |
| persia | ratmice___: How do you mean? Also, for non-C and non-C hosted, why libc? | 11:15 |
| persia | ssam2: Oh yes, when the compilers break, that's annoying. I presume that in most cases one would revert the commits that broke the compiler and just build the old compiler known to be self-hosting on the blessed blob. | 11:16 |
| persia | And then it becomes an upstream bug: "foo-X can't build foo-X+1, breaking bootstrapping. Please tell another bootstrapping story." | 11:16 |
| persia | And, for architecture support variations, I think the right answer is to not include runtimes in the build-arch.morph files if they are unsupported by that arch. | 11:17 |
| persia | As they become supported, they can be added there. | 11:17 |
| ssam2 | ok, I think I'm OK with this other than the fact that 'build' may become huge | 11:18 |
| persia | That's the core of my original question: how big would we expect 'build' to get if we expand from "C" to "C, Java, Pascall, Haskell, Go"? | 11:19 |
| persia | If one of those is particularly large, and particularly unpopular, then maybe we have some rationale to use to talk to upstream about why they are getting less support, and how they can help resolve that. | 11:20 |
| ssam2 | I think i'd still like to have a 'build-c' system in this case though. It's just a kind of gut feeling that we *should* continue to treat C specially. | 11:24 |
| richard_maw | I think the limit of treating it specially should be that build-essential is C, but we have separate java-bootstrap, pascal-bootstrap, haskell-bootstrap and go-bootstrap strata | 11:25 |
| richard_maw | and all the foo-bootstrap strata can depend on build-esential for sufficient shell utilities to be able to invoke the build | 11:26 |
| paulsher1ood | +1 | 11:28 |
| ssam2 | fair enough | 11:28 |
| * paulsher1ood thought jmacs did build java? | 11:28 | |
| persia | My only concern is that this applies to linux systems. | 11:28 |
| persia | paulsher1ood: Not without a blob. | 11:28 |
| paulsher1ood | ok | 11:28 |
| persia | For systems that don't depend on the C toolchain to build at all, C isn't build-essential. | 11:28 |
| paulsher1ood | and the go implementation we have is not good enough somehow? | 11:29 |
| ratmice___ | persia: because binaries for different c runtime from libc.. It probably sits more on the OS side of things than people would typically consider I guess, if morph can't build a blessed binary _somehow_, you are reliant on upstreams ability to port it, not sure how be it something like treb, or some mechanism for mounting images, I just concerned when I see it consuming rather than producing images | 11:29 |
| persia | But that's a implementation detail for folk building those systems in the future. | 11:29 |
| * ratmice___ likes how ellcc looks | 11:29 | |
| richard_maw | persia: true, but bootstrapping up a shell is intertwined with C, and you need a shell to invoke the builds of other languages' toolchains | 11:30 |
| ratmice___ | I think it comes with qemu and stuff so it can even bootstrap itself under emulation :) | 11:30 |
| persia | richard_maw: Depends on the OS: there are OSes written in Pascal and Java that have other ways to spawn jobs, but yes. | 11:30 |
| ssam2 | persia: I'm confused that you say 'not without a blob', when I thought we agreed that blobs will be necessary, and they will be in the build-system | 11:31 |
| persia | So, suitably informed, I'm +1 to richard_maw's model | 11:31 |
| persia | ssam2: We did. Part of why I developed the opinion that I brought to this discussion was the results from jmacs' work on Java. | 11:31 |
| ssam2 | ah, right | 11:31 |
| persia | ratmice___: Ah, so arbitrary compilers that need to interact with Linux end up using a C ABI to do so, such that all languages are essentially dependent on the C toolchain implementation? | 11:33 |
| ratmice___ | persia: yeah, like the MLWorks compiler, they have binaries for it, and the source code :), but any way at rebuilding it has been an effort, so I don't think there is any linux port, even though there do actually exist commercial binaries for linux somewhere :) | 11:36 |
| ratmice___ | someone had to go back i believe and rebootstrap it with the smlnj compiler that it was initially bootstrapped from | 11:37 |
| paulsher1ood | richard_maw: what about stage1-bootstrap, stage2-bootstrap, then c-bootstrap, then the other languages? | 11:39 |
| persia | ratmice___: Aha. I suppose we'll end up doing a few of those as we improve our blessing process. Thanks for the warning. | 11:40 |
| ssam2 | seems clear (also stage1-c-bootstrap and stage2-c-bootstrap might be clearer still) | 11:40 |
| persia | I like stageN-c-bootstrap | 11:40 |
| richard_maw | paulsher1ood: not possible, the result of a bootstrap stratum must contain chunks that aren't built in bootstrap mode | 11:41 |
| richard_maw | so you can't do stage1 | 11:41 |
| persia | But the point that we're using linux, means that it's really a "bootstrap", not just a C bootstrap, because of ratmice___'s point. | 11:41 |
| *** pacon has joined #baserock | 11:42 | |
| richard_maw | and given the incestuous relationship between libc and /bin/sh, you probably want to keep those two together too | 11:42 |
| paulsher1ood | richard_maw: so just bootstrap, c-bootstrap or bootstrap-essential for stage1+2? | 11:43 |
| persia | Stack Overflow seems to share the opinion that there are no POSIX shells implemented in non-C | 11:43 |
| richard_maw | paulsher1ood: let me check something first. | 11:44 |
| richard_maw | paulsher1ood: stage1, stage2 and stage3 all need to be in the same stratum unless we want to re-work how we define a stratum's products | 11:45 |
| paulsher1ood | ah | 11:54 |
| persia | I think we can safely separate the languages by strata. Just have strata for each runtime, each of which starts with bootstrapping, based on a minimal set of blessed binaries included in build systems. | 11:54 |
| persia | And then we can just leave the build-essential stack as it is today. | 11:54 |
| ssam2 | got that tiagogomes? :) | 12:01 |
| * richard_maw pushes another version of https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/morph+branch:master+topic:sourceresolver-objects | 12:32 | |
| richard_maw | this one should have proper caching, though I had to throw away the memoise implementation for some more wrapper objects | 12:33 |
| *** mdunford has quit IRC | 12:39 | |
| *** mdunford has joined #baserock | 12:51 | |
| *** pacon has quit IRC | 13:03 | |
| *** mdunford has quit IRC | 13:11 | |
| jonathanmaw | I'd like to be able to ssh into cache.baserock.org to look at why my build isn't using the cached artifacts radiofree uploaded | 13:15 |
| jonathanmaw | I suspect it's because I'm using the wrong version of morph, but I don't think I can work out which version of morph it was without looking at the cached artifacts. | 13:16 |
| SotK | ISTR radiofree uploaded artifacts to git.baserock.org rather than cache.baserock.org, but I may be wrong | 13:16 |
| jonathanmaw | ah, I have access to git.baserock.org. I'll see what I can find. | 13:17 |
| pedroalvarez | jonathanmaw: if they were uploaded to g.b.o, you have to remove cache.baserock.org from your morph.conf | 13:18 |
| pedroalvarez | but nothing seems to habe been uploaded on 2015 to g.b.o cache | 13:19 |
| jonathanmaw | pedroalvarez: I see files on g.b.o in the cache that were created June 17th 2015 | 13:21 |
| pedroalvarez | meh, I'd like to know how you found them | 13:21 |
| jonathanmaw | `ls -lt /home/cache/artifacts | head` on g.b.o | 13:22 |
| * pedroalvarez facepalms | 13:23 | |
| *** mdunford has joined #baserock | 13:24 | |
| pedroalvarez | I didn't know that `ls` doesn't show the year when it was created on the current year | 13:24 |
| tiagogomes_ | Is it possible to start an interactive shell in replacement of the current program using cliapp.runcmd? I can do it with subprocess. | 13:29 |
| Kinnison | given that under the hood runcmd is subprocess.Popen, I'd expect so | 13:29 |
| tiagogomes_ | experimentation suggests otherwise | 13:30 |
| *** mdunford has quit IRC | 13:31 | |
| *** mdunford has joined #baserock | 13:35 | |
| Kinnison | Find and ask the cliapp author? | 13:35 |
| persia | To avoid the need to SSH, can we expose the cache filesystem in some way for debugging? | 13:37 |
| ssam2 | persia: morph-cache-server is probably the best way to do that, if someone is interested in doing it. it already runs on git.baserock.org and cache.baserock.org (on port 8080) and provides an API for performing operations on the cache | 13:38 |
| persia | jonathanmaw: For your debugging, what extensions would you need to the morph-cache-server API? | 13:38 |
| ssam2 | tiagogomes_: if you want to replace the current program, surely you want execl() rather than subprocess.Popen() or cliapp.runcmd() ? | 13:38 |
| *** mdunford has quit IRC | 13:47 | |
| jonathanmaw | persia: To do it without SSHing in, I'd have to be able to ask the cache server which artifacts were created within a certain date range, which is really only necessary for troubleshooting, where you'll be in contact with someone who has access to the cache server, anyway | 13:47 |
| richard_maw | tiagogomes_: If you want the shell to be useable properly the first step is to make runcmd not replace the file descriptors. Set stdin=None, stdout=None and stderr=None | 13:50 |
| tiagogomes_ | ssam2 maybe. I am exploring the possibility of based in some flag, make morph drop into the chroot environment to debug an build failure | 13:50 |
| ssam2 | i see, cool. for me, just printing out a suitable command that I can copy n paste would be enough | 13:53 |
| ssam2 | bonus points if you can solve it for distbuild ;) | 13:53 |
| pedroalvarez | heh, that would be a lot of bounus points | 13:53 |
| richard_maw | spawn an sshd in the chroot on a random port and copy the public key from the caller into its authorized_keys | 13:55 |
| tiagogomes_ | ssam2 that was my original plan. But after I seeing the length of the command that morph uses to chroot I gave up of the idea. The command even has shell script embedded. | 13:55 |
| ssam2 | true.. to reproduce the chroot exactly is a long cmdline | 13:56 |
| ssam2 | i'd like to replace that part of the code with http://github.com/codethinklabs/sandboxlib | 13:56 |
| ssam2 | but that would make it harder still to paste a commandline | 13:57 |
| richard_maw | hmm, parhaps we ought to see if we can get the logic for whitelisting writable paths into linux-user-chroot, but sandboxlib includes a nice change that should make it possible to get rid of the embedded shell script AIUI | 13:57 |
| ssam2 | the only blocker to using sandboxlib in Morph is that there needs to be a way of either running a pipeline, or piping stdout and stderr into more than 1 place | 13:57 |
| ssam2 | i was having a look at the 2nd option, so we wouldn't have to create a pipeline with `tee` any more | 13:58 |
| richard_maw | ssam2: you're in asyncore/asynchat territory there, we might be able to re-use some of the code we used for write extensions there | 13:59 |
| ssam2 | way ahead of you ;) | 13:59 |
| ssam2 | just trying to write some tests for it | 13:59 |
| persia | jonathanmaw: Unfortunately, the ops team doesn't have 24 hour coverage, so many users end up not being able to ask that sort of thing from a human. Would being able to get a list of all the artifacts with creation dates be sufficient, for local filtering? | 13:59 |
| * richard_maw would prefer porting to python 3.4 and using asyncio | 13:59 | |
| ssam2 | i'd rather sandboxlib stayed usable by python 2.7 for the time being, otherwise that'd be great | 14:00 |
| SotK | ssam2: cliapp.runcmd() runs a pipeline if you give it more than one list (e.g. `cliapp.runcmd(cmd1, cmd2)`) | 14:00 |
| * SotK may be misunderstanding the problem | 14:01 | |
| richard_maw | SotK: sandboxlib does not use cliapp | 14:01 |
| richard_maw | ssam2: https://pypi.python.org/pypi/trollius may be of interest then | 14:01 |
| SotK | ah, the problem is in sandboxlib... I was indeed misunderstanding the problem then | 14:03 |
| jonathanmaw | persia: it'd be sufficient (alongside knowing how to use this API). I had some concerns about whether that'd be open to abuse, but sending the whole file list would come to about 4Mb, whereas individual files can come to up to 400Mb. | 14:06 |
| *** mdunford has joined #baserock | 14:18 | |
| *** petefoth has quit IRC | 14:23 | |
| *** mdunford has quit IRC | 14:23 | |
| *** mdunford has joined #baserock | 14:23 | |
| richard_maw | has anyone got a few moments to look over https://gerrit.baserock.org/#/c/444/ ? It'd be nice to get a couple of months old change merged. | 14:31 |
| richard_maw | It mostly looks fine, but SotK may have opinions, since it adds a write extension. | 14:34 |
| * SotK takes a look at it | 14:35 | |
| *** benbrown1 is now known as benbrown_ | 14:39 | |
| richard_maw | hrm, pushing a new version of a patch to gerrit isn't sufficient to change its owner | 15:05 |
| richard_maw | my google bubble does not allow me to search for documentation on how to change an aspect of a change in gerrit | 15:12 |
| Kinnison | https://groups.google.com/forum/#!topic/repo-discuss/aqNgmuiCtyk | 15:13 |
| richard_maw | Kinnison: I saw that one, that's about pushing a new change as someone else, the title is misleading. | 15:14 |
| Kinnison | maybe ssam2 can do it as a gerrit admin | 15:15 |
| ssam2 | why do you need to change the owner? | 15:17 |
| ssam2 | google bubbles can be fixed by using duckduckgo ;) | 15:17 |
| richard_maw | because straycat is no longer working on those changes, and I promised I'd adopt them | 15:18 |
| ssam2 | right. maybe abandon and resubmit as yourself? | 15:18 |
| ssam2 | hmm, in fact you might need an admin to abandon them. I can do that. I'd rather not poke around in Gerrit's internals though | 15:19 |
| *** zoli__ has quit IRC | 15:19 | |
| richard_maw | right, I'll google around a bit more to see if there is anything, but if not I'll submit them as new changes and let you or another admin know which ones need to be abandoned | 15:20 |
| richard_maw | hrm, there's no magic %foo=bar option on push either | 15:24 |
| * richard_maw begins resubmission | 15:25 | |
| richard_maw | aJcTYQpgZvuTbR5eHcm6WTIFvmW6lwkedD3sSVl2ZQ | 15:26 |
| richard_maw | bah, wrong window | 15:26 |
| Kinnison | perdon? | 15:27 |
| *** gary_perkins has quit IRC | 15:27 | |
| richard_maw | Kinnison: Yes, one bad submission per river. | 15:27 |
| Kinnison | sounds uncomfortable | 15:27 |
| * richard_maw generates a new gerrit password | 15:29 | |
| Kinnison | :) | 15:29 |
| richard_maw | hm, don't have time to rebase all the changes properly, so I'll jut re-submit those for now | 15:40 |
| richard_maw | I don't know why gerrit generates passwords that have a / in them, but I get authentication failures when trying to use them | 15:42 |
| Kinnison | :( | 15:43 |
| * richard_maw laughs maniacally as he cherry-picks a change to a write extension in morph, into definitions.git | 15:47 | |
| * rjek steps away | 15:47 | |
| *** a1exhughe5 has quit IRC | 15:48 | |
| *** jonathanmaw has quit IRC | 16:01 | |
| *** mdunford has quit IRC | 16:03 | |
| richard_maw | ssam2: I've adopted All in https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/morph+branch:master+topic:straycat/remove-obsolete-features https://gerrit.baserock.org/#/c/705/ https://gerrit.baserock.org/#/c/704/ https://gerrit.baserock.org/#/c/67/1 and https://gerrit.baserock.org/#/c/219/ | 16:11 |
| richard_maw | I understood that SotK offered to adopt https://gerrit.baserock.org/#/c/666/ (The patch-set of the beast!) | 16:12 |
| * SotK thought that perryl offered to do that, but might be misremembering | 16:12 | |
| richard_maw | I think https://gerrit.baserock.org/#/c/912/1 must have come in after I offered to adopt things, as I don't have sufficient understanding to import to adopt https://gerrit.baserock.org/#/c/912/1 | 16:13 |
| richard_maw | SotK: I could easily be mistaken | 16:13 |
| perryl | yes, i've adopted 666 | 16:13 |
| * pedroalvarez now gets "the patchset of the beast" when thinking about doing the same joke | 16:15 | |
| perryl | speaking of, having looked at patchset 666, the comments mention reading in the database path and database age from a config file. is there a defined way of doing this, or should i just create a distbuild.conf file, check it exists and if so parse in the files? | 16:17 |
| SotK | maybe put them in morph-controller.conf? | 16:18 |
| ssam2 | richard_maw: so should I abandon all the patch sets you linked to now? | 16:19 |
| richard_maw | ssam2: yes please | 16:19 |
| ssam2 | ok | 16:20 |
| perryl | SotK: is there a morph-controller.conf? i'm looking at the distbuild page and it only mentions setting up morph.conf | 16:24 |
| SotK | perryl: yeah, I think the distbuild configure extension creates it | 16:24 |
| pedroalvarez | should be in install-files/distbuild | 16:25 |
| perryl | thanks, i'll check | 16:25 |
| paulsher1ood | ssam2: does abandonment dstroy the work? i hope we can still use it if/when? | 16:26 |
| SotK | paulsher1ood: you can still see abandoned changes | 16:26 |
| paulsher1ood | SotK: i guess i mean, are we keeping such work in branches anywhere, eg on g.b.o? | 16:26 |
| SotK | paulsher1ood: I *assume* you can still get the work from gerrit using git-review or such tools, but I haven't tried it. | 16:27 |
| SotK | the changes definitely still exist, so I don't see why it wouldn't work | 16:27 |
| pedroalvarez | it should be possible, and you can recover also recover them | 16:28 |
| richard_maw | also, I explicitly made sure that I pushed my own version of the patches that ssam2 is abandoning | 16:28 |
| paulsher1ood | ah, ok, thanks :) | 16:28 |
| *** mariaderidder has quit IRC | 16:29 | |
| paulsher1ood | ssam2: so an example use-case of the many-definitions-versions problem - there are many versions/flavours of OpenWRT - how would we practically describe the many different versions, all as first-class citizens, run ci on some/all, compare them, take bits from one combine them with another etc | 16:34 |
| paulsher1ood | (while dealing with schema changes) | 16:34 |
| rdale | i'm very interested in this | 16:36 |
| paulsher1ood | rdale: me too.. .do you have any potential solutions? :) | 16:37 |
| rdale | well 'being interested' is i feel a good first step | 16:38 |
| ssam2 | paulsher1ood: ok, thanks for that | 16:39 |
| paulsher1ood | rdale: is there a definitions branch with a MIPS openWRT example system anywhere? | 16:47 |
| rdale | no, nowster has that on github | 16:48 |
| rdale | https://github.com/nowster/definitions.git openwrt-musl branch | 16:49 |
| pedroalvarez | we can call that a definitions branch too | 16:50 |
| paulsher1ood | :) | 16:50 |
| * paulsher1ood can't find the actual system definition in that | 16:51 | |
| rdale | look for edgerouterpro | 16:52 |
| pedroalvarez | systems/build-system-mips32l.morph ? | 16:52 |
| *** bashrc_ has quit IRC | 16:52 | |
| *** Guest27260 has quit IRC | 16:54 | |
| rdale | pedroalvarez: https://github.com/nowster/definitions/blob/openwrt-musl/systems/build-system-mips64b.morph | 16:58 |
| paulsher1ood | rdale: would be great to get that tidied up, work with latest deployments etc? all those configure scripts in the root dir look really messy now :) | 17:07 |
| rdale | i hadn't noticed the configure scripts. i've sent nowster's patch series for mips/openwrt/musl to our internal dev list so that we can discuss how best to get it merged into the baserock definitions master. unfortunately nowster is off on leave and i don't want to do anything without discussing it with him | 17:10 |
| *** ssam2 has quit IRC | 17:15 | |
| *** edcragg has quit IRC | 17:24 | |
| *** fay_ has quit IRC | 17:40 | |
| *** lachlanmackenzie has quit IRC | 17:40 | |
| persia | rdale: My recommendation would be to clean things and submit upstream. The existing branch can remain, so nothing is lost. If a wrong choice is made, it can be fixed in a later update. | 18:28 |
| persia | Waiting indefinitely to submit something because it might not be right is somewhat counterproductive, as without undergoing review by the review team, one ends up trying to guess what they might be thinking. | 18:29 |
| persia | In my experience, humans tend to have limited capacities for understanding what other humans are thinking in any detail: one may have functional models for the results, but the process is usually opaque. | 18:30 |
| *** zoli__ has joined #baserock | 18:46 | |
| *** edcragg has joined #baserock | 20:04 | |
| *** zoli__ has quit IRC | 20:39 | |
| *** zoli__ has joined #baserock | 20:49 | |
| *** franred has joined #baserock | 21:04 | |
| *** franred has quit IRC | 21:34 | |
| *** zoli__ has quit IRC | 22:13 | |
| *** edcragg has quit IRC | 23:45 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!