| *** gtristan has joined #baserock | 04:06 | |
| *** tiagogomes_ has joined #baserock | 06:42 | |
| *** tiagogomes has quit IRC | 06:42 | |
| *** paulw has joined #baserock | 07:24 | |
| *** ctbruce has joined #baserock | 07:26 | |
| *** ctbruce has joined #baserock | 07:27 | |
| *** paulw has quit IRC | 07:49 | |
| *** toscalix__ has joined #baserock | 07:55 | |
| *** toscalix__ is now known as toscalix | 07:59 | |
| *** rdale has joined #baserock | 08:23 | |
| *** locallycompact has joined #baserock | 08:34 | |
| *** jude_ has joined #baserock | 08:46 | |
| gtristan | Basement ! | 09:11 |
|---|---|---|
| paulsher1ood | ? | 09:11 |
| paulsher1ood | you are proposing to rename the project? | 09:12 |
| paulsher1ood | :) | 09:12 |
| gtristan | hahaha | 09:12 |
| gtristan | it just hit me while composing mail | 09:13 |
| gtristan | "The project where theoretical ideas are perused to no end without materializing into something concrete and useable" : Basement ! | 09:13 |
| * gtristan ducks incoming chunks of rock | 09:14 | |
| leeming | reminiscing about discussions with a certain someone perhaps? | 09:14 |
| paulsher1ood | gtristan: are there some specific ideas that you need to shoe in concrete? | 09:15 |
| gtristan | Sigh, I should finish the email first | 09:15 |
| * paulsher1ood hasn't had enough time for ml responses lately | 09:15 | |
| paulsher1ood | iiuc, from some public and some private conversations, it seems that you are wanting to move to a more oo implementation for tooling, either as a derivative of ybd or new base... whereas locallycompact is pushing a more functional programming approach? | 09:17 |
| * paulsher1ood can only comment that from his personal perspective, fp seems too hard for most programmers... | 09:18 | |
| gtristan | paulsher1ood, I want full extensibility of build pipelines, allowing people to implement their own custom elements - that is the extent of what needs to be OO | 09:18 |
| gtristan | paulsher1ood, I think fp has it's uses, and you probably find yourself using fp approaches at times, within OO code bases without realizing it | 09:19 |
| paulsher1ood | gtristan: that seems like a worthwhile objective. certainly there are usecases where extensibility would be useful | 09:20 |
| paulsher1ood | eg if someone wants to add testing/analysis at any stage | 09:20 |
| gtristan | paulsher1ood, check out the new pdf | 09:21 |
| gtristan | paulsher1ood, dont even bother reading it, jump to page 2 or 3 where pretty pictures will tell you what I'm after | 09:21 |
| gtristan | but by all means, read it if you have time :D | 09:21 |
| paulsher1ood | gtristan: is this PDF public? | 09:22 |
| gtristan | paulsher1ood, it's on the private list for now | 09:23 |
| gtristan | attached to email sent out yesterday | 09:23 |
| paulsher1ood | gtristan: any reason we can't invite community members to chime in? or are you trying to avoid the rocks? :) | 09:23 |
| * tiagogomes_ reminds that morph was extensible through the use of plugins. | 09:23 | |
| * paulsher1ood separately wonders whether a google doc, or wiki page, would be more collaborative than a pdf | 09:24 | |
| gtristan | paulsher1ood, I will reiterate on it once more and do so, but first I wanted to limit comments to those where were going to work on the project | 09:24 |
| paulsher1ood | ack | 09:24 |
| gtristan | s/where/who | 09:25 |
| tiagogomes_ | But regarding the build tool, I would much prefer that existing build tools were improved, instead of repeating the cycle. | 09:27 |
| paulsher1ood | tiagogomes_: i have a slightly different perspective. i'm happy to deprecate all current code, so long as there is a smooth path for users, which leads them to improved experience | 09:31 |
| tiagogomes_ | new tool, new bugs | 09:32 |
| paulsher1ood | true of course | 09:33 |
| paulsher1ood | tiagogomes_: https://gitlab.com/baserock/ybd/merge_requests/254 - did you take into account multiinstance when you made this change? | 09:34 |
| paulsher1ood | tiagogomes_: the current layout ensures that if two instances try to create the same artifact, precisely one succeeds | 09:37 |
| gtristan | tiagogomes_, fwiw I originally agreed with you, and the code will certainly be written by taking all the knowledge gained from ybd | 09:40 |
| pedroalvarez | not morph, morph is bad | 09:41 |
| paulsher1ood | lol | 09:41 |
| paulsher1ood | ybd took as much as it could from morph :) | 09:41 |
| pedroalvarez | oh | 09:41 |
| pedroalvarez | ybd is bad too! | 09:41 |
| paulsher1ood | all code is bad | 09:41 |
| pedroalvarez | I say it again, this is a very complicated problem | 09:42 |
| paulsher1ood | yup | 09:42 |
| pedroalvarez | and every time we think about it, we put more requirements on top | 09:42 |
| paulsher1ood | actually it's a set of very complicated problems | 09:42 |
| anahuelamo_ | tiagogomes_, I tried https://gitlab.com/baserock/ybd/commit/2a7963a3960c07c9e4caa1e7a4c0b69b800bd452 to build the wip for the /usr merge and it worked, is there any reason you didn't try to merge that in ybd? | 09:43 |
| * gtristan out for food | 09:45 | |
| *** anahuelamo_ is now known as anahuelamo | 09:45 | |
| paulsher1ood | pedroalvarez: would you rather we ignore the requirements? | 09:46 |
| leeming | the best approach is to have the requirements before starting :) If the previous code is written well enough, sure there is time spent re-implementing, but re-factoring should be so much easier | 09:47 |
| pedroalvarez | nope, I wasn't suggesting that. | 09:48 |
| * leeming is jumping in and out of channels, so may have missed something | 09:48 | |
| pedroalvarez | actually, maybe we need a set of solutions for these set of very complicated problems? | 09:49 |
| pedroalvarez | s/these/this/ | 09:49 |
| tiagogomes_ | paulsher1ood I did not. Looking at the code, it could fail as there are now two consecutive runs of shutil.move(). I'll have a look at it. | 09:49 |
| *** gtristan has quit IRC | 09:50 | |
| tiagogomes_ | What is multi-instance? Parallel instances of ybd or the use of threads inside ybd? | 09:50 |
| paulsher1ood | ybd forks itself, but also n users could run ybd on the same machinre | 09:54 |
| tiagogomes_ | why would they want that? | 09:55 |
| paulsher1ood | leeming: in the real world, we learn about requirements as we go. defining requirements up front is often (but not always) counter-productive | 09:56 |
| tiagogomes_ | anahuelamo, I didn't create a PR because honestly I don't know how artifact split is supposed to work. So I am not sure if my fix is the right approach. | 09:56 |
| paulsher1ood | tiagogomes_: the forking is for build performance - some builds are parallelizable | 09:56 |
| paulsher1ood | and it would not be unreasonable, on (say) a many-core machine, for there to be more than one user, sharing artifacts | 09:59 |
| paulsher1ood | it would be unreasonable to insist that only one instance of ybd can run at a time, imo | 09:59 |
| paulsher1ood | i think nobody is now clear on how splitting is supposed to work. the morph implementation was never really used in anger iiuc, and the original ybd implementation exposed some corner-cases/requirements | 10:01 |
| paulsher1ood | hence gtristan's mails about this recently | 10:02 |
| tiagogomes_ | In case of a multi-tenancy machine, I'd expect the share of artifacts to be mediated with a server. But I understand if you don't want to impose that requirement. | 10:05 |
| tiagogomes_ | I thought tristan email was more about the relation of artifact splitting and s-i commands. | 10:07 |
| paulsher1ood | tiagogomes_: you mean 'impose that limitation' i think. but there are also the case where an organisation buys a humungous 'build server' | 10:09 |
| paulsher1ood | tiagogomes_: that the relation between splitting and si is still open to discussion confirms my suggestion that 'nobody is now clear on how splitting is supposed to work', i think? | 10:10 |
| tiagogomes_ | paulsher1ood true :) | 10:11 |
| tiagogomes_ | The semantics of artifact splitting should have be written somewhere, so that I could determine if I had to fix ybd or definitions such that all the systems included build-essential-minimal. | 10:13 |
| tiagogomes_ | On the branch pointed by Ana, I chose to fix ybd as I thought it is how morph behaved. | 10:14 |
| pedroalvarez | the semantics are written in morph codebase | 10:15 |
| tiagogomes_ | pedroalvarez, are you volunteering to map hard to grasp code to documentation ;) | 10:16 |
| pedroalvarez | I implemented the SI commands | 10:17 |
| pedroalvarez | I am the creator | 10:18 |
| tiagogomes_ | basegod | 10:18 |
| *** gtristan has joined #baserock | 10:40 | |
| locallycompact | paulsher1ood, no that's absolutely not the major difference. The major difference is gtristan wants to talk about declaring pipelines and I'm not interesting in declaring pipelines. I declare systems and I talk about systems | 10:58 |
| * tiagogomes_ doesn't follow | 11:10 | |
| *** locallycompact has quit IRC | 11:38 | |
| *** toscalix has quit IRC | 11:40 | |
| *** toscalix has joined #baserock | 11:56 | |
| *** locallycompact has joined #baserock | 12:08 | |
| tiagogomes_ | paulsher1ood, can you provide some insight about the following line of code in ybd: `app.config.get("TMPDIR", "/tmp")` ? How does TMPDIR get set? | 12:31 |
| tiagogomes_ | That is in sandbox.py | 12:32 |
| mwilliam1_ct | tiagogomes_: doesnt that get the values from the config files? | 12:32 |
| *** mwilliam1_ct is now known as mwilliams_ct | 12:32 | |
| tiagogomes_ | Reading the code, it looks possible to do it through the config files. Although it is odd to have an uppercase config key. | 12:36 |
| tiagogomes_ | But I wonder if the correct instruction would be `os.environ.get("TMPDIR", "/tmp")` | 12:37 |
| *** gtristan has quit IRC | 13:02 | |
| * paulsher1ood is as confused as tiagogomes_ | 13:31 | |
| paulsher1ood | it seems to have been originally introduced in 66cf5ba0 | 13:36 |
| paulsher1ood | and then all subsequent fiddlings with the code must have assumed it was correct | 13:36 |
| paulsher1ood | the actual value that ybd cares about is config['tmp'] which is set in the loop at https://gitlab.com/baserock/ybd/blob/master/ybd/app.py#L193 | 13:40 |
| paulsher1ood | (for normal tmp file creation) | 13:41 |
| paulsher1ood | tiagogomes_: i think you're right that it should be `os.environ.get("TMPDIR", "/tmp")` | 13:42 |
| paulsher1ood | or just config['tmp'] since os.environ['TMPDIR'] has already been set that earlier in sandbox.py | 13:43 |
| tiagogomes_ | I think setting to config['tmp'] would be problematic. | 13:47 |
| paulsher1ood | well, as i say, os.environ['TMPDIR'] has already been set to that | 13:48 |
| tiagogomes_ | The sandboxes are created in config['tmp'], and inside the sandoxes only config['tmp']/sandboxfoo/chunk.inst, config['tmp']/sandboxfoo/chunk.build and config['tmp']/sandboxfoo/tmp should be writable for building chunks | 13:48 |
| tiagogomes_ | If you use config['tmp'] there, every directory in the sandbox dir will be writable | 13:49 |
| paulsher1ood | in other news 'Downloads:100204' | 13:49 |
| paulsher1ood | tiagogomes_: in that case, can we just drop the line, and have writable_paths = [dn['checkout'], dn['install']] ? | 13:50 |
| tiagogomes_ | No. When building a chunk, they need a writable /tmp directory. | 13:51 |
| paulsher1ood | ok. so this has been working, but only because most host systems have a writeable /tmp | 13:54 |
| paulsher1ood | if someone were to set TMPDIR: as a conf variable that woudl work also, for a non-standard system. | 13:55 |
| paulsher1ood | so oddly enough, i think the current code is reasonable? | 13:55 |
| paulsher1ood | but maybe there needs to be # TMPDIR entry in ybd.conf | 13:56 |
| tiagogomes_ | tbh I think it is reasonable to assume the presence of /tmp, either a directory, mounting point or symlink to the canonical place (e.g. /run/tmp) | 14:03 |
| tiagogomes_ | Or maybe not, if the build tool is to run on non-Linux systems. | 14:04 |
| pedroalvarez | stop thinking about building in linux | 14:07 |
| pedroalvarez | in windows, I mean | 14:07 |
| tiagogomes_ | I was about to say it would be easy with the Microsoft partnership with Canonical, but I doubt the they can translate the kernel syscalls needed for bubblewrap | 14:09 |
| *** toscalix has quit IRC | 14:10 | |
| *** toscalix has joined #baserock | 14:22 | |
| tiagogomes_ | I wonder if using ccache on the build essential components was tested. | 14:53 |
| *** ctbruce has quit IRC | 14:54 | |
| *** ctbruce has joined #baserock | 14:55 | |
| tiagogomes_ | Looks like it will not work, as it is not setting CCACHE_BASEDIR. But morph also doesn't set it | 15:11 |
| paulsher1ood | tiagogomes_: iirc, gtristan had ccache working in his aboriginal variant | 15:18 |
| *** ctbruce has quit IRC | 15:59 | |
| *** franred has quit IRC | 16:41 | |
| *** jude_ has quit IRC | 16:56 | |
| *** locallycompact has quit IRC | 17:29 | |
| *** toscalix has quit IRC | 17:38 | |
| *** jude_ has joined #baserock | 18:04 | |
| *** jude_ has quit IRC | 18:08 | |
| *** jude_ has joined #baserock | 18:08 | |
| *** gtristan has joined #baserock | 18:47 | |
| *** gtristan has quit IRC | 19:16 | |
| *** jude_ has quit IRC | 19:44 | |
| *** jude_ has joined #baserock | 19:52 | |
| *** jude_ has quit IRC | 22:36 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!