| *** edcragg has quit IRC | 00:38 | |
| *** thrace_ has joined #baserock | 00:48 | |
| *** thrace has quit IRC | 00:49 | |
| *** toscalix has joined #baserock | 01:10 | |
| *** persia has quit IRC | 01:35 | |
| *** persia has joined #baserock | 01:38 | |
| *** rdale has quit IRC | 02:40 | |
| *** rdale has joined #baserock | 02:41 | |
| *** toscalix has quit IRC | 02:58 | |
| *** gtristan has quit IRC | 04:54 | |
| *** gtristan has joined #baserock | 05:14 | |
| *** thrace_ has quit IRC | 05:40 | |
| *** thrace_ has joined #baserock | 05:46 | |
| *** thrace_ has quit IRC | 06:02 | |
| *** thrace_ has joined #baserock | 06:08 | |
| *** CTtpollard has joined #baserock | 07:55 | |
| *** edcragg has joined #baserock | 08:09 | |
| *** ctbruce has joined #baserock | 08:25 | |
| *** jjardon has quit IRC | 08:35 | |
| *** N-a-N has quit IRC | 08:36 | |
| *** lachlanmackenzie has quit IRC | 08:36 | |
| *** lachlanmackenzie has joined #baserock | 08:39 | |
| *** N-a-N has joined #baserock | 08:40 | |
| *** jjardon has joined #baserock | 08:43 | |
| *** bashrc has joined #baserock | 09:02 | |
| *** edcragg has quit IRC | 09:14 | |
| pedroalvarez | HI | 09:25 |
|---|---|---|
| pedroalvarez | I'm pondering paramiko as a way of uploading artifact to a cache server, passwordless and secure, using SSH | 09:26 |
| pedroalvarez | any thoughts? | 09:27 |
| pedroalvarez | This is to make workers 'push' the artifacts instead making the cache-server 'pull' the artifacts | 09:28 |
| pedroalvarez | one of my goals is to remove the need of accessing the workers from anywhere (cache-server, controller, etc) | 09:28 |
| *** edcragg has joined #baserock | 09:33 | |
| *** jonathanmaw has joined #baserock | 09:39 | |
| *** franred has joined #baserock | 09:46 | |
| gtristan | w00t, core stratum built... time for dinner | 10:10 |
| pedroalvarez | I'm a bit worried that we will forget about that weird behaviour with ybd/aboriginal | 10:12 |
| pedroalvarez | but I'm sure it will bite us in the future :) | 10:12 |
| *** gtristan has quit IRC | 10:16 | |
| edcragg | i saw some weird behaviour building in chroot recently, maybe similar | 10:55 |
| paulsherwood | pedroalvarez: i'm against further work on morph-cache-server. kbas is better | 10:59 |
| paulsherwood | (and ybd already pushes artifacts) | 11:00 |
| pedroalvarez | would you be interested on having a more secure way of pushing artifacts? | 11:01 |
| paulsherwood | 'weird behaviour' - please raise an issue, with as much info as you have, so we *don't* forget | 11:01 |
| paulsherwood | pedroalvarez: of course | 11:01 |
| pedroalvarez | well, the thing is that nothing needs to change in any cache server | 11:05 |
| pedroalvarez | so I'm ok with you being against working on morph-cache-server :P | 11:05 |
| *** jonathanmaw_ has joined #baserock | 11:09 | |
| paulsherwood | but you're changing morph? | 11:12 |
| *** jonathanmaw has quit IRC | 11:13 | |
| pedroalvarez | not changing, creating a basic PoC of a gearman distributed build network | 11:14 |
| pedroalvarez | I'm not going to touch distbuild. I want to create a replacement | 11:15 |
| paulsherwood | pedroalvarez: *why*? ybd has proved that there's no real need for a distbuild network already? | 11:15 |
| paulsherwood | and once tristan has the aboriginal stuff integrated... everything can happen on x86 anyway | 11:16 |
| pedroalvarez | because doing it is interesting, I'm learning new things, and I think it will be useful | 11:16 |
| paulsherwood | ok, fair enough :) | 11:17 |
| pedroalvarez | not everybody has access to big buld machines though | 11:17 |
| *** jonathanmaw_ has quit IRC | 11:17 | |
| *** CTtpollard has quit IRC | 11:17 | |
| paulsherwood | you can run ybd on *small* machines, sharing a cache server | 11:17 |
| *** jonathanmaw has joined #baserock | 11:17 | |
| *** CTtpollard has joined #baserock | 11:18 | |
| pedroalvarez | I know that works, but I don't like how that works | 11:18 |
| pedroalvarez | you have to access to these machines | 11:18 |
| pedroalvarez | trigger a build in every of them, etc | 11:19 |
| paulsherwood | ack | 11:19 |
| *** gtristan has joined #baserock | 11:20 | |
| paulsherwood | pedroalvarez: so you could try your gearman idea with ybd :) | 11:27 |
| pedroalvarez | yup | 11:29 |
| * paulsherwood wonders how this works without modifying cache server though | 11:30 | |
| pedroalvarez | paramiko will push over ssh, you won't need anything on the server side to receive the file | 11:30 |
| pedroalvarez | like with scp | 11:30 |
| pedroalvarez | well yes, I gues you need openssh running | 11:30 |
| paulsherwood | pedroalvarez: atomically? | 11:32 |
| pedroalvarez | I guess you can make it upload to a temporary place, and then rename the file atomically | 11:33 |
| paulsherwood | please don't guess... re-use that soln from kbas :) | 11:33 |
| paulsherwood | https://github.com/devcurmudgeon/ybd/blob/master/kbas/__main__.py#L133 | 11:34 |
| paulsherwood | it also checks that the uploded artifact is extractable, before doing the switch | 11:35 |
| pedroalvarez | looking at that code right now | 11:36 |
| paulsherwood | nb it's renaming a directory, not the artifact | 11:36 |
| *** CTtpollard has quit IRC | 11:36 | |
| pedroalvarez | paulsherwood: wouldn't be enough to call "tar -tf" to check that it's sane? | 11:36 |
| paulsherwood | Kinnison recommended this | 11:36 |
| paulsherwood | pedroalvarez: possibly | 11:36 |
| paulsherwood | if you think so, i'm happy to fix that | 11:37 |
| pedroalvarez | It will make it faster, but I don't know if that checking that would be enough, sorry | 11:37 |
| paulsherwood | does anyone else know? if we "tar -tf" ok, can we be confidend that the file is a valid tarball that can be extracted? | 11:38 |
| paulsherwood | rjek, for example? | 11:39 |
| rjek | Try passing tar a file that isn't a tar and see what the exist status is :) | 11:40 |
| pedroalvarez | echo $? | 11:41 |
| pedroalvarez | 2 | 11:41 |
| paulsherwood | that's not *quite* the same thing... | 11:41 |
| rjek | Now pass it a tar file and see if it returns zero. | 11:42 |
| rjek | t can be expensive; it does involve processing the entire tar file | 11:42 |
| paulsherwood | is it possible that a corrupted tarfile could still behave correctly for -tf | 11:42 |
| rjek | `file` might be quicker to identify if it /looks/ like a tar | 11:42 |
| rjek | paulsherwood: I think the whole point of -t is to "test", but it may be implementation-specific | 11:42 |
| paulsherwood | the requrement is to be sure we can untar it, not to know if it is tar-like | 11:43 |
| rjek | -t is as expensive as -x | 11:43 |
| paulsherwood | ah, ok | 11:43 |
| rjek | In every way apart from disc space usage | 11:43 |
| rjek | So if you want to be sure, extract to a temp. location, check return code, delete temp. location | 11:43 |
| paulsherwood | ack, thanks | 11:45 |
| rjek | Are the tar files from a trusted source? | 11:45 |
| pedroalvarez | ybd, so, no | 11:46 |
| pedroalvarez | hahah jk | 11:46 |
| rjek | :) | 11:46 |
| pedroalvarez | rjek: does that change anything? | 11:46 |
| rjek | It would be a shame if somebody could upload arbitrary tars and own a machine | 11:46 |
| rjek | pedroalvarez: The tar might contain a file called ../../../../../../../../../../../../../../../../../../../../../../etc/shadow | 11:46 |
| pedroalvarez | ouch! | 11:47 |
| paulsherwood | pedroalvarez: would you mind offering a patch for that, then? | 11:47 |
| *** CTtpollard has joined #baserock | 11:50 | |
| pedroalvarez | Will do if I find the time | 11:51 |
| paulsherwood | i can do it :) | 11:53 |
| paulsherwood | done | 11:56 |
| paulsherwood | gtristan: http://paste.baserock.org/awezodelaw | 12:27 |
| paulsherwood | and http://sprunge.us/AHOe | 12:29 |
| pedroalvarez | that's why it was a weird error | 12:30 |
| paulsherwood | i didn't understand the conversation... was just responding to gtristan's question if anyone coudl build it | 12:31 |
| paulsherwood | and still there's no issue actually describing the problem, afaict? :) | 12:31 |
| pedroalvarez | I assume the error is caused by some of the changes he is doing in build-essential | 12:31 |
| pedroalvarez | what do you mean? | 12:32 |
| gtristan | paulsherwood, the problem was only in ybd-aboriginal, however the patch removing libtool.morph removes a redundant step in building libtool, since it otherwise runs ./bootstrap twice | 12:43 |
| paulsherwood | pedroalvarez: i must have misunderstood, sorry | 12:50 |
| paulsherwood | i thought folks had observed weird behaviour in ybd. i would want to fix that :) | 12:51 |
| gtristan | paulsherwood, I suspected it might have been, but it wasnt, rdale confirmed it was building | 13:00 |
| *** N-a-N has quit IRC | 13:06 | |
| gtristan | What creates /etc/mtab as a symlink to /proc/mounts ? | 13:13 |
| gtristan | this seems to be done by ybd in the install dir of the fhs-dirs.morph chunk | 13:13 |
| benbrown_ | is there something up with http://openid.baserock.org/ ? I'm getting "504 Gateway Timeout" | 13:40 |
| pedroalvarez | benbrown_: wfm, what are you doing to get that error? | 13:41 |
| benbrown_ | trying to sign in to gerrit.baserock.org | 13:42 |
| benbrown_ | although it just worked | 13:42 |
| * benbrown_ confusses | 13:42 | |
| gtristan | fwiw, virtfs craziness works: https://github.com/gtristan/ybd/blob/aboriginal/ybd/virtfs.py | 13:52 |
| * gtristan gotta run | 13:52 | |
| paulsherwood | gtristan: seems like most of your crazy plans now work :-) | 13:58 |
| paulsherwood | w00t :) | 13:58 |
| *** gtristan has quit IRC | 14:02 | |
| *** tiagogomes_ has joined #baserock | 14:52 | |
| *** tiagogomes has quit IRC | 14:55 | |
| *** N-a-N has joined #baserock | 15:16 | |
| *** ctbruce has quit IRC | 15:51 | |
| *** CTtpollard has quit IRC | 16:41 | |
| *** gtristan has joined #baserock | 16:48 | |
| *** franred has quit IRC | 17:19 | |
| *** jonathanmaw has quit IRC | 17:28 | |
| pedroalvarez | edcragg: btw, did you test that this patch is needed? https://gerrit.baserock.org/#/c/1932/ | 17:31 |
| *** tiagogomes_ has quit IRC | 17:33 | |
| edcragg | pedroalvarez: it doesn't seem to be strictly needed at the moment, abandoned | 17:42 |
| pedroalvarez | oh, ok | 17:42 |
| pedroalvarez | I just wanted to remove my -2 :) | 17:42 |
| edcragg | oh :) | 17:45 |
| *** tiagogomes_ has joined #baserock | 17:45 | |
| edcragg | it might help to have them eventually, but it doesn't seem to be required to build something runnable at the moment | 17:45 |
| pedroalvarez | right | 17:46 |
| pedroalvarez | i'd like to eventually fix the real error that those lines in the morphology try to fix | 17:46 |
| * edcragg didn't realise they were a fix | 17:46 | |
| edcragg | i thought it was more for architecture dependent optimisation | 17:47 |
| pedroalvarez | a bug in GMP, iirc | 17:48 |
| *** bashrc has quit IRC | 18:01 | |
| *** rdale has quit IRC | 19:56 | |
| *** edcragg has quit IRC | 20:04 | |
| *** N-a-N has quit IRC | 20:26 | |
| *** edcragg has joined #baserock | 20:44 | |
| *** cosm_ has joined #baserock | 21:20 | |
| *** cosm has quit IRC | 21:23 | |
| *** edcragg has quit IRC | 22:05 | |
| *** edcragg has joined #baserock | 22:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!