IRC logs for #baserock for Thursday, 2016-03-10

*** edcragg has quit IRC00:38
*** thrace_ has joined #baserock00:48
*** thrace has quit IRC00:49
*** toscalix has joined #baserock01:10
*** persia has quit IRC01:35
*** persia has joined #baserock01:38
*** rdale has quit IRC02:40
*** rdale has joined #baserock02:41
*** toscalix has quit IRC02:58
*** gtristan has quit IRC04:54
*** gtristan has joined #baserock05:14
*** thrace_ has quit IRC05:40
*** thrace_ has joined #baserock05:46
*** thrace_ has quit IRC06:02
*** thrace_ has joined #baserock06:08
*** CTtpollard has joined #baserock07:55
*** edcragg has joined #baserock08:09
*** ctbruce has joined #baserock08:25
*** jjardon has quit IRC08:35
*** N-a-N has quit IRC08:36
*** lachlanmackenzie has quit IRC08:36
*** lachlanmackenzie has joined #baserock08:39
*** N-a-N has joined #baserock08:40
*** jjardon has joined #baserock08:43
*** bashrc has joined #baserock09:02
*** edcragg has quit IRC09:14
pedroalvarezHI09:25
pedroalvarezI'm pondering paramiko as a way of uploading artifact to a cache server, passwordless and secure, using SSH09:26
pedroalvarezany thoughts?09:27
pedroalvarezThis is to make workers 'push' the artifacts instead making the cache-server 'pull' the artifacts09:28
pedroalvarezone of my goals is to remove the need of accessing the workers from anywhere (cache-server, controller, etc)09:28
*** edcragg has joined #baserock09:33
*** jonathanmaw has joined #baserock09:39
*** franred has joined #baserock09:46
gtristanw00t, core stratum built... time for dinner10:10
pedroalvarezI'm a bit worried that we will forget about that weird behaviour with ybd/aboriginal10:12
pedroalvarezbut I'm sure it will bite us in the future :)10:12
*** gtristan has quit IRC10:16
edcraggi saw some weird behaviour building in chroot recently, maybe similar10:55
paulsherwoodpedroalvarez: i'm against further work on morph-cache-server. kbas is better10:59
paulsherwood(and ybd already pushes artifacts)11:00
pedroalvarezwould you be interested on having a more secure way of pushing artifacts?11:01
paulsherwood'weird behaviour' - please raise an issue, with as much info as you have, so we *don't* forget11:01
paulsherwoodpedroalvarez: of course11:01
pedroalvarezwell, the thing is that nothing needs to change in any cache server11:05
pedroalvarezso I'm ok with you being against working on morph-cache-server :P11:05
*** jonathanmaw_ has joined #baserock11:09
paulsherwoodbut you're changing morph?11:12
*** jonathanmaw has quit IRC11:13
pedroalvareznot changing, creating a basic PoC of a gearman distributed build network11:14
pedroalvarezI'm not going to touch distbuild. I want to create a replacement11:15
paulsherwoodpedroalvarez: *why*? ybd has proved that there's no real need for a distbuild network already?11:15
paulsherwoodand once tristan has the aboriginal stuff integrated... everything can happen on x86 anyway11:16
pedroalvarezbecause doing it is interesting, I'm learning new things, and I think it will be useful11:16
paulsherwoodok, fair enough :)11:17
pedroalvareznot everybody has access to big buld machines though11:17
*** jonathanmaw_ has quit IRC11:17
*** CTtpollard has quit IRC11:17
paulsherwoodyou can run ybd on *small* machines, sharing a cache server11:17
*** jonathanmaw has joined #baserock11:17
*** CTtpollard has joined #baserock11:18
pedroalvarezI know that works, but I don't like how that works11:18
pedroalvarezyou have to access to these machines11:18
pedroalvareztrigger a build in every of them, etc11:19
paulsherwoodack11:19
*** gtristan has joined #baserock11:20
paulsherwoodpedroalvarez: so you could try your gearman idea with ybd :)11:27
pedroalvarezyup11:29
* paulsherwood wonders how this works without modifying cache server though11:30
pedroalvarezparamiko will push over ssh, you won't need anything on the server side to receive the file11:30
pedroalvarezlike with scp11:30
pedroalvarezwell yes, I gues you need openssh running11:30
paulsherwoodpedroalvarez: atomically?11:32
pedroalvarezI guess you can make it upload to a temporary place, and then rename the file atomically11:33
paulsherwoodplease don't guess... re-use that soln from kbas :)11:33
paulsherwoodhttps://github.com/devcurmudgeon/ybd/blob/master/kbas/__main__.py#L13311:34
paulsherwoodit also checks that the uploded artifact is extractable, before doing the switch11:35
pedroalvarezlooking at that code right now11:36
paulsherwoodnb it's renaming a directory, not the artifact11:36
*** CTtpollard has quit IRC11:36
pedroalvarezpaulsherwood: wouldn't be enough to call "tar -tf" to check that it's sane?11:36
paulsherwoodKinnison recommended this11:36
paulsherwoodpedroalvarez: possibly11:36
paulsherwoodif you think so, i'm happy to fix that11:37
pedroalvarezIt will make it faster, but I don't know if that checking that would be enough, sorry11:37
paulsherwooddoes anyone else know? if we "tar -tf" ok, can we be confidend that the file is a valid tarball that can be extracted?11:38
paulsherwoodrjek, for example?11:39
rjekTry passing tar a file that isn't a tar and see what the exist status is :)11:40
pedroalvarezecho $?11:41
pedroalvarez211:41
paulsherwoodthat's not *quite* the same thing...11:41
rjekNow pass it a tar file and see if it returns zero.11:42
rjekt can be expensive; it does involve processing the entire tar file11:42
paulsherwoodis it possible that a corrupted tarfile could still behave correctly for -tf11:42
rjek`file` might be quicker to identify if it /looks/ like a tar11:42
rjekpaulsherwood: I think the whole point of -t is to "test", but it may be implementation-specific11:42
paulsherwoodthe requrement is to be sure we can untar it, not to know if it is tar-like11:43
rjek-t is as expensive as -x11:43
paulsherwoodah, ok11:43
rjekIn every way apart from disc space usage11:43
rjekSo if you want to be sure, extract to a temp. location, check return code, delete temp. location11:43
paulsherwoodack, thanks11:45
rjekAre the tar files from a trusted source?11:45
pedroalvarezybd, so, no11:46
pedroalvarezhahah jk11:46
rjek:)11:46
pedroalvarezrjek: does that change anything?11:46
rjekIt would be a shame if somebody could upload arbitrary tars and own a machine11:46
rjekpedroalvarez: The tar might contain a file called ../../../../../../../../../../../../../../../../../../../../../../etc/shadow11:46
pedroalvarezouch!11:47
paulsherwoodpedroalvarez: would you mind offering a patch for that, then?11:47
*** CTtpollard has joined #baserock11:50
pedroalvarezWill do if I find the time11:51
paulsherwoodi can do it :)11:53
paulsherwooddone11:56
paulsherwoodgtristan: http://paste.baserock.org/awezodelaw12:27
paulsherwoodand http://sprunge.us/AHOe12:29
pedroalvarezthat's why it was a weird error12:30
paulsherwoodi didn't understand the conversation... was just responding to gtristan's question if anyone coudl build it12:31
paulsherwoodand still there's no issue actually describing the problem, afaict? :)12:31
pedroalvarezI assume the error is caused by some of the changes he is doing in build-essential12:31
pedroalvarezwhat do you mean?12:32
gtristanpaulsherwood, the problem was only in ybd-aboriginal, however the patch removing libtool.morph removes a redundant step in building libtool, since it otherwise runs ./bootstrap twice12:43
paulsherwoodpedroalvarez: i must have misunderstood, sorry12:50
paulsherwoodi thought folks had observed weird behaviour in ybd. i would want to fix that :)12:51
gtristanpaulsherwood, I suspected it might have been, but it wasnt, rdale confirmed it was building13:00
*** N-a-N has quit IRC13:06
gtristanWhat creates /etc/mtab as a symlink to /proc/mounts ?13:13
gtristanthis seems to be done by ybd in the install dir of the fhs-dirs.morph chunk13:13
benbrown_is there something up with http://openid.baserock.org/ ? I'm getting "504 Gateway Timeout"13:40
pedroalvarezbenbrown_: wfm, what are you doing to get that error?13:41
benbrown_trying to sign in to gerrit.baserock.org13:42
benbrown_although it just worked13:42
* benbrown_ confusses13:42
gtristanfwiw, virtfs craziness works: https://github.com/gtristan/ybd/blob/aboriginal/ybd/virtfs.py13:52
* gtristan gotta run13:52
paulsherwoodgtristan: seems like most of your crazy plans now work :-)13:58
paulsherwoodw00t :)13:58
*** gtristan has quit IRC14:02
*** tiagogomes_ has joined #baserock14:52
*** tiagogomes has quit IRC14:55
*** N-a-N has joined #baserock15:16
*** ctbruce has quit IRC15:51
*** CTtpollard has quit IRC16:41
*** gtristan has joined #baserock16:48
*** franred has quit IRC17:19
*** jonathanmaw has quit IRC17:28
pedroalvarezedcragg: btw, did you test that this patch is needed? https://gerrit.baserock.org/#/c/1932/17:31
*** tiagogomes_ has quit IRC17:33
edcraggpedroalvarez: it doesn't seem to be strictly needed at the moment, abandoned17:42
pedroalvarezoh, ok17:42
pedroalvarezI just wanted to remove my -2 :)17:42
edcraggoh :)17:45
*** tiagogomes_ has joined #baserock17:45
edcraggit might help to have them eventually, but it doesn't seem to be required to build something runnable at the moment17:45
pedroalvarezright17:46
pedroalvarezi'd like to eventually fix the real error that those lines in the morphology try to fix17:46
* edcragg didn't realise they were a fix17:46
edcraggi thought it was more for architecture dependent optimisation17:47
pedroalvareza bug in GMP, iirc17:48
*** bashrc has quit IRC18:01
*** rdale has quit IRC19:56
*** edcragg has quit IRC20:04
*** N-a-N has quit IRC20:26
*** edcragg has joined #baserock20:44
*** cosm_ has joined #baserock21:20
*** cosm has quit IRC21:23
*** edcragg has quit IRC22:05
*** edcragg has joined #baserock22:46

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!