IRC logs for #baserock for Tuesday, 2015-09-15

*** zoli__ has joined #baserock00:17
*** zoli___ has quit IRC00:17
*** zoli__ has quit IRC03:03
*** zoli__ has joined #baserock03:03
*** persia has quit IRC03:29
*** persia has joined #baserock03:30
*** persia has quit IRC03:30
*** persia has joined #baserock03:30
*** persia has quit IRC03:41
*** persia has joined #baserock03:42
*** zoli__ has quit IRC04:08
*** zoli__ has joined #baserock04:37
*** zoli__ has quit IRC04:52
*** zoli__ has joined #baserock04:53
*** zoli___ has joined #baserock05:22
*** zoli__ has quit IRC05:22
*** zoli___ has quit IRC06:04
*** paulw has joined #baserock06:44
richard_maw20:34 < paulsherwood> http://stackoverflow.com/questions/5255220/fcntl-flock-how-to-implement-a-timeout ?07:11
richard_mawyep, that's how you'd implement it directly in python07:11
richard_mawand it requires you to mess with your process global state07:12
richard_mawand since you can only have one alarm for the process, things get difficult if you ever need to take multiple locks07:14
richard_mawor want to use SIGALRM for anything else07:14
richard_mawbasically… the API Linux provides for taking a lock with a timeout is not compartmentalisable07:15
richard_mawsince it causes side-effects throughtout your whole process07:15
richard_mawhence why I think forking off a subprocess to take the lock is more appropriate, and the neatest way to do that is to exec the flock command07:17
*** zoli__ has joined #baserock07:22
* richard_maw resolves to discover why YBD on AWS is making artifacts with broken directory permissions07:47
richard_mawhm, well first data point: the staging areas' root directory has incorrect permissions07:48
richard_mawhm, not by itself sufficient to cause problems, since morph on Baserock has the same permissions for its tempdirs07:50
*** toscalix__ has joined #baserock08:03
*** bashrc has joined #baserock08:04
richard_mawfound the problem08:10
richard_mawpython-twisted, in its ineffible wisdom decided that umask for processes should default to 07708:10
richard_mawtomorrow I will submit patches to morph and YBD to force the umask to something sensible08:15
paulsherwoodrichard_maw: strange... ybd doesn't use twisted? :)08:18
pedroalvarezI believe this is buildbot when calling ybd08:19
paulsherwoodaha08:20
richard_mawI wouldn't recommend making it use twisted either08:22
* richard_maw is unhappy with the framework for other reasons too08:23
richard_mawhttps://github.com/twisted/twisted/tree/trunk/twisted/spread08:28
*** mariaderidder has joined #baserock08:29
*** zoli__ has quit IRC08:29
*** zoli__ has joined #baserock08:57
*** ssam2 has joined #baserock09:15
*** ChanServ sets mode: +v ssam209:15
*** bashrc_ has joined #baserock09:19
*** bashrc has quit IRC09:19
richard_mawpaulsherwood: we got a OSError: [Errno 2] No such file or directory: '/src/gits/git___cu010_trove_codethink_com_delta_glib.tmp' error while building, I take it that is what https://github.com/devcurmudgeon/ybd/commit/21d2da9168db7c0ce55e7dda2c7907d3aea9ca78 was for?10:26
*** franred has joined #baserock10:29
paulsherwoodrichard_maw: yup. please upgrade10:30
*** persia_ has quit IRC10:31
paulsherwoodrichard_maw: not sure if it's clear, but simplest thing is to put custom config in a new ybd.conf file in the ybd directory - custom settings will override default config/ybd.conf and avoid clashes with git merges10:36
richard_mawpaulsherwood: have you considered using locks in your artifact cache to prevent your concurrent builders from attempting to build the same component10:39
paulsherwoodrichard_maw: not so far10:40
richard_mawit's worth a ponder10:40
paulsherwoodi agree10:40
paulsherwoodin some cases it would lead to builders having to wait, but in others they could move to next-in-line10:41
paulsherwoodnow i've got a better understanding of locks, i may be able to attempt it :)10:41
paulsherwoodbut ideally the soln would be generic across n machines, not just builders in the same machine?10:42
richard_mawyep, take your list of potential things to build, shuffle it, attempt to take an exclusive lock, and if you can't remove it from the list and try something else, if you have an empty list then take a blocking lock on one of the components you depend on at random10:42
richard_mawyou'd need distributed locking at that point, which is beyond my experience10:43
KinnisonNFS-safe locks?10:43
richard_mawKinnison: with a recent enough and appropriately configured NFS server, flock works10:43
Kinnisonoooh flock() works over NFSv4 ?10:43
richard_mawI think so, if you have a lockd working10:44
Kinnisonnice10:44
paulsherwoodactually, maybe it would just be a call to the artifact server and let it do the locking10:44
paulsherwoodi don't think we can force depend on a given version of NFS?10:45
richard_mawpaulsherwood: then you can't rely on flock on NFS: http://0pointer.de/blog/projects/locking.html10:47
ssam2seems there would also be a risk of a process crashing, leaving a stale lock and blocking subsequent builds forever10:52
richard_mawssam2: that's why flock's good, the lock is automatically released if the process crashes10:53
ssam2oh, that's nice10:53
richard_mawthe article is a little dated, as since then (in 3.15) the F_OFD_SETLK family of locking fcntl operations were added, which are like flock, but let you specify a range of data within the file to lock10:56
richard_mawso yeah, locking on old kernels isn't great10:59
pdarpaulsherwood: heya, I was playing with the instance number when deploying and noticed that the deploys failed when the number of instances was 5, but worked fine when it was 1. I cant remember the error I got but I can rerun and report further if it isnt already known about?12:41
richard_mawpdar: can you send me a link to the log in private?12:42
pdardoes ybd store its logs anywhere?12:49
*** mariaderidder has quit IRC13:06
*** toscalix__ is now known as toscalix13:16
*** mariaderidder has joined #baserock13:20
radiofreepdar: stdout only i think13:21
paulsherwoodpdar: only where it says it does (build logs)13:21
tiagogomes_ybd ybd ybd, is morph dead? Just curious13:23
pedroalvarezno, morph is just finished, and users don't complain about it not working :P13:24
paulsherwoodheh13:24
richard_mawtiagogomes_: AFAICT it's just that paulsherwood is the only one reliably able to find time to work on it13:24
pedroalvarezjust kidding :)13:24
richard_mawthat, and for various environmental reasons, we couldn't use morph for my current task, since YBD was the path of least resistance13:26
richard_mawwhich unfortunately means there wasn't time to make morph usable there13:27
* richard_maw hopes to solve this by making it possible to build AWS compatible systems in Baserock13:36
ratmice___hmm, wrt -j wonder how much pre-build artifact installs are priming the fs cache/negating one of the benefits seen with -j (vs on a non-build dedicated machine)14:12
*** bashrc_ has quit IRC14:16
*** zoli__ has quit IRC14:17
*** zoli__ has joined #baserock14:18
*** ssam2 has quit IRC14:18
*** nowster has quit IRC14:18
*** edcragg_ has joined #baserock14:19
*** edcragg has quit IRC14:19
*** gary_perkins has quit IRC14:19
*** tiagogomes_ has quit IRC14:19
*** petefoth has quit IRC14:19
*** flatmush has quit IRC14:19
*** paulw has quit IRC14:19
*** fay_ has quit IRC14:20
*** mariaderidder has quit IRC14:20
*** ssam2 has joined #baserock14:20
*** ChanServ sets mode: +v ssam214:20
*** paulw has joined #baserock14:20
*** mariaderidder has joined #baserock14:20
*** gary_perkins has joined #baserock14:20
*** fay_ has joined #baserock14:21
*** tiagogomes_ has joined #baserock14:23
*** zoli__ has quit IRC14:24
*** gary_perkins has quit IRC14:25
*** mariaderidder has quit IRC14:25
*** mariaderidder has joined #baserock14:25
*** bashrc has joined #baserock14:27
*** paulwaters_ has joined #baserock14:27
*** bashrc has quit IRC14:28
*** tiagogomes_ has quit IRC14:31
*** mariaderidder has quit IRC14:31
*** paulw has quit IRC14:32
*** tiagogomes_ has joined #baserock14:32
*** ssam2 has quit IRC14:39
*** fay_ has quit IRC14:39
*** tiagogomes_ has quit IRC14:42
*** edcragg_ has quit IRC14:44
*** ssam2 has joined #baserock14:52
*** ChanServ sets mode: +v ssam214:52
*** fay_ has joined #baserock14:52
*** tiagogomes_ has joined #baserock14:52
*** bashrc has joined #baserock14:52
*** nowster has joined #baserock14:52
*** flatmush has joined #baserock14:54
paulsherwoodratmice___: i'm unable to comment on that, maybe others understand this better14:54
*** edcragg has joined #baserock14:56
*** gary_perkins has joined #baserock14:58
*** gary_perkins has joined #baserock14:58
*** paulwaters_ has quit IRC14:59
*** mariaderidder has joined #baserock15:06
*** straycat has joined #baserock15:18
straycatHi, I've created http://git.baserock.org/cgi-bin/cgit.cgi/delta/cpan/List-MoreUtils-tarball.git/commit/?h=baserock/richardipsum/no-dotgit-assumption I'd like to push a ref at baserock/master to point to this. I'll hopefully be sending some definitions soon which will add List-MoreUtils amongst others to a perl-common stratum.15:19
richard_mawcool15:21
*** toscalix has quit IRC15:24
straycatrichard_maw, is that a +1? :) i'll wait around for another +1 in any case before pushing anything15:29
richard_mawI'm not sure I parsed a request in there15:29
* richard_maw re-parses15:29
wdutchrichard_maw: I'm having trouble with the other irc network15:30
wdutchlet me reboot orchestration again, more patches15:30
straycatsorry yes, I'm asking for permission to push a baserock/master ref to that :)15:30
richard_mawstraycat: yep, I see what you're doing now. Certainly, count it as a +115:31
richard_mawstraycat: call it a +2 if you don't get any objections for an hour or so15:32
ssam2is it possible to fix this upstream?15:32
ssam2it's awkward carrying patches for things15:32
ssam2+1 for this as a temporary solution, but would be nice if we could avoid actually forking it15:32
richard_mawthe alternative would be to put something in pre-configure-commands to either remove those lines, or rm the .git15:32
richard_mawor rename it15:33
* straycat nods15:35
straycatI can try and find the maintainer and ask them about it15:35
ssam2that'd be nice15:35
richard_mawMy talk submission for systemd.conf has been accepted15:50
pedroalvarezrichard_maw: is it baserock related? (to include it here http://wiki.baserock.org/conferences/)15:52
straycatrichard_maw, cool :)15:52
Kinnisonrichard_maw: congratulations15:53
paulsherwoodrichard_maw: w00t!16:00
richard_mawpedroalvarez: in that we build systems that include systemd and do atomic updates, yes, and if we end up with a consensus that it's something systemd should do, I'd like to add support to Baserock16:02
pedroalvareznice!16:05
*** mariaderidder has quit IRC16:58
*** bashrc has quit IRC17:02
*** Walkerdine has joined #baserock17:02
*** ssam2 has quit IRC17:28
*** Walkerdine has quit IRC17:38
*** Walkerdine has joined #baserock17:55
*** zoli__ has joined #baserock18:05
*** franred has quit IRC18:09
*** franred has joined #baserock18:16
*** Walkerdine has quit IRC18:34
*** zoli__ has quit IRC18:37
*** franred has quit IRC18:48
*** zoli__ has joined #baserock22:18
*** zoli__ has quit IRC22:44

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!