| *** zoli__ has joined #baserock | 00:17 | |
| *** zoli___ has quit IRC | 00:17 | |
| *** zoli__ has quit IRC | 03:03 | |
| *** zoli__ has joined #baserock | 03:03 | |
| *** persia has quit IRC | 03:29 | |
| *** persia has joined #baserock | 03:30 | |
| *** persia has quit IRC | 03:30 | |
| *** persia has joined #baserock | 03:30 | |
| *** persia has quit IRC | 03:41 | |
| *** persia has joined #baserock | 03:42 | |
| *** zoli__ has quit IRC | 04:08 | |
| *** zoli__ has joined #baserock | 04:37 | |
| *** zoli__ has quit IRC | 04:52 | |
| *** zoli__ has joined #baserock | 04:53 | |
| *** zoli___ has joined #baserock | 05:22 | |
| *** zoli__ has quit IRC | 05:22 | |
| *** zoli___ has quit IRC | 06:04 | |
| *** paulw has joined #baserock | 06:44 | |
| richard_maw | 20:34 < paulsherwood> http://stackoverflow.com/questions/5255220/fcntl-flock-how-to-implement-a-timeout ? | 07:11 |
|---|---|---|
| richard_maw | yep, that's how you'd implement it directly in python | 07:11 |
| richard_maw | and it requires you to mess with your process global state | 07:12 |
| richard_maw | and since you can only have one alarm for the process, things get difficult if you ever need to take multiple locks | 07:14 |
| richard_maw | or want to use SIGALRM for anything else | 07:14 |
| richard_maw | basically… the API Linux provides for taking a lock with a timeout is not compartmentalisable | 07:15 |
| richard_maw | since it causes side-effects throughtout your whole process | 07:15 |
| richard_maw | hence why I think forking off a subprocess to take the lock is more appropriate, and the neatest way to do that is to exec the flock command | 07:17 |
| *** zoli__ has joined #baserock | 07:22 | |
| * richard_maw resolves to discover why YBD on AWS is making artifacts with broken directory permissions | 07:47 | |
| richard_maw | hm, well first data point: the staging areas' root directory has incorrect permissions | 07:48 |
| richard_maw | hm, not by itself sufficient to cause problems, since morph on Baserock has the same permissions for its tempdirs | 07:50 |
| *** toscalix__ has joined #baserock | 08:03 | |
| *** bashrc has joined #baserock | 08:04 | |
| richard_maw | found the problem | 08:10 |
| richard_maw | python-twisted, in its ineffible wisdom decided that umask for processes should default to 077 | 08:10 |
| richard_maw | tomorrow I will submit patches to morph and YBD to force the umask to something sensible | 08:15 |
| paulsherwood | richard_maw: strange... ybd doesn't use twisted? :) | 08:18 |
| pedroalvarez | I believe this is buildbot when calling ybd | 08:19 |
| paulsherwood | aha | 08:20 |
| richard_maw | I wouldn't recommend making it use twisted either | 08:22 |
| * richard_maw is unhappy with the framework for other reasons too | 08:23 | |
| richard_maw | https://github.com/twisted/twisted/tree/trunk/twisted/spread | 08:28 |
| *** mariaderidder has joined #baserock | 08:29 | |
| *** zoli__ has quit IRC | 08:29 | |
| *** zoli__ has joined #baserock | 08:57 | |
| *** ssam2 has joined #baserock | 09:15 | |
| *** ChanServ sets mode: +v ssam2 | 09:15 | |
| *** bashrc_ has joined #baserock | 09:19 | |
| *** bashrc has quit IRC | 09:19 | |
| richard_maw | paulsherwood: we got a OSError: [Errno 2] No such file or directory: '/src/gits/git___cu010_trove_codethink_com_delta_glib.tmp' error while building, I take it that is what https://github.com/devcurmudgeon/ybd/commit/21d2da9168db7c0ce55e7dda2c7907d3aea9ca78 was for? | 10:26 |
| *** franred has joined #baserock | 10:29 | |
| paulsherwood | richard_maw: yup. please upgrade | 10:30 |
| *** persia_ has quit IRC | 10:31 | |
| paulsherwood | richard_maw: not sure if it's clear, but simplest thing is to put custom config in a new ybd.conf file in the ybd directory - custom settings will override default config/ybd.conf and avoid clashes with git merges | 10:36 |
| richard_maw | paulsherwood: have you considered using locks in your artifact cache to prevent your concurrent builders from attempting to build the same component | 10:39 |
| paulsherwood | richard_maw: not so far | 10:40 |
| richard_maw | it's worth a ponder | 10:40 |
| paulsherwood | i agree | 10:40 |
| paulsherwood | in some cases it would lead to builders having to wait, but in others they could move to next-in-line | 10:41 |
| paulsherwood | now i've got a better understanding of locks, i may be able to attempt it :) | 10:41 |
| paulsherwood | but ideally the soln would be generic across n machines, not just builders in the same machine? | 10:42 |
| richard_maw | yep, take your list of potential things to build, shuffle it, attempt to take an exclusive lock, and if you can't remove it from the list and try something else, if you have an empty list then take a blocking lock on one of the components you depend on at random | 10:42 |
| richard_maw | you'd need distributed locking at that point, which is beyond my experience | 10:43 |
| Kinnison | NFS-safe locks? | 10:43 |
| richard_maw | Kinnison: with a recent enough and appropriately configured NFS server, flock works | 10:43 |
| Kinnison | oooh flock() works over NFSv4 ? | 10:43 |
| richard_maw | I think so, if you have a lockd working | 10:44 |
| Kinnison | nice | 10:44 |
| paulsherwood | actually, maybe it would just be a call to the artifact server and let it do the locking | 10:44 |
| paulsherwood | i don't think we can force depend on a given version of NFS? | 10:45 |
| richard_maw | paulsherwood: then you can't rely on flock on NFS: http://0pointer.de/blog/projects/locking.html | 10:47 |
| ssam2 | seems there would also be a risk of a process crashing, leaving a stale lock and blocking subsequent builds forever | 10:52 |
| richard_maw | ssam2: that's why flock's good, the lock is automatically released if the process crashes | 10:53 |
| ssam2 | oh, that's nice | 10:53 |
| richard_maw | the article is a little dated, as since then (in 3.15) the F_OFD_SETLK family of locking fcntl operations were added, which are like flock, but let you specify a range of data within the file to lock | 10:56 |
| richard_maw | so yeah, locking on old kernels isn't great | 10:59 |
| pdar | paulsherwood: heya, I was playing with the instance number when deploying and noticed that the deploys failed when the number of instances was 5, but worked fine when it was 1. I cant remember the error I got but I can rerun and report further if it isnt already known about? | 12:41 |
| richard_maw | pdar: can you send me a link to the log in private? | 12:42 |
| pdar | does ybd store its logs anywhere? | 12:49 |
| *** mariaderidder has quit IRC | 13:06 | |
| *** toscalix__ is now known as toscalix | 13:16 | |
| *** mariaderidder has joined #baserock | 13:20 | |
| radiofree | pdar: stdout only i think | 13:21 |
| paulsherwood | pdar: only where it says it does (build logs) | 13:21 |
| tiagogomes_ | ybd ybd ybd, is morph dead? Just curious | 13:23 |
| pedroalvarez | no, morph is just finished, and users don't complain about it not working :P | 13:24 |
| paulsherwood | heh | 13:24 |
| richard_maw | tiagogomes_: AFAICT it's just that paulsherwood is the only one reliably able to find time to work on it | 13:24 |
| pedroalvarez | just kidding :) | 13:24 |
| richard_maw | that, and for various environmental reasons, we couldn't use morph for my current task, since YBD was the path of least resistance | 13:26 |
| richard_maw | which unfortunately means there wasn't time to make morph usable there | 13:27 |
| * richard_maw hopes to solve this by making it possible to build AWS compatible systems in Baserock | 13:36 | |
| ratmice___ | hmm, wrt -j wonder how much pre-build artifact installs are priming the fs cache/negating one of the benefits seen with -j (vs on a non-build dedicated machine) | 14:12 |
| *** bashrc_ has quit IRC | 14:16 | |
| *** zoli__ has quit IRC | 14:17 | |
| *** zoli__ has joined #baserock | 14:18 | |
| *** ssam2 has quit IRC | 14:18 | |
| *** nowster has quit IRC | 14:18 | |
| *** edcragg_ has joined #baserock | 14:19 | |
| *** edcragg has quit IRC | 14:19 | |
| *** gary_perkins has quit IRC | 14:19 | |
| *** tiagogomes_ has quit IRC | 14:19 | |
| *** petefoth has quit IRC | 14:19 | |
| *** flatmush has quit IRC | 14:19 | |
| *** paulw has quit IRC | 14:19 | |
| *** fay_ has quit IRC | 14:20 | |
| *** mariaderidder has quit IRC | 14:20 | |
| *** ssam2 has joined #baserock | 14:20 | |
| *** ChanServ sets mode: +v ssam2 | 14:20 | |
| *** paulw has joined #baserock | 14:20 | |
| *** mariaderidder has joined #baserock | 14:20 | |
| *** gary_perkins has joined #baserock | 14:20 | |
| *** fay_ has joined #baserock | 14:21 | |
| *** tiagogomes_ has joined #baserock | 14:23 | |
| *** zoli__ has quit IRC | 14:24 | |
| *** gary_perkins has quit IRC | 14:25 | |
| *** mariaderidder has quit IRC | 14:25 | |
| *** mariaderidder has joined #baserock | 14:25 | |
| *** bashrc has joined #baserock | 14:27 | |
| *** paulwaters_ has joined #baserock | 14:27 | |
| *** bashrc has quit IRC | 14:28 | |
| *** tiagogomes_ has quit IRC | 14:31 | |
| *** mariaderidder has quit IRC | 14:31 | |
| *** paulw has quit IRC | 14:32 | |
| *** tiagogomes_ has joined #baserock | 14:32 | |
| *** ssam2 has quit IRC | 14:39 | |
| *** fay_ has quit IRC | 14:39 | |
| *** tiagogomes_ has quit IRC | 14:42 | |
| *** edcragg_ has quit IRC | 14:44 | |
| *** ssam2 has joined #baserock | 14:52 | |
| *** ChanServ sets mode: +v ssam2 | 14:52 | |
| *** fay_ has joined #baserock | 14:52 | |
| *** tiagogomes_ has joined #baserock | 14:52 | |
| *** bashrc has joined #baserock | 14:52 | |
| *** nowster has joined #baserock | 14:52 | |
| *** flatmush has joined #baserock | 14:54 | |
| paulsherwood | ratmice___: i'm unable to comment on that, maybe others understand this better | 14:54 |
| *** edcragg has joined #baserock | 14:56 | |
| *** gary_perkins has joined #baserock | 14:58 | |
| *** gary_perkins has joined #baserock | 14:58 | |
| *** paulwaters_ has quit IRC | 14:59 | |
| *** mariaderidder has joined #baserock | 15:06 | |
| *** straycat has joined #baserock | 15:18 | |
| straycat | Hi, I've created http://git.baserock.org/cgi-bin/cgit.cgi/delta/cpan/List-MoreUtils-tarball.git/commit/?h=baserock/richardipsum/no-dotgit-assumption I'd like to push a ref at baserock/master to point to this. I'll hopefully be sending some definitions soon which will add List-MoreUtils amongst others to a perl-common stratum. | 15:19 |
| richard_maw | cool | 15:21 |
| *** toscalix has quit IRC | 15:24 | |
| straycat | richard_maw, is that a +1? :) i'll wait around for another +1 in any case before pushing anything | 15:29 |
| richard_maw | I'm not sure I parsed a request in there | 15:29 |
| * richard_maw re-parses | 15:29 | |
| wdutch | richard_maw: I'm having trouble with the other irc network | 15:30 |
| wdutch | let me reboot orchestration again, more patches | 15:30 |
| straycat | sorry yes, I'm asking for permission to push a baserock/master ref to that :) | 15:30 |
| richard_maw | straycat: yep, I see what you're doing now. Certainly, count it as a +1 | 15:31 |
| richard_maw | straycat: call it a +2 if you don't get any objections for an hour or so | 15:32 |
| ssam2 | is it possible to fix this upstream? | 15:32 |
| ssam2 | it's awkward carrying patches for things | 15:32 |
| ssam2 | +1 for this as a temporary solution, but would be nice if we could avoid actually forking it | 15:32 |
| richard_maw | the alternative would be to put something in pre-configure-commands to either remove those lines, or rm the .git | 15:32 |
| richard_maw | or rename it | 15:33 |
| * straycat nods | 15:35 | |
| straycat | I can try and find the maintainer and ask them about it | 15:35 |
| ssam2 | that'd be nice | 15:35 |
| richard_maw | My talk submission for systemd.conf has been accepted | 15:50 |
| pedroalvarez | richard_maw: is it baserock related? (to include it here http://wiki.baserock.org/conferences/) | 15:52 |
| straycat | richard_maw, cool :) | 15:52 |
| Kinnison | richard_maw: congratulations | 15:53 |
| paulsherwood | richard_maw: w00t! | 16:00 |
| richard_maw | pedroalvarez: in that we build systems that include systemd and do atomic updates, yes, and if we end up with a consensus that it's something systemd should do, I'd like to add support to Baserock | 16:02 |
| pedroalvarez | nice! | 16:05 |
| *** mariaderidder has quit IRC | 16:58 | |
| *** bashrc has quit IRC | 17:02 | |
| *** Walkerdine has joined #baserock | 17:02 | |
| *** ssam2 has quit IRC | 17:28 | |
| *** Walkerdine has quit IRC | 17:38 | |
| *** Walkerdine has joined #baserock | 17:55 | |
| *** zoli__ has joined #baserock | 18:05 | |
| *** franred has quit IRC | 18:09 | |
| *** franred has joined #baserock | 18:16 | |
| *** Walkerdine has quit IRC | 18:34 | |
| *** zoli__ has quit IRC | 18:37 | |
| *** franred has quit IRC | 18:48 | |
| *** zoli__ has joined #baserock | 22:18 | |
| *** zoli__ has quit IRC | 22:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!