| *** edcragg has quit IRC | 01:52 | |
| *** flatmush has quit IRC | 01:52 | |
| *** fay_ has quit IRC | 01:52 | |
| *** nowster has quit IRC | 01:52 | |
| *** petefoth has joined #baserock | 01:52 | |
| *** edcragg has joined #baserock | 01:52 | |
| *** nowster has joined #baserock | 01:53 | |
| *** gary_perkins has quit IRC | 01:53 | |
| *** fay_ has joined #baserock | 01:53 | |
| *** flatmush has joined #baserock | 01:54 | |
| *** gary_perkins has joined #baserock | 01:54 | |
| *** zoli__ has joined #baserock | 05:48 | |
| *** paulw has joined #baserock | 06:10 | |
| petefoth | Morning all | 06:26 |
|---|---|---|
| *** paulw has quit IRC | 06:52 | |
| *** paulw has joined #baserock | 06:54 | |
| *** zoli__ has quit IRC | 07:39 | |
| *** toscalix__ has joined #baserock | 07:42 | |
| paulsherwood | morning petefoth | 07:54 |
| paulsherwood | Kinnison, pdar - fwiw i've found in testing that 4 instances of ybd on aws c4.8xlarge is as fast as or slightly faster than 5 on the m4.10xlarge | 08:01 |
| paulsherwood | https://raw.githubusercontent.com/devcurmudgeon/build-logs/master/aws-c4.8x-large-4x14.log | 08:01 |
| paulsherwood | https://raw.githubusercontent.com/devcurmudgeon/build-logs/master/aws.ci-again.log | 08:01 |
| paulsherwood | and the c4 is 30% cheaper: $1.763 vs $2.52 | 08:03 |
| *** bashrc_ has joined #baserock | 08:03 | |
| Kinnison | paulsherwood: interesting, thank you | 08:06 |
| *** paulw has quit IRC | 08:13 | |
| *** paulw has joined #baserock | 08:14 | |
| *** persia__ is now known as persia | 08:21 | |
| *** persia has joined #baserock | 08:21 | |
| *** tiagogomes_ has joined #baserock | 08:21 | |
| *** paulw has quit IRC | 08:22 | |
| *** paulw has joined #baserock | 08:25 | |
| *** mariaderidder has joined #baserock | 08:26 | |
| *** zoli__ has joined #baserock | 08:27 | |
| *** ssam2 has joined #baserock | 08:44 | |
| *** ChanServ sets mode: +v ssam2 | 08:44 | |
| paulsherwood | also ybd master now should clear tmp on start | 09:21 |
| paulsherwood | (reliably0 | 09:21 |
| Kinnison | Useful to know | 09:25 |
| *** mariaderidder has quit IRC | 10:01 | |
| *** mariaderidder has joined #baserock | 10:16 | |
| tiagogomes_ | meh `./check --full` is failing on master | 10:43 |
| Kinnison | Erk | 10:55 |
| tiagogomes_ | https://gerrit.baserock.org/#/c/1105/ | 10:57 |
| *** Guest73814 has quit IRC | 11:09 | |
| *** persia_ has joined #baserock | 11:10 | |
| *** franred has joined #baserock | 11:46 | |
| *** zoli__ has quit IRC | 11:52 | |
| *** zoli__ has joined #baserock | 12:08 | |
| *** paulw has quit IRC | 12:16 | |
| *** paulw has joined #baserock | 12:25 | |
| *** zoli__ has quit IRC | 12:26 | |
| *** jjardon has quit IRC | 12:53 | |
| *** jjardon has joined #baserock | 12:54 | |
| *** franred has quit IRC | 13:16 | |
| *** persia_ has quit IRC | 13:53 | |
| *** persia_ has joined #baserock | 13:54 | |
| *** paulw has quit IRC | 14:02 | |
| *** paulw has joined #baserock | 14:03 | |
| *** zoli__ has joined #baserock | 14:06 | |
| *** zoli__ has quit IRC | 14:21 | |
| *** paulw has quit IRC | 14:36 | |
| *** flatmush has quit IRC | 15:45 | |
| *** flatmush has joined #baserock | 15:46 | |
| *** bashrc_ has quit IRC | 15:50 | |
| *** bashrc_ has joined #baserock | 15:51 | |
| *** flatmush has quit IRC | 15:54 | |
| *** flatmush has joined #baserock | 15:55 | |
| *** bashrc_ has quit IRC | 15:55 | |
| *** fay_ has quit IRC | 15:57 | |
| *** petefoth has quit IRC | 15:57 | |
| *** ssam2 has quit IRC | 15:57 | |
| *** mariaderidder_ has joined #baserock | 15:58 | |
| *** edcragg has quit IRC | 15:58 | |
| *** flatmush has quit IRC | 15:58 | |
| *** tiagogomes_ has quit IRC | 15:58 | |
| *** nowster has quit IRC | 15:58 | |
| *** edcragg has joined #baserock | 15:58 | |
| *** petefoth has joined #baserock | 15:58 | |
| *** ssam2 has joined #baserock | 15:58 | |
| *** ChanServ sets mode: +v ssam2 | 15:58 | |
| *** fay_ has joined #baserock | 15:58 | |
| *** gary_perkins has quit IRC | 15:58 | |
| *** nowster has joined #baserock | 15:58 | |
| *** tiagogomes_ has joined #baserock | 15:58 | |
| *** flatmush has joined #baserock | 15:58 | |
| *** mariaderidder has quit IRC | 15:59 | |
| *** gary_perkins has joined #baserock | 15:59 | |
| *** bashrc_ has joined #baserock | 16:01 | |
| paulsherwood | ssam2: thanks for the packaging of ybd! i'm certainly not against it. | 16:23 |
| ssam2 | ok, cool | 16:23 |
| paulsherwood | could/should it include dependencies? | 16:23 |
| ssam2 | this is https://github.com/devcurmudgeon/ybd/pull/123 | 16:24 |
| ssam2 | paulsherwood: the setup.py file? hmm | 16:24 |
| ssam2 | i thought ybd didn't have any :-) | 16:24 |
| ssam2 | i guess there's sandboxlib | 16:24 |
| paulsherwood | http://wiki.baserock.org/ybd/ shows pyyaml sandboxlib jsonschema requests | 16:24 |
| paulsherwood | jsonschema is optional | 16:24 |
| paulsherwood | requests is required for kbas | 16:24 |
| ssam2 | oh yeah | 16:24 |
| ssam2 | we could add requirements.txt | 16:25 |
| paulsherwood | is that how deps are specified for pythong packages? | 16:25 |
| ssam2 | for the required dependencies. then the optional dependencies go in setup.cfg | 16:26 |
| ssam2 | python packaging is a wilderness. but that's how PBR recommends doing it | 16:26 |
| paulsherwood | ssam2: i'd be tempted to ignore the optional for now | 16:26 |
| ssam2 | http://docs.openstack.org/developer/pbr/#requirements | 16:26 |
| ssam2 | so it's just sandboxlib. easy enough to add | 16:26 |
| ssam2 | possibly only useful if you plan to release YBD on PyPI | 16:26 |
| paulsherwood | the jsonschema stuff depends on us implementing schemas for definitions | 16:26 |
| paulsherwood | pyyaml seems to be missing on some python default implementations, same with requests? | 16:27 |
| paulsherwood | (they're both in baserock by default, i know) | 16:27 |
| ssam2 | actually yes, neither of those are built-in to Python, so we should list them | 16:27 |
| paulsherwood | i assume this is only for python packages...? | 16:29 |
| paulsherwood | how would one automagically install git, autotools, wget etc? | 16:29 |
| radiofree | i hate apps that try and call apt or yum | 16:30 |
| * paulsherwood too | 16:30 | |
| ssam2 | using Baserock, perhaps | 16:30 |
| paulsherwood | :) | 16:30 |
| radiofree | you can't use baserock-chroot-manager on baserock | 16:31 |
| pedroalvarez | If the app fails suggesting what to do to solve it, would be enough | 16:31 |
| paulsherwood | ack | 16:31 |
| ssam2 | radiofree: that's annoying! | 16:34 |
| ssam2 | paulsherwood: i updated the pull request with requirements.txt file | 16:36 |
| paulsherwood | ssam2: what's the process of pushing this to pypi? | 16:39 |
| * paulsherwood wonders if he should do a 'release' prior to merging this, for safety | 16:39 | |
| ssam2 | paulsherwood: see https://github.com/CodethinkLabs/sandboxlib/blob/master/HACKING.rst | 16:40 |
| ssam2 | that said, it seems like PBR isn't picking up the version number from the tags in YBD.git | 16:41 |
| ssam2 | oh, that might be because it only looks at 'annotated' tags | 16:41 |
| ssam2 | seems that a 15-38 tag causes PBR to call the package 'ybd-15.post38', though | 16:42 |
| ssam2 | would you consider switching to 15.38 instead of 15-38 for numbers? | 16:42 |
| ssam2 | otherwise, you could manually set a version number in setup.cfg, but it seems like that would be more faff in the long run | 16:43 |
| *** mariaderidder_ has quit IRC | 16:44 | |
| paulsherwood | yup, happy to switch | 16:45 |
| paulsherwood | please hold | 16:45 |
| paulsherwood | ssam2: https://github.com/devcurmudgeon/ybd/releases/tag/13.38 | 16:48 |
| ssam2 | 13.38? I don't understand this numbering scheme! | 16:51 |
| paulsherwood | ffs! | 16:51 |
| paulsherwood | should be 15.38 | 16:51 |
| pedroalvarez | :) | 16:51 |
| * paulsherwood hopes no-one will mind if i delete it | 16:52 | |
| ssam2 | there must be a 5 minute rule or something. | 16:52 |
| paulsherwood | done | 16:52 |
| paulsherwood | https://github.com/devcurmudgeon/ybd/releases/tag/15.38 | 16:53 |
| paulsherwood | yy.ww | 16:53 |
| richard_maw | paulsherwood: did you know that when you fork off a subprocess, that subprocess also holds a reference to your lock, hence why when your original process closes the file descriptor it locked, when it exits the `with open` scope, the process doing the cleanup still has the shared lock, hence your cleanup is protected from another simultaneous cleanup attempt | 16:53 |
| paulsherwood | richard_maw: i think i did? but since you're asking, maybe i'm missing something? | 16:54 |
| paulsherwood | richard_maw: iiuc my merged version includes your recommendation, and should be safe as you describe above | 16:55 |
| richard_maw | I know, I had a look at the version you eventually merged. | 16:55 |
| paulsherwood | i hope you are happy with it... did i miss something? | 16:56 |
| richard_maw | I would have done it by returning the shared-locked file descriptor, but I'm trying to work out whether you accidentally or purposefully produced a more interesting solution. | 16:56 |
| paulsherwood | lol | 16:56 |
| paulsherwood | accident probably | 16:56 |
| richard_maw | ☺ | 16:56 |
| paulsherwood | i was aware that the shared lock would be held by both processes while in the cleanup function, though :) | 16:57 |
| *** bashrc_ has quit IRC | 16:57 | |
| richard_maw | you end up closing the reference to the lock in your main process after you exit the cleanup function, since your reference to the lock is kept in the file which you close | 16:57 |
| richard_maw | which I suspected may be accidental, but since you're keeping the reference in the child process, it still works | 16:58 |
| richard_maw | there's 1 race left I can see, the failure mode for which is just that you need to retry starting a build, but it can happen when you go a long time without cleaning up tempdirs, and a process is holding the exclusive lock and calling os.listdir while another is started | 17:01 |
| richard_maw | the second process could try to take the exclusive lock, fail, and then also fail to take the shared lock, since the other process is still trying to os.listdir | 17:01 |
| paulsherwood | ack | 17:02 |
| richard_maw | which is why in my e-mail I recommended a blocking lock with a timeout | 17:02 |
| paulsherwood | ok i'll look at that | 17:02 |
| paulsherwood | thanks for the continued feedback :) | 17:03 |
| * paulsherwood has to dash | 17:03 | |
| * richard_maw does too | 17:03 | |
| richard_maw | a blocking lock will do most of the time, as would a retry loop, but the former is bad if it's gotten totally stuck in os.listdir (I've seen this happen in btrfs) and the latter can end up missing the window where it's allowed to take the lock (unlikely) and is slower than blocking when the other process behaves | 17:06 |
| richard_maw | unfortunately the only way to do a blocking lock with a timeout involves signals, and despite the process overhead involved in doing it by shelling out to http://man7.org/linux/man-pages/man1/flock.1.html , I still prefer it, as it doesn't mess with your own process' state so badly | 17:08 |
| *** toscalix__ has quit IRC | 17:12 | |
| *** toscalix__ has joined #baserock | 17:13 | |
| *** ssam2 has quit IRC | 17:20 | |
| paulsherwood | http://stackoverflow.com/questions/5255220/fcntl-flock-how-to-implement-a-timeout ? | 19:35 |
| *** toscalix__ has quit IRC | 19:37 | |
| *** zoli__ has joined #baserock | 19:40 | |
| *** zoli___ has joined #baserock | 19:48 | |
| *** zoli__ has quit IRC | 19:48 | |
| *** zoli__ has joined #baserock | 20:13 | |
| *** zoli___ has quit IRC | 20:13 | |
| *** zoli__ has quit IRC | 20:46 | |
| *** zoli__ has joined #baserock | 20:47 | |
| paulsherwood | radiofree: incidentally, with swap, on scaleway qtwebkit built ok in 02:53:49 | 21:07 |
| radiofree | paulsherwood: it builds, with swap, on a jets | 21:08 |
| radiofree | On.. In about an hour | 21:08 |
| radiofree | On the mustang it can probably build without swap in less time than thst | 21:08 |
| persia | How much RAM is on a mustang? | 21:09 |
| radiofree | But apparently no one here wants a fast machine populating arm cache | 21:09 |
| radiofree | persia: 16G | 21:09 |
| persia | I'd like one :) | 21:09 |
| paulsherwood | radiofree: i'd like one too | 21:10 |
| paulsherwood | i guess we need mustangs in the cloud... queue https://www.youtube.com/watch?v=qMM3MgS4yxc | 21:12 |
| * paulsherwood doesn't know why he suddenly thought of the osmonds screaming | 21:16 | |
| radiofree | paulsherwood: you already one | 21:17 |
| jjardon | Hi, Is there a generic armv7 image available somewhere? | 21:22 |
| paulsherwood | yes | 21:25 |
| *** zoli__ has quit IRC | 21:25 | |
| * paulsherwood hunts | 21:25 | |
| *** zoli__ has joined #baserock | 21:25 | |
| paulsherwood | jjardon: what's your use-case? | 21:25 |
| paulsherwood | http://212.47.232.3/artifacts/genivi* | 21:31 |
| paulsherwood | http://212.47.232.3/artifacts/build-system* | 21:32 |
| jjardon | Id like to try to add support for a new board | 21:32 |
| jjardon | Cheers | 21:32 |
| paulsherwood | jjardon: note these are ybd images | 21:33 |
| paulsherwood | jjardon: wget http://212.47.232.3/get/build-system-armv7lhf-jetson.cee219f7d667c5d65d86b57c92878ae43453cc173f824918d3a8b97182b8b189 for example | 21:34 |
| * paulsherwood has not tested them, also | 21:34 | |
| paulsherwood | actually, i may be misleading you here.. these are system artifacts created by build, not deployment | 21:35 |
| jjardon | paulsherwood: I think that's good enough, thanks. By any chance you dont have a Build system chroot , isnt it? | 21:43 |
| paulsherwood | jjardon: fraid not, i'm not using chroots at all | 21:44 |
| jjardon | paulsherwood: that's ok. btw, any opinion about https://gerrit.baserock.org/#/c/1098/ ? it would be good to have a agreement about that kind of updates | 21:48 |
| paulsherwood | jjardon: i would be delighted to see it, just not sure how to address richard ipsum's valid point | 21:50 |
| paulsherwood | i tihnk by default we should adopt each new major kernel release as soon as it comes out | 21:52 |
| paulsherwood | in other news... https://docs.google.com/spreadsheets/d/149jS3VVJ7jA14dSJybeOr4otlEFrzCVJjbxMSppiMcg/pubchart?oid=1593295908&format=interactive | 21:56 |
| paulsherwood | so i think our assumption (in morph and ybd) that max-jobs = 1.5*cores is not actually the best number for AWS at least | 21:57 |
| paulsherwood | increasing max-jobs past 20 actually SLOWS DOWN the build! | 21:57 |
| paulsherwood | (cxlarge has 36 cores, mxlarge has 40 cores) | 21:57 |
| paulsherwood | and cx machine is 10% faster than mx, as well as being 30% cheaper per hour | 21:58 |
| paulsherwood | s/faster/faster for builds/ | 21:58 |
| paulsherwood | to be fair, other components may vary a bit - qtwebkit did get slightly faster for -j > 20 | 22:05 |
| * radiofree looks at backscroll... saves it for tomorrow | 22:05 | |
| * paulsherwood should write a blog post | 22:07 | |
| *** zoli__ has quit IRC | 22:09 | |
| *** zoli___ has joined #baserock | 22:10 | |
| jjardon | paulsherwood: yeah; as I said in the comments I did the opposite before (update all the BSPs, warning is not tested), but others didnt like it, with valid points as well. I only want to merge my patch :) | 22:11 |
| paulsherwood | jjardon: i would prefer the opposite, tbh | 22:11 |
| paulsherwood | ie keep things in step, even if we don't test all the things | 22:12 |
| jjardon | paulsherwood: Id prefer keep updating everything, but only merge when someone test it; if time passes and the changes are not merged, remove the BSP as it has not being maintained (or move it to a "unsupported" directory or something) | 22:14 |
| paulsherwood | jjardon: did you manage to get that artifact? i'm seeing some errors on the server side, timeout for example | 22:15 |
| paulsherwood | jjardon: makes sense | 22:15 |
| jjardon | nope, didn't receive the board yet, only asking to prepare when the moment arrive :) | 22:16 |
| paulsherwood | ok | 22:16 |
| paulsherwood | well, someone seems to have downloaded it, after a couple of tries :) | 22:17 |
| paulsherwood | and now the world and their friends are probing it | 22:18 |
| paulsherwood | 104.148.44.191 - - [14/Sep/2015 18:16:09] "GET http://www.sogou.com/index.html HTTP/1.1" 404 782 | 22:18 |
| paulsherwood | 104.148.44.191 - - [14/Sep/2015 18:16:10] "CONNECT www.paypal.com:443 HTTP/1.1" 404 761 | 22:18 |
| bjdooks | paulsherwood: the sad fact of public facing services | 22:25 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!