| jjardon | gtristan: that looks very promising :) | 00:01 |
|---|---|---|
| *** zoli_ has joined #baserock | 00:27 | |
| *** zoli_ has quit IRC | 00:32 | |
| gtristan | jjardon, :) | 01:43 |
| * gtristan back in 15min | 01:43 | |
| *** gtristan has quit IRC | 01:43 | |
| *** gtristan has joined #baserock | 01:58 | |
| *** zoli_ has joined #baserock | 02:16 | |
| *** zoli_ has quit IRC | 02:20 | |
| gtristan | eek | 02:43 |
| gtristan | http://git.baserock.org/cgi-bin/cgit.cgi/delta/xorg-lib-libxkbfile.git/ <-- 500 internal server error ? | 02:43 |
| *** zoli_ has joined #baserock | 04:05 | |
| *** zoli_ has quit IRC | 04:09 | |
| *** zoli_ has joined #baserock | 05:21 | |
| paulsherwood | erk... g.b.o is completely down | 05:24 |
| *** zoli_ has quit IRC | 05:25 | |
| paulsherwood | or rather, the cgit is down... gitano is still functioning | 05:25 |
| gtristan | no worry, I got through with http://cgit.freedesktop.org/xorg/ | 05:29 |
| gtristan | there must be a better/automated way... but I crawled through those upstreams picking off the commit IDs for the morphs | 05:29 |
| *** paulwaters_ has joined #baserock | 06:31 | |
| gtristan | mkay... looks like we're almost there... just another [1/141/347] to [re]build, due to systemd.morph missing out on --enable-compat-libs, now required by new xserver | 06:48 |
| gtristan | and I thought I almost had a build | 06:48 |
| * gtristan snap | 06:48 | |
| *** ssam2 has joined #baserock | 07:02 | |
| *** ChanServ sets mode: +v ssam2 | 07:02 | |
| ssam2 | git.baserock.org's cgit page was giving a 500 error, i've restarted lighttpd-git.service | 07:08 |
| ssam2 | which has fixed the issue. no idea what the problem was | 07:08 |
| gtristan | we noticed :) | 07:09 |
| *** zoli_ has joined #baserock | 07:09 | |
| pedroalvarez | ssam2: that explains why there were 2 restarts of the service | 07:13 |
| pedroalvarez | :) | 07:13 |
| ssam2 | we really need to upgrade that thing! | 07:14 |
| *** zoli_ has quit IRC | 07:14 | |
| pedroalvarez | I restarted it too and tried to figure out what happened | 07:14 |
| ssam2 | cgit doesn't seem to log anything anywhere | 07:14 |
| pedroalvarez | Cgit? | 07:14 |
| ssam2 | well, g.b.o in general | 07:14 |
| ssam2 | i was thinking we should add trove to the ci.morph cluster in order to remove some of the friction in upgrading it | 07:15 |
| ssam2 | it doesn't take very long to build the trove bits when the rest of the system is in the cache, but still .. | 07:15 |
| pedroalvarez | ssam2: I have to say that the friction now is also none | 07:15 |
| pedroalvarez | I've upgraded it recently | 07:16 |
| ssam2 | ahhh, nice | 07:16 |
| pedroalvarez | 4th of September last time | 07:16 |
| ssam2 | we should keep a log somewhere ... | 07:16 |
| pedroalvarez | S/also/almost/ | 07:16 |
| ssam2 | i thought maybe the git log of infrastructure.git would be enough to know, but it's not really | 07:16 |
| pedroalvarez | Hmm, true.. I upgraded it from master of definitions.git | 07:17 |
| pedroalvarez | I shall do that next time, and put a | 07:18 |
| pedroalvarez | Note in the commit message | 07:18 |
| *** gtristan has quit IRC | 07:29 | |
| *** gtristan has joined #baserock | 07:29 | |
| jjardon | gtristan: I think I've sent a patch to the xserver so if you upgrade it you should not need enable compat libs in systemd | 07:36 |
| gtristan | jjardon, umm, I believe I have master | 07:36 |
| gtristan | or maybe not, most of xlibs I have master | 07:37 |
| gtristan | in anycase, it's building already | 07:37 |
| gtristan | [12/91/347] now, and both xserver & systemd successfully out of the way | 07:37 |
| * gtristan has ~20 commits he'd like to submit once this thing builds... | 07:38 | |
| jjardon | http://cgit.freedesktop.org/xorg/xserver/commit/?id=f8a2c087c89ed5b638a2e62987f255d3f04f914e | 07:38 |
| gtristan | jjardon, I have exactly http://cgit.freedesktop.org/xorg/xserver/commit/?id=880d4e78b1823bcc0e66e68270af1a475f097bb5 | 07:40 |
| gtristan | jjardon, looking here http://cgit.freedesktop.org/xorg/xserver/tree/configure.ac#n828, it looks like it should have passed | 07:43 |
| gtristan | but somehow it did not, in anycase, this time it does find libsystemd-daemon | 07:43 |
| jjardon | gtristan: btw, I'd be interested in you r thoughs about using baserock as a development tool, and, if you have, ideas to make the workflow better. I personally think is a bit painful atm | 07:43 |
| * gtristan did not touch systemd version but left at v225... dont want to screw with that and have to rebuild the kernel | 07:43 | |
| gtristan | its incredibly painful | 07:44 |
| jjardon | gtristan: :) | 07:44 |
| gtristan | jjardon, 1.) ybd should ideally be able to grok the lorries I think, or there should be some story for adding dependencies locally and testing the build *without* needing to patch upstream baserock directly... 2.) It would be great to have a 'buildone' command to only build a single artifact and update it in the target image directly | 07:45 |
| gtristan | the alternative of checking out sources and building, make, make install directly in a VM is enough in some cases | 07:46 |
| gtristan | but when it becomes interesting is embedded, your target most probably does not have build tools installed, and it probably doesnt have SSD and quad i7s | 07:46 |
| gtristan | would be great to reduce the build cycle to deploy and test on a real target hardware | 07:47 |
| gtristan | s/reduce/reduce time needed | 07:47 |
| * gtristan is sure to accumulate more ideas hehe... | 07:48 | |
| ssam2 | it's possible to do that already with Morph, but it's a bit of a faff and not really used | 07:48 |
| ssam2 | building just one component can be done by adding a new stratum on top of all the others that contains that component | 07:48 |
| gtristan | I know that the whole thing is build-server oriented, but it seems to me it would be awesome if the whole thing could be a 'checkout' with local state | 07:48 |
| gtristan | i.e. to view the entire build system, repos, etc, as if it were a single 'git' | 07:49 |
| ssam2 | yeah, that would be cool | 07:49 |
| ssam2 | again, the bits are there to do that in Morph, would just be 20 lines to write a 'checkout everything for this build' plugin | 07:50 |
| gtristan | ssam2, and then I'd like to see with one command... what are all my deltas ahead of upstream ? | 07:50 |
| gtristan | things like that | 07:50 |
| gtristan | 'what bugs have I closed locally but are still open upstream, which of my patchsets address this' | 07:51 |
| ssam2 | gtristan: yeah, that would be awesome | 07:51 |
| ssam2 | `morph diff` command has the basis of that already... it shows you which component 'ref' fields have changed between two versions of definitions | 07:52 |
| * gtristan has been dreaming of distributed (really integrated with VCS) bug tracking for years | 07:52 | |
| ssam2 | could be extended to show the actual diff between those refs | 07:52 |
| jjardon | gtristan: agree, can you open a new story (I'm in my phone ATM, sorry) with "reduce the build cycle to deploy" so we can put our thoughs there? | 07:52 |
| jjardon | For the other thing too :) | 07:52 |
| gtristan | jjardon, sure... it's always good to have *something* to do with 61 builds remaining before a test (wink) | 07:57 |
| *** gary_perkins has joined #baserock | 08:05 | |
| *** bashrc_ has joined #baserock | 08:09 | |
| *** toscalix__ has joined #baserock | 08:09 | |
| *** toscalix has quit IRC | 08:09 | |
| *** 20WAB8TAT has joined #baserock | 08:13 | |
| Kinnison | gtristan: I believe Lars Wirzenius (liw) has been experimenting with distributed bug/issue tracking | 08:14 |
| gtristan | Kinnison, I've seen a couple projects which do bug tracking and use distributed version control as backends, but I think that misses the point | 08:15 |
| Kinnison | gtristan: He's still very early in his experiments, but http://liw.fi/distix/ is what he was playing with | 08:15 |
| Kinnison | gtristan: And I'm sure he'd welcome input on "missing the point" since he's kinda paused distix until he has spare brain again | 08:15 |
| gtristan | Kinnison, I would love to visit a sort of cgit page... where I can view a bug, and see which branches it's still open on - what stable branches it's been backported to, etc | 08:15 |
| gtristan | and I think it needs some though as to, how maintainers review branches to merge, instead of just patch sets, one should have a connection between the bugs which the branch "claims to close" and which patches are attached to that closing of a bug | 08:16 |
| gtristan | Kinnison, nice to hear someone is working on that :) | 08:17 |
| * gtristan looks at url | 08:17 | |
| Kinnison | gtristan: aah, I'd love to do that from my terminal, rather than a browser :-) | 08:17 |
| gtristan | (or even *thinking* of working on that) | 08:17 |
| gtristan | yes, even in a terminal, of course as a distributed thing, it would all be a part of the state of the repository | 08:18 |
| gtristan | the main issue I think is collecting bug reports from the wild, and how that would integrate | 08:18 |
| gtristan | Kinnison, ideally, it could also remove the need to A.) always show your commit IDs in the bug report when closing a bug, and B.) Always mention the bug reference in the commit message | 08:19 |
| gtristan | which... anyway would bring everything closer together and reduce error | 08:20 |
| Kinnison | It'd require some interesting variant tracking though | 08:21 |
| Kinnison | since different commits in different lines may eliminate the same bug in different ways | 08:21 |
| Kinnison | also tracking how a bug was introduced can be hard | 08:21 |
| Kinnison | rjek gave some thought to this for tracking security issues, he might have input | 08:21 |
| rjek | I hate all bug trackers and the people who garden them, basically. | 08:21 |
| gtristan | haha | 08:23 |
| gtristan | jjardon, filed https://storyboard.baserock.org/#!/story/59 ... | 08:25 |
| gtristan | jjardon, and https://storyboard.baserock.org/#!/story/60 | 08:28 |
| * gtristan wonders if he gets email notifications when people comment | 08:29 | |
| * gtristan wouldnt mind receiving notifications for addition of any-and-all bugs, just to help get a feel for what's going on | 08:29 | |
| gtristan | yay... less than 30 to build :) | 08:31 |
| Kinnison | gtristan: happy hangul birthday btw | 08:31 |
| gtristan | yeah that's today.. everyone is off work around here :D | 08:31 |
| gtristan | hehe, there is a beer festival in Gangnam I mean to go meet people at | 08:31 |
| gtristan | after this build completes and I manage to upload some patches ! | 08:32 |
| gtristan | hah | 08:32 |
| ssam2 | gtristan: email notifications isn't implemented in storyboard at the moment. you can bug zara and sotk to do it, though :-) | 08:33 |
| Zara | gtristan: the email notifications thing is on my todo list for storyboard master. Baserock currently runs an old version so Baserock would need to update it first, to get the changes from master. And since storyboard upstream currently means me and SotK, it could take a while. | 08:34 |
| Zara | we hang out on #storyboard on freenode, and I'm hoping to do a storyboard session at the OpenStack summit, so hopefully we can attract more developers there. | 08:35 |
| *** mariaderidder has joined #baserock | 08:36 | |
| gtristan | Zara, I find myself adding [placeholder] tasks on baserock's storyboard btw | 08:36 |
| Zara | I'm also going to look into reviewing on other openstack projects. the issue is with getting to know the codebases well enough to review usefully, though the same could be said in reverse. | 08:36 |
| gtristan | as often; when reporting something, it's not yet clear what needs to be done to address the issue | 08:36 |
| * pedroalvarez remembers he never downloaded the storyboard snapshot | 08:36 | |
| pedroalvarez | glance image-download is awesome: | 08:37 |
| pedroalvarez | --file <FILE> Local file to save downloaded image data to. If this is not | 08:37 |
| pedroalvarez | specified the image data will be written to stdout. | 08:37 |
| pedroalvarez | why on earth would I want a diskimage in stdout!!? | 08:37 |
| Kinnison | mmm pipey | 08:37 |
| gtristan | to stream it into gzip or such | 08:37 |
| gtristan | right | 08:38 |
| pedroalvarez | but the default..? | 08:38 |
| 20WAB8TAT | pedroalvarez, `glance image-download --file image | gzip > myimage.csv.gz` | 08:38 |
| 20WAB8TAT | 20WAB8TAT wonders who 20WAB8TAT is | 08:38 |
| pedroalvarez | thanks 20WAB8TAT | 08:38 |
| *** 20WAB8TAT is now known as tiagogomes | 08:38 | |
| * tiagogomes is annoyed that that last glance client seems to not be able to create an image from an URL anymore | 08:40 | |
| Zara | gtristan: btw, if you have issues with storyboard or the webclient itself, it's best leaving stories here: https://storyboard.openstack.org/#!/project/457 or here: https://storyboard.openstack.org/#!/project/456 so we have a more permanent record. or better yet seeing if you can help us hack around; a lot of the time we know about the bugs, we just have a lot to work on and there's only so much we can do at once! The people who wrote mos | 08:40 |
| Zara | webclient gerrit is here, if anyone would like to help on the reviewing side: https://review.openstack.org/#/q/status:open+project:openstack-infra/storyboard-webclient,n,z | 08:41 |
| pedroalvarez | Zara: "The people who wrote mos...." ? | 08:41 |
| Zara | t of the code originally quit in fury some months ago over a lack | 08:41 |
| Zara | of resources, so it's also hard for us to get answers about the current implementation except by looking at the code. | 08:41 |
| Zara | there you go :P | 08:41 |
| Zara | I just want to explain why it takes a while, so you know it's not a case of us not listening! | 08:42 |
| Zara | storyboard gerrit here: https://review.openstack.org/#/q/status:open+project:openstack-infra/storyboard,n,z | 08:42 |
| gtristan | ok, so: echo "Bug report" | Zara --file-report... is not a viable option... I understand :D | 08:42 |
| gtristan | Zara, do you know if it's still an issue upstream ? | 08:43 |
| gtristan | I mean... ability to file a bug without adding any task ? | 08:43 |
| Zara | :) I have a lot of backscroll and things to focus on, so I can note that you care about it but it might get lost. | 08:43 |
| Zara | I don't think it is, no. | 08:43 |
| gtristan | ok, baserock seems to force a task, that's what I was saying... | 08:44 |
| Zara | oh, sorry, I think it is an issue, misread | 08:44 |
| gtristan | if you like... I could file it | 08:44 |
| gtristan | ok | 08:44 |
| * gtristan guesses he'll find out while trying to file one on openstack ;-) | 08:44 | |
| Zara | Yeah, I think it's not implemented yet, sorry. I don't think I've tried to do that myself! :) | 08:45 |
| Zara | if you get an irritating 401, that's what I'm currently looking into, just so you know, haha | 08:45 |
| *** jonathanmaw has joined #baserock | 08:46 | |
| Zara | (the session in japan is going to be effectively 'we have loads of bugs, we have very few people; what should we prioritise?' I was going to ask for stuff on this on the infra mailing list, but if baserock also have priorities, let me know.) | 08:47 |
| Zara | meanwhile, I'll try to keep the task placeholders in mind. | 08:47 |
| gtristan | Zara, filed as: https://storyboard.openstack.org/#!/story/2000375 | 08:48 |
| gtristan | [74/91/347]... and trucking... chug chug, huff puff... | 08:49 |
| Zara | great, thanks. :) please just understand that SotK updating baserock's storyboard will mean development on storyboard master will slow down, though we're doing our best! :) | 08:54 |
| paulsherwood | richard_maw: you were right! on moonshot 2 instances max-jobs 8 beats one... | 08:54 |
| paulsherwood | but amazingly 3 beats 2, and 4 just about beats 3 !!! | 08:54 |
| SotK | pedroalvarez: did you manage to obtain that snapshot last night? :) | 08:55 |
| pedroalvarez | SotK: nope, sorry, I had some last minute meetings and forgot about it | 08:56 |
| pedroalvarez | I didn't have enough space in the disk anyway.. :/ | 08:57 |
| SotK | ah, no worries :) | 08:58 |
| pedroalvarez | downloading now ;) | 08:58 |
| SotK | thanks! | 08:58 |
| *** edcragg has joined #baserock | 09:00 | |
| *** toscalix__ has quit IRC | 09:13 | |
| *** fay_ has joined #baserock | 09:21 | |
| *** wschaller has joined #baserock | 09:24 | |
| *** lachlanmackenzie has quit IRC | 09:34 | |
| gtristan | jjardon, unfortunately... new X server is having a rough time starting up | 09:41 |
| * gtristan hangs head in disappointment... after all that time building | 09:41 | |
| *** toscalix__ has joined #baserock | 09:42 | |
| gtristan | I'm sure that probably either a less recent xserver, or some tweaking of the config will solve it, and then gnome-shell wont hang anymore | 09:42 |
| gtristan | but dont have the patience on friday night for another 3 hour build :-/ | 09:42 |
| *** toscalix__ has quit IRC | 09:49 | |
| *** toscalix__ has joined #baserock | 09:49 | |
| * SotK looks at the stories gtristan filed earlier | 09:56 | |
| SotK | gtristan: it should be possible to build without lorrying just by changing the "repo" field for the relevant component in your stratum | 09:57 |
| SotK | any URL which works with `git clone` should work there I believe | 09:57 |
| SotK | Having the repo lorried is only a requirement to merge changes into master of definitions | 09:57 |
| gtristan | that is true, and it's what I was doing in fact | 09:59 |
| * pedroalvarez is playing with a rpi2 | 09:59 | |
| pedroalvarez | I'll try to put baserock on it :) | 09:59 |
| gtristan | I suppose it means; it's impossible to send your patch set to definitions without having a lorry commit accepted, and it's impossible to see what the lorry patch does without having the new definitions either | 10:00 |
| SotK | what do you mean "see what the lorry patch does"? | 10:01 |
| SotK | you can run lorry locally in a devel VM to check the patch works as expected | 10:01 |
| gtristan | well, if you are on the system maintainer side, you receive a lorry patch... want to see what justifies it right ? | 10:01 |
| SotK | ah, I see what you mean | 10:02 |
| SotK | I agree it would be good to come up with a better solution to that than asking "What is this for" in here every time | 10:02 |
| gtristan | SotK, right, and "in here" presumes that nobody else is hosting a baserock environment :) | 10:05 |
| Zara | yeah, that used to be a problem for me, too. I'd also find I had to repeat my reasons a lot when adding lorries. I tended to put a note in gerrit but it wasn't ideal. | 10:05 |
| *** paulwaters__ has joined #baserock | 10:07 | |
| *** paulwaters_ has quit IRC | 10:10 | |
| *** paulwaters_ has joined #baserock | 10:13 | |
| gtristan | jjardon, I'm going to try xserver-1.17-branch anyway... that's 63 builds, I'll let it run over dinner and let you know how it went | 10:15 |
| * gtristan thinks it will pass and then we'll see a gnome-shell finally | 10:15 | |
| gtristan | but let's see, if not, we're missing some additional component, or config | 10:16 |
| * gtristan better get food | 10:16 | |
| *** gtristan has quit IRC | 10:17 | |
| *** paulwaters__ has quit IRC | 10:17 | |
| pedroalvarez | this rpi kernel is taking almost an hour to compile.. | 10:21 |
| pedroalvarez | I think it would be good to disable some things that were only enabled for openstack | 10:24 |
| jonathanmaw | Is there a running kbas server anywhere I can use? I'd prefer not to have to rebuild an armv7lhf from scratch | 10:27 |
| *** gtristan has joined #baserock | 10:43 | |
| *** toscalix__ is now known as toscalix | 10:46 | |
| pedroalvarez | jonathanmaw: nope | 10:46 |
| pedroalvarez | Well, at least is not part of baserock infra | 10:47 |
| Kinnison | kbas is entirely insecure and ybd logs the password (so beware of posting unadulterated logs) | 10:47 |
| pedroalvarez | We do have a morph cache artifact full of armv7 artifacts | 10:47 |
| pedroalvarez | Artifact server I mean | 10:49 |
| *** lachlanmackenzie has joined #baserock | 10:50 | |
| pedroalvarez | I wonder if anyone around has any experience with rpi2 to point me to all the steps they think are needed to put baserock in there | 10:55 |
| pedroalvarez | edcragg: I've heard you did something with rpi1 | 10:57 |
| * pedroalvarez finds a script from kali that creates the kali OS and creates also the image | 11:00 | |
| pedroalvarez | it might be useful to understand what's needed | 11:00 |
| pedroalvarez | https://github.com/offensive-security/kali-arm-build-scripts/blob/master/rpi2.sh | 11:00 |
| *** paulwaters_ has quit IRC | 11:06 | |
| *** paulwaters_ has joined #baserock | 11:10 | |
| *** cosm has joined #baserock | 11:25 | |
| pedroalvarez | hm.. these instructions clone the entire linux-firmware repo in /lib/firmware | 11:26 |
| pedroalvarez | I'm trying to work out what files are needed | 11:26 |
| *** zoli__ has joined #baserock | 11:36 | |
| *** zoli__ has quit IRC | 11:41 | |
| paulsherwood | jonathanmaw: http://212.47.249.244/artifacts/* | 11:48 |
| paulsherwood | now the only thing is to identify which definitions that set was built from | 11:48 |
| * jonathanmaw tries to figure out how to configure ybd to use that server | 11:51 | |
| paulsherwood | jonathanmaw: 18242ab38f8514eb36852b29e2dd676f605ae523 of definitions | 11:52 |
| paulsherwood | jonathanmaw: set kbas-url: http://212.47.249.244/ | 11:53 |
| jonathanmaw | paulsherwood: that's newer than the GDP branch, by the looks of it. | 11:53 |
| paulsherwood | jonathanmaw: you wanted a rootfs, i thought? | 11:53 |
| pedroalvarez | jonathanmaw: it may be newer, but if we want to put it in mainline, we have to rebase the work somehow | 11:54 |
| jonathanmaw | paulsherwood: I'm currently making sure that I can build the GDP in YBD. Once that's done, I can look at merging with master, then appropriate fixes to the GDP. | 11:54 |
| paulsherwood | aha | 11:55 |
| paulsherwood | jonathanmaw: were you seeing a kbas server to push to, rather than pull from? | 11:55 |
| paulsherwood | seeking | 11:55 |
| jonathanmaw | paulsherwood: pull from, initially, in the hopes I can shorten the build time. | 11:56 |
| paulsherwood | ack | 11:56 |
| paulsherwood | can you do the work on x86 first? | 11:58 |
| paulsherwood | alternatively i can kick off build of the gdp on a set of scaleway machines to hit that kbas server | 11:59 |
| jonathanmaw | armv7lhf is currently the most comprehensively-tested, but I can leave that running in the background for now. | 11:59 |
| jonathanmaw | now to see if I can remember my credentials for my x86 builder... | 12:00 |
| paulsherwood | jonathanmaw: if you mail me your public key i can add you to some scaleway machines | 12:00 |
| jonathanmaw | paulsherwood: biff | 12:02 |
| *** brlogger has joined #baserock | 12:42 | |
| pedroalvarez | here we go :) | 12:42 |
| *** toscalix has quit IRC | 13:02 | |
| *** petefoth has quit IRC | 13:14 | |
| *** tiagogomes has quit IRC | 13:14 | |
| *** gary_perkins has quit IRC | 13:14 | |
| *** wschaller has quit IRC | 13:14 | |
| *** edcragg has quit IRC | 13:14 | |
| *** mariaderidder has quit IRC | 13:14 | |
| *** bashrc_ has quit IRC | 13:14 | |
| *** CTtpollard has quit IRC | 13:14 | |
| *** nowster has quit IRC | 13:14 | |
| *** jonathanmaw has quit IRC | 13:14 | |
| *** edcragg has joined #baserock | 13:14 | |
| *** tiagogomes has joined #baserock | 13:14 | |
| *** CTtpollard has joined #baserock | 13:14 | |
| *** wschaller has joined #baserock | 13:14 | |
| *** bashrc_ has joined #baserock | 13:14 | |
| *** mariaderidder has joined #baserock | 13:14 | |
| *** nowster has joined #baserock | 13:14 | |
| *** jonathanmaw has joined #baserock | 13:14 | |
| *** petefoth has joined #baserock | 13:14 | |
| *** gary_perkins has joined #baserock | 13:15 | |
| *** toscalix has joined #baserock | 13:29 | |
| *** paulwaters_ has quit IRC | 13:54 | |
| richard_maw | hmm, webapp.db doesn't contain a jobs table… | 14:08 |
| richard_maw | currently the cleanup is failing because it doesn't have a jobs table, so there are no jobs listed to clean up | 14:09 |
| richard_maw | ssam2: how did you rotate the webapp.db? | 14:09 |
| richard_maw | ssam2: yeah… git.baserock.org's lorry-controller is not working, the minions are failing to request jobs, I think because the jobs table is missing | 14:15 |
| Kinnison | richard_maw: eep | 14:16 |
| richard_maw | …it will initialize the tables if when the StateDB is accessed the file doesn't exist, but there is a race between opening and using, so if the DB was rotated at the wrong time then it won't be initializde | 14:17 |
| ssam2 | richard_maw: I just rename it | 14:17 |
| ssam2 | sometimes I have to restart the process after, I guess I forgot this time | 14:18 |
| ssam2 | restarting it will fix it | 14:18 |
| * richard_maw isn't so sure about that unless you also remove the DB | 14:22 | |
| ssam2 | oh, yeah. | 14:24 |
| richard_maw | as for why it was created in a poor state, I'm guessing it's a race between the file being removed and it being used | 14:25 |
| Kinnison | that's worrisome since the webapp is supposed to not be racy at all for the db | 14:26 |
| richard_maw | for general use it's race safe from locking when using it, but there's a TOCCTOU bug where it checks whether the file exists, stores that result boolean, then connects to the db, and will conditionally initialize it based on the boolean | 14:27 |
| Kinnison | urgh | 14:28 |
| richard_maw | so if you're unlucky enough to remove the file after it's been checked, but during use, then it will create it empty, and fail to notice it needs to initialize it | 14:28 |
| Kinnison | eww | 14:33 |
| Kinnison | Then again, if the old job report cleanup was working, we'd not have to keep rming it, no? | 14:33 |
| richard_maw | correct, but rotating the DB like this isn't a great solution, so we need to fix or replace in-place cleanup surely | 14:35 |
| richard_maw | hrm, the sqlite3 module doesn't let you pass a file descriptor (other than hackily with /proc/self/fd/%d), so we can't do an open to race-freely determine whether it needs initializing. We could uncionditionally run some create statements that will create the table if we don't have it, though that's more overhead | 14:38 |
| Kinnison | I'd rather we just stop rotating it | 14:39 |
| Kinnison | If we can fix the issue with the size | 14:39 |
| richard_maw | yep | 14:39 |
| *** fay_ has quit IRC | 14:54 | |
| richard_maw | ssam2: did you take action on lorry-controller, or should I? | 15:00 |
| richard_maw | hrm, the C api to sqlite3 requires the file path, though _it_ has flags which would let us open/create it race-free, however they didn't make it to the python level API | 15:15 |
| Kinnison | If we only ever rotate the database by stopping the webapp first, then we're safe, yes? | 15:16 |
| richard_maw | yes | 15:17 |
| richard_maw | AIUI | 15:17 |
| *** franred has joined #baserock | 15:18 | |
| Kinnison | in that case I'm less bothered about trying to code in fixes to people doing bad things, and more interested in the cleanups. Did the l-c on gbo have the cleanup stuff in it, and if so why wasn't it working usefully? | 15:19 |
| ssam2 | richard_maw: i've not, go ahead and restart if you're logged in anyway | 15:20 |
| ssam2 | it seems to have the cleanup service there (I didn't realise that before), so not sure why it doesn't work | 15:21 |
| richard_maw | and we can't with the DB broken | 15:21 |
| * richard_maw cleans up | 15:21 | |
| ssam2 | thanks! | 15:23 |
| *** wschaller has quit IRC | 15:25 | |
| *** cosm has quit IRC | 15:29 | |
| *** wschaller has joined #baserock | 15:41 | |
| *** persia has quit IRC | 15:46 | |
| *** persia has joined #baserock | 15:47 | |
| pedroalvarez | oh well, I think I have something I can try to deploy into this sdcard for the raspberry | 15:49 |
| richard_maw | Kinnison: the default period for what's considered too old a job is 1 year, which would explain why it's not cleaned any up | 15:53 |
| Kinnison | richard_maw: aah | 15:53 |
| Kinnison | richard_maw: Frankly a week is probably enough | 15:54 |
| Kinnison | richard_maw: but perhaps set it to a day, and we'll see how it looks on Monday? | 15:54 |
| richard_maw | I was going to go with 2d, but 1d is probably more responsive | 15:54 |
| richard_maw | ok, set to 1d now | 15:58 |
| richard_maw | the logic for cleanup is tested by yarns, so it ought to work, just we had inappropriate config | 15:58 |
| Kinnison | Seems plausible | 15:59 |
| *** CTtpollard has quit IRC | 16:00 | |
| richard_maw | we might want smarter logic, like keeping the logs for the most recent success, and the start and end of a chain of failures | 16:00 |
| Kinnison | Hmm, for now I'll settle for just not needing to rotate the db :_) | 16:01 |
| ssam2 | richard_maw: so simple ! are you fixing the default or just the value in git.baserock.org ? | 16:03 |
| ssam2 | the former would be better I think, so other troves don't have the same problem | 16:03 |
| richard_maw | trying the value on g.b.o first | 16:03 |
| *** franred has quit IRC | 16:15 | |
| *** ssam2 has quit IRC | 16:15 | |
| pedroalvarez | edcragg: I see you have added a boolean to use partitioning or not :) | 16:16 |
| *** jonathanmaw has quit IRC | 16:16 | |
| pedroalvarez | I guess with that we are good to not break the flashing script until we create another one | 16:16 |
| *** tiagogomes has quit IRC | 16:18 | |
| pedroalvarez | hm.. no, this image is not going to boot :/ | 16:30 |
| SotK | :( | 16:30 |
| pedroalvarez | I need to put somewhere where is the rootfs | 16:31 |
| pedroalvarez | instead of "root=/dev/mmcblk0p2" | 16:32 |
| pedroalvarez | it might be a matter of adding extra info for btrfs | 16:33 |
| pedroalvarez | footfstype and footflags | 16:33 |
| pedroalvarez | s/^f/r/ | 16:33 |
| pedroalvarez | anyway, I'm going to try it to see how it crashes | 16:35 |
| pedroalvarez | meh | 16:37 |
| * SotK wishes pedroalvarez luck | 16:38 | |
| pedroalvarez | SotK: thanks | 16:39 |
| pedroalvarez | I'm going to need it :) | 16:39 |
| pedroalvarez | First of all, the /boot partition is empty. My partitioning file may be wroing... or the code broken. Will investigate | 16:39 |
| pedroalvarez | I may need something like a serial cable to be able to do this,, I wonder if that exists | 16:40 |
| pedroalvarez | yay!!! kernel panic!!! | 16:47 |
| * pedroalvarez dances | 16:47 | |
| pedroalvarez | great | 16:47 |
| pedroalvarez | so I have to work out why the boot partition is not been populated, and then work out how to make the kernel find the rootfs | 16:48 |
| pedroalvarez | :D | 16:48 |
| *** mariaderidder has quit IRC | 16:53 | |
| pedroalvarez | hm.. modifying the cmdline.txt file in the boot partition didn't work | 16:57 |
| Zara | :( | 16:59 |
| *** bashrc_ has quit IRC | 17:00 | |
| *** wschaller has quit IRC | 17:00 | |
| pedroalvarez | I kind of think that if I use an ext3 partition it will work | 17:01 |
| pedroalvarez | but I'll leave that for another moment | 17:01 |
| Zara | heh, hope it works out. 'night. | 17:01 |
| pedroalvarez | night Zara! | 17:02 |
| Zara | :) | 17:02 |
| *** edcragg has quit IRC | 17:16 | |
| *** gary_perkins has quit IRC | 17:18 | |
| *** gtristan has quit IRC | 17:39 | |
| *** toscalix has quit IRC | 17:46 | |
| * pedroalvarez discovers BOOT_PRINTK_DELAY | 18:39 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!