| *** genii [~quassel@ubuntu/member/genii] has quit [Read error: Connection reset by peer] | 01:16 | |
| *** dutch_ [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 07:12 | |
| *** tiagogomes [~tiagogome@213.15.255.100] has joined #baserock | 07:29 | |
| *** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 07:44 | |
| *** dutch_ [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Quit] | 07:54 | |
| *** dutch_ [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 07:54 | |
| *** mwilliams_ct [~mikewilli@access.ducie-dc1.codethink.co.uk] has quit [] | 08:18 | |
| *** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 08:23 | |
| *** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 08:35 | |
| richard_maw | I saw that problem before when I'd done an SVN import of a project that liked large binaries. I think the issue was cgit loading some of the objects into memory, or part of generating the per-commit tarballs. | 08:37 |
|---|---|---|
| richard_maw | there's probably something we can tune, and a more recent cgit may behave better | 08:37 |
| Kinnison | We might want to refresh to a more recent cgit anyway | 08:37 |
| richard_maw | but I think it's the mariadb import that's causing problems | 08:37 |
| Kinnison | it adds things like gravatar for authors/committers | 08:37 |
| Kinnison | mariadb is imported now and should be clean unless git itself is killing things | 08:38 |
| *** violeta [~violeta@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 08:38 | |
| richard_maw | the specific error is lighttpd being unable to fork because it doesn't have enough memory | 08:43 |
| pedroalvarez | patch sent to fix libevent | 08:44 |
| Kinnison | gbo has 12G of RAM | 08:44 |
| Kinnison | I wonder if we have too many vCPUs on it | 08:44 |
| Kinnison | git operations can chew a lot of RAM | 08:45 |
| Kinnison | Hmm, the mariadb repack is consuming 5G | 08:46 |
| Kinnison | which is odd because lorry is supposed to configure it to limit the window size etc | 08:46 |
| persia | Could run vmstat into a log for a while to check | 08:46 |
| Kinnison | Unfortunately busybox doesn't have vmstat | 08:46 |
| paulsher1ood | Kinnison: i suggest we blow away mariadb before folks start to depend on it | 08:48 |
| Kinnison | I can do so, but it's a bit crappy | 08:48 |
| persia | Do we not expect it to stabilise once the repack is complete? | 08:48 |
| * Kinnison is examining things | 08:48 | |
| *** mwilliams_ct [~mikewilli@access.ducie-dc1.codethink.co.uk] has joined #baserock | 08:53 | |
| ctgriffiths | Kinnison: If you need to nuke it we have other options, could you let me know what you decide? | 09:02 |
| Kinnison | ctgriffiths: Sure | 09:02 |
| *** ctalex [~alex@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 09:11 | |
| Kinnison | Mariadb is the largest repo on gbo by a factor of five | 09:16 |
| Kinnison | I am tempted to drop the bzr import and request tarball imports | 09:16 |
| ssam2 | that seems reasonable to me. I can't speak for everyone, but I've no desire to use unstable versions of MariaDB anyway | 09:21 |
| Kinnison | ctgriffiths: Would tarball imports be okay? | 09:23 |
| ctgriffiths | Kinnison: I don't know, maybe paulsher1ood could advise? | 09:25 |
| Kinnison | ctgriffiths: are you expecting to do mariadb development, or just use it? | 09:25 |
| *** liw-orc [~liw@access.ducie-dc1.codethink.co.uk] has joined #baserock | 09:26 | |
| ctgriffiths | Kinnison: just using it | 09:26 |
| * Kinnison shall cancel the delta set from bzr import because it's just too damned big | 09:26 | |
| Kinnison | please hold | 09:26 |
| *** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds] | 09:32 | |
| *** fay [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 09:32 | |
| fay is now known as Guest21397 | 09:32 | |
| paulsher1ood | is there a way to verify that dbus is alive on a genivi baseline system? | 09:36 |
| Guest21397 is now known as fay_ | 09:37 | |
| paulsher1ood | or any system, for that matter? | 09:37 |
| richard_maw | if you can use systemctl, there's some form of dbus working, though it may just be the systemd private bus | 09:38 |
| persia | If you have a dbus client, you can check with that, but I doubt the dbus debug clients are part of a baseline system | 09:38 |
| Kinnison | dbus may also be socket-activated and thus not show up until you use it | 09:38 |
| ssam2 | the baseline probably has `gdbus` | 09:39 |
| paulsher1ood | yes, gdbus is here | 09:40 |
| ssam2 | Ok, I can work out a suitable invocation to test D-Bus. It'll take a few minutes :) | 09:40 |
| persia | `gdbus monitor --system` ought be a start, no? | 09:40 |
| ssam2 | yep | 09:41 |
| paulsher1ood | Error: destination is not specified | 09:41 |
| persia | What dbus do you want to watch? | 09:42 |
| ssam2 | paulsherwood: gdbus call --session --dest=org.freedesktop.DBus --object-path=/ --method=org.freedesktop.DBus.ListNames | 09:42 |
| ssam2 | will list everything on the session bus, or fail if there's no session bus | 09:42 |
| ssam2 | change --session to --system for the system bus | 09:42 |
| ssam2 | I don't know if we start a session bus automatically in Baserock, I'd suspect not due to lack of PAM | 09:43 |
| persia | For instance, to check NetworkManager on my desktop, I can use `gdbus --monitor --system --dest org.freedesktop.NetworkManager | 09:43 |
| paulsher1ood | ssam2: http://fpaste.org/134154/ | 09:45 |
| paulsher1ood | http://fpaste.org/134156/ | 09:46 |
| ssam2 | paulsher1ood: that means there's no session bus, and gdbus tried to start one automatically | 09:46 |
| ssam2 | but it failed because for some reason it must have an X session to do that | 09:46 |
| ssam2 | if you set 'DISPLAY=:0' you might see success | 09:46 |
| ssam2 | but of course, and empty bus | 09:46 |
| ssam2 | *an empty bus | 09:47 |
| radiofree | is that still the case, needs an X session? | 09:47 |
| ssam2 | looks like the system bus is fine | 09:47 |
| pedroalvarez | genivi-baseline doesn't have X | 09:47 |
| ssam2 | It might work anyway. I'm not an expert on why it needs that. | 09:47 |
| paulsher1ood | so if system bus is fine can i assume dbus has survived my mangling? | 09:48 |
| ssam2 | yeah | 09:48 |
| paulsher1ood | cool! | 09:48 |
| ssam2 | what did you mangle ? | 09:48 |
| richard_maw | kdbus? | 09:48 |
| paulsher1ood | i'll email the list... | 09:48 |
| radiofree | paulsher1ood: i believe you can launch it without X | 09:48 |
| radiofree | just set DBUS_SESSION_BUS_ADDRESS and make sure any app that needs it has that as well | 09:49 |
| paulsher1ood | no, just updatd - genivi-dev had notice of some vulnerabilities | 09:49 |
| ssam2 | oh, yeah, it's only the autolaunch thing that needs X, not the daemon itself | 09:49 |
| paulsher1ood | radiofree: i'm sure you're right, but your instructions missed my head due to altitude :) | 09:50 |
| radiofree | paulsher1ood: http://dbus.freedesktop.org/doc/dbus-launch.1.html | 09:50 |
| radiofree | To start a D-Bus session within a text\(hymode session, do not use dbus-launch. Instead, see dbus-run-session(1). | 09:50 |
| *** dutch_ [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 272 seconds] | 09:53 | |
| Kinnison | I'm going to rm the mariadb stuff now | 09:57 |
| persia | it won't complete lorrying? | 09:58 |
| Kinnison | It does, but at over 5.1G of repo it's a total pig | 09:59 |
| richard_maw | it's lorried AIUI, but it crashes the lighttpd serving cgit | 09:59 |
| Kinnison | aye | 09:59 |
| persia | That's annoying. | 09:59 |
| Kinnison | Aye | 10:00 |
| pedroalvarez | an upgrade of cgit was also suggested, no? | 10:00 |
| *** dutch_ [~william@92.40.19.162.threembb.co.uk] has joined #baserock | 10:00 | |
| persia | Let's try that instead | 10:00 |
| Kinnison | I've removed the public side and the lorry but left the work area | 10:00 |
| Kinnison | so if we decide gbo *can* cope with mariadb, it won't have to do the 60G reconvert | 10:01 |
| persia | Ah, good. | 10:01 |
| Kinnison | that's the best option I feel | 10:01 |
| persia | What's the best way to test this? Does testing depend on upgrades of g.b.o, or can we test a new lighttpd/cgit somewhere else, and only upgrade if it works? | 10:02 |
| Kinnison | We ought to be able to test elsewhere | 10:03 |
| Kinnison | we can copy the repo manually | 10:03 |
| persia | Sounds like a reasonable plan | 10:04 |
| paulsher1ood | i've mailed the list on my dbus adventure | 10:06 |
| * paulsher1ood looked at cgit updates a while ago and this seemed to be tightly coupled to git version | 10:16 | |
| richard_maw | that's why cgit embeds git in its repo | 10:17 |
| pedroalvarez | good point | 10:19 |
| richard_maw | it may be better if it used libgit2, but AIUI it didn't exist when cgit needed it, and libgit2 isn't stable yet anyway, so you'd end up swapping a submodule of git for a submodule of libgit2 | 10:21 |
| persia | It wouldn't hurt to update git anyway: useful new features | 10:21 |
| richard_maw | potential breakages though, since I think git's done the 1.x to 2.x transition now | 10:22 |
| persia | Right, but the converse is becoming true, as more and more upstream projects become managed with git 2+ | 10:26 |
| *** tiagogomes [~tiagogome@213.15.255.100] has quit [Ping timeout: 272 seconds] | 10:45 | |
| Kinnison | I think modern cgit has updated to git 2.x anyway | 10:47 |
| pedroalvarez | can anybody take a look at the libevent patch? | 10:48 |
| pedroalvarez | I can't build distbuild systems now | 10:48 |
| pedroalvarez | It has already a +1 from Sam, so I think the approach is good | 10:48 |
| pedroalvarez | ssam2: is your +1 to the trove-setup patch (UPSTREAM_TROVE) still valid? | 11:07 |
| pedroalvarez | I changed the implementation, so I don't want to take that +1 without your approval | 11:08 |
| liw-orc | pedroalvarez, I'd already skimmed through that patch series for libevent, but a belated +1 now | 11:09 |
| pedroalvarez | grand! | 11:10 |
| pedroalvarez | I will merge it | 11:10 |
| *** tiagogomes [~tiagogome@213.15.255.100] has joined #baserock | 12:11 | |
| *** tiagogomes [~tiagogome@213.15.255.100] has quit [Read error: Connection reset by peer] | 12:15 | |
| *** nemos [~user@astra.technisat-digital.de] has joined #baserock | 12:46 | |
| pdar | /NETSPLIT | 13:23 |
| De|ta | Not recently? | 13:25 |
| paulsher1ood | :) | 13:28 |
| *** dutch_ [~william@92.40.19.162.threembb.co.uk] has quit [Quit: Quit] | 13:39 | |
| *** dutch_ [~william@92.40.19.162.threembb.co.uk] has joined #baserock | 13:39 | |
| *** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds] | 13:49 | |
| *** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 14:19 | |
| *** jjardon [sid723@gateway/web/irccloud.com/x-gaxyjbccvverqxew] has quit [Read error: Connection reset by peer] | 14:36 | |
| Kinnison | Anyone object to me looking at a newer cmake version? | 14:38 |
| Kinnison | A project I want to build (smartdevicelink) needs 2.8.11 and we have 2.8.9 | 14:38 |
| persia | \o/ updates! | 14:38 |
| * Kinnison was going to update us to 2.8.12.2 | 14:38 | |
| Kinnison | as the latest non 3.x release | 14:38 |
| Kinnison | (who knows how compatible 3.x might be) | 14:39 |
| persia | Do you have infrastructure that would let you test-build with 3.x without intense pain, just to see? | 14:39 |
| Kinnison | Right now I'm working with a small ARM distbuild cluster, so no, not really | 14:39 |
| Kinnison | Also given I'm not a cmake guru, I wouldn't know whether it was a user fault or a cmake incompatibility, or a bug | 14:40 |
| persia | makes sense | 14:42 |
| * Kinnison kicks off a build with 2.8.12.2 | 14:42 | |
| Kinnison | 2014-09-17 14:42:56 Build steps in total: 1498 | 14:42 |
| Kinnison | 2014-09-17 14:42:58 Progress: Need to build 1151 artifacts | 14:42 |
| Kinnison | this may be some time | 14:42 |
| Kinnison | At least I have 4 workers \o/ | 14:42 |
| pedroalvarez | good :) | 14:43 |
| persia | There are 1151/1498 artifacts that require *cmake* ?!? | 14:43 |
| Kinnison | persia: No, but cmake is in core | 14:44 |
| Kinnison | persia: so things built with cmake end up as build-deps of other things | 14:44 |
| Kinnison | and so the graph grows | 14:45 |
| persia | Ah, hrm. | 14:45 |
| persia | core is perhaps overbroad, but I suspect that there's not much savings available from more granular stratification | 14:45 |
| Kinnison | If we had 'morph hack' I could have tested modern cmake with the one chunk I wanted, and then later done the full test | 14:45 |
| Kinnison | but we don't (yet) | 14:45 |
| persia | What would `morph hack` do? | 14:46 |
| Kinnison | allow me to pull cmake up to my top level stratum where this thing is which needs it updating, let me update it and continue trying to integrate this tool, without needing to rebuild everything which b-d'd on the original cmake | 14:46 |
| *** jjardon [sid723@gateway/web/irccloud.com/x-wudxwymclsuvfioj] has joined #baserock | 14:46 | |
| Kinnison | then at system construction time, it'd sub in, instead of the original cmake | 14:47 |
| Kinnison | when putting the system together | 14:47 |
| *** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 246 seconds] | 14:47 | |
| Kinnison | i.e. not good for releases, but really handy for hacking | 14:47 |
| persia | (As an aside, I find that I'm coming to like the use of "stratum" as a technical term, because all the natural grammatical uses happen to already exist as words in English) | 14:47 |
| Kinnison | I think Rob Taylor originally suggested it | 14:47 |
| Kinnison | Perhaps even on the ML | 14:47 |
| liw-orc | it was given to me when I started, before the ML existed, I think | 14:48 |
| persia | I'm not movitivted enough to complain in depth now, but shall do so if anyone submits a patch to do that. | 14:48 |
| persia | It's deeply and fundamentally broken. | 14:48 |
| ssam2 | robtaylor has since suggested using GNU Stow to achieve the same thing | 14:51 |
| ssam2 | actually no, that was to achieve a different thing | 14:51 |
| ssam2 | ignore me! | 14:51 |
| paulsher1ood | Kinnison: would it make sense to do this on x86 disbtuild first? | 14:51 |
| Kinnison | paulsher1ood: it might, but I'm all set up on the arm distbuild, and it's good to give the Jetsons a workout I think :-) | 14:52 |
| paulsher1ood | sure | 14:53 |
| * Kinnison is trying to get a more modern SDL than the one you integrated | 14:53 | |
| Kinnison | so that it has the Qt HMI | 14:53 |
| Kinnison | and that means more modern cmake | 14:53 |
| paulsher1ood | oh, hold on a sec | 14:53 |
| Kinnison | I think I have everything else under control though | 14:53 |
| paulsher1ood | i ithink i integrated latest stable? | 14:53 |
| Kinnison | You integrated release 1.0 | 14:55 |
| Kinnison | release 3.0 is latest stable | 14:55 |
| paulsher1ood | bah. my y mistake | 14:55 |
| * paulsher1ood goes to check if he pushed the wrong branch | 14:55 | |
| paulsher1ood | i rememeber now - i moved the morph into definitions as first step, thne forgot about second step | 14:58 |
| Kinnison | :-) | 14:58 |
| paulsher1ood | :( | 14:58 |
| Kinnison | You'd have hit the need for a cmake update if you had | 14:58 |
| *** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 14:59 | |
| paulsher1ood | Kinnison: while you are on, please could you consider adopting/merging my baserock/ps/tidy-jetson patch? | 15:41 |
| *** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds] | 15:43 | |
| * Kinnison looks at it | 15:44 | |
| Kinnison | Sure that all looks sane | 15:45 |
| Kinnison | In fact it might make more sense to merge that to master and then I can rebase on top of it | 15:45 |
| * Kinnison wonders if anyone else here has access to a Jetson and wouldn't mind taking Paul's patch, merging it to master and verifying nothing breaks? | 15:46 | |
| Kinnison | All my build nodes are "busy" | 15:46 |
| paulsher1ood | Kinnison: how could anything break? i rename linux to something sensible, and add an upgrade cluster morph, is all :) | 15:53 |
| paulsher1ood | i did *test* it before i submitted, too :-) | 15:53 |
| paulsher1ood | but if you insist i have a jetson here. what would you like me to try? | 15:54 |
| Kinnison | either a merge or rebase on top of master | 15:54 |
| Kinnison | and a build | 15:54 |
| paulsher1ood | cherry-pick? | 15:55 |
| *** ctalex [~alex@82-70-136-246.dsl.in-addr.zen.co.uk] has quit ["Ex-Chat"] | 15:55 | |
| Kinnison | paulsher1ood: I appreciate that you are telling me what you did and that I agree I *see* it's right, but I'm good at failing to see syntax errors in yaml and json :-) | 15:55 |
| Kinnison | paulsher1ood: and I'd be a bad person to ask for a merge without at least a parse-the-build-graph check :-) | 15:55 |
| Kinnison | cherry-pick, rebase or plain merge is fine | 15:55 |
| * paulsher1ood misspooke, he has -several- jetsons here | 15:56 | |
| * Kinnison chuckles | 15:58 | |
| Kinnison | you're rich in jetsons | 15:58 |
| Kinnison | Just don't "borrow" any of mine | 15:58 |
| Kinnison | I'm using 'em all | 15:58 |
| paulsher1ood | sadly, i have zero kettle leads. what's up with that ? | 15:59 |
| Kinnison | doh | 16:01 |
| *** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock | 16:06 | |
| * paulsher1ood misspoke again - they were ghiding in his briefcase | 16:10 | |
| Kinnison | hehe | 16:11 |
| *** dutch_ [~william@92.40.19.162.threembb.co.uk] has quit [Quit: Quit] | 16:14 | |
| paulsher1ood | Kinnison: does http://fpaste.org/134276/ convince you? | 16:15 |
| paulsher1ood | ssam2: does http://fpaste.org/134276/ convince you that i'm seeing minutes deciding wht to build? | 16:15 |
| persia | Is that three minutes before it does anything useful? | 16:18 |
| paulsher1ood | yup | 16:18 |
| persia | Taking a minute to calculate the build set doesn't seem unreasonable, although I suspect it could usually start with a random order in parallel with that with equivalent or better performance. | 16:20 |
| paulsher1ood | to be fair, this is morph from a couple of months ago, it may be that current is faster | 16:20 |
| paulsher1ood | note this is not distbuild | 16:20 |
| persia | But what part takes two minutes to decide to build the file requested for build? Is this leftovers from the morphs-in-definitions stuff? | 16:20 |
| paulsher1ood | i have no clue | 16:20 |
| pedroalvarez | paulsher1ood: have you used morph edit in that workspace? | 16:21 |
| paulsher1ood | nope | 16:22 |
| paulsher1ood | it's fresh today | 16:22 |
| paulsher1ood | it's on a jetson, though | 16:22 |
| paulsher1ood | ssd | 16:22 |
| persia | I wonder if those two minutes are spent validating the cache objects | 16:23 |
| paulsher1ood | all i did was morph branch b:b/definitions, then cherry-pick, then build | 16:23 |
| persia | You had previously worked in a separate workspace in that /src? | 16:24 |
| ssam2 | paulsherwood: I need to go fairly soon, but running that command again with -v would be helpful to diagnose | 16:25 |
| *** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving] | 16:25 | |
| ssam2 | I have a hunch that the problem is the many individual network requests it makes to the cache, and the problem is much worse for you than for me due to some difference in network config or which trove we're using | 16:26 |
| paulsher1ood | i'm using gbo | 16:26 |
| ssam2 | I thought so | 16:26 |
| paulsher1ood | i used to have my own trove. i may spin one up again soon :-) | 16:26 |
| paulsher1ood | (since it seems to have become trivial) | 16:27 |
| ssam2 | we have an internal one that mirrors git.baserock.org, too, named 'ct-mcr-1' | 16:27 |
| paulsher1ood | you may be disappointed by -v output :) | 16:27 |
| * persia would like less dependency on network operations | 16:27 | |
| * ssam2 would like more clarity over when network is required and advice how to achieve a setup where network is not be required | 16:28 | |
| persia | While they *can* be fixed by "just deploying a trove", this has non-trivial bandwidth and storage requirements. | 16:28 |
| * paulsher1ood would like the moon on a stick | 16:28 | |
| ssam2 | *advice on | 16:28 |
| ssam2 | ultimately, if you want to be able to build anything from the internet, you need network access | 16:29 |
| straycat | a 'default' trove still requires a fair amount of memory though, I think. | 16:29 |
| ssam2 | if you want to be able to build anything on a given Trove system, you either need network access to that Trove, or for the entire contents of that Trove to be stored locally | 16:29 |
| Kinnison | paulsher1ood: looks good to me | 16:30 |
| paulsher1ood | is that a +2? :-) | 16:30 |
| ssam2 | (we can decide whether or not a given method of mirroring gits should be called 'Trove' or not, of course) | 16:30 |
| persia | ssam2: Yes, but having hundreds of miscellaneous things happen adds up quickly when latencies grow, to the point where it isn't interesting to use the software unless you happen to be network-near a trove. Surely there's some compromise. | 16:30 |
| Kinnison | paulsher1ood: Aye, you can have a +2 :-) | 16:31 |
| persia | When working with traditional distros, I download a catalogue of available bimaries, and then download individual binaries, with an error if it was removed in the meantime. | 16:31 |
| ssam2 | persia: there are several possible compromises, the trouble is: which ones? | 16:31 |
| paulsher1ood | Kinnison: tvm. all i need now is someone with the super powers to merge it | 16:31 |
| Kinnison | ssam2: Feel like merging paul's jetson cleanup? | 16:32 |
| ssam2 | need to leave soon, sorry | 16:32 |
| persia | ssam2: So, what are the network operations that get performed? I'd be happy to suggest which ones are uninteresting to me. | 16:32 |
| Kinnison | ssam2: bah :-) | 16:33 |
| Kinnison | pedroalvarez: You? | 16:33 |
| * Kinnison would do it but currently has all his stuff geared up on a local trove | 16:33 | |
| Kinnison | and I'm bound to fluff it if I try and do a partial revert to gbo | 16:33 |
| pedroalvarez | I will | 16:33 |
| ssam2 | persia: my hunch about paulsherwood's system's slowness is correct, it's a bunch of calls to 'morph-cache-server' which do 'git rev-parse', 'git ls-files' and 'git cat-file' in order to calculcate a build graph and build instructions | 16:34 |
| Kinnison | ssam2: Mmm, I agree that your idea to batch those if possible would help a *lot* | 16:34 |
| ssam2 | Kinnison: I've looked at doing it, but you're right that there's quite a bit of code rework needed | 16:34 |
| paulsher1ood | here's the -v output http://fpaste.org/134280/ | 16:34 |
| persia | ssam2: That's about what I thought. Hrm. I'd like to not to that, but that conflicts with my other want to not have local git cache anyway, so perhaps I'm not quite as happy as I thought :) | 16:35 |
| * persia ponders | 16:35 | |
| ssam2 | paulsherwood: thanks | 16:35 |
| paulsher1ood | yw | 16:35 |
| ssam2 | seems that the first 2 minutes is spent working out if a temporary build branch needs to be created | 16:35 |
| ssam2 | by examining all the files in your definitions working tree | 16:35 |
| paulsher1ood | so, let's scotch that. never do temp branches | 16:36 |
| persia | +1 | 16:36 |
| pedroalvarez | just aksing: temp branches in a local build? | 16:36 |
| paulsher1ood | as persia has suggested - only build head | 16:36 |
| ssam2 | then there's 1 minute calculating the build graph and build instructions, mostly through local git cache access but some remote cache access too | 16:37 |
| paulsher1ood | user commits, then commit --amend to repeat | 16:37 |
| persia | Note that this does add effort for developers, who have to commit before building, and then clean up if they made a silly mistake with --amend | 16:37 |
| ssam2 | I find the temporary build branch stuff takes 5 to 10 seconds, which I find annoying | 16:37 |
| ssam2 | presumably paulsherwood's VM has very slow IO | 16:37 |
| paulsher1ood | that's not effort, persia. i'm tempted to script it :) | 16:37 |
| paulsher1ood | it's not a VM | 16:37 |
| ssam2 | oh | 16:37 |
| paulsher1ood | it's baserock, on jetson | 16:38 |
| ssam2 | ah, right | 16:38 |
| persia | Shouldn't be too slow IO, if it's SSD | 16:38 |
| *** genii [~quassel@ubuntu/member/genii] has joined #baserock | 16:38 | |
| *** genii [~quassel@ubuntu/member/genii] has quit [Read error: Connection reset by peer] | 16:38 | |
| ssam2 | yeah, that's a little strange. | 16:38 |
| ssam2 | maybe you have lots of chunk repos checked out, so it's looking in those too | 16:38 |
| ssam2 | certainly I'd be in favour of making it possible to disabling temporary build branches | 16:38 |
| ssam2 | *disable | 16:38 |
| paulsher1ood | i have nothing checked out | 16:39 |
| paulsher1ood | at least not in this branch | 16:39 |
| ssam2 | right. I'm surprised it takes so long, then. I won't speculate further without seeing it in action | 16:39 |
| paulsher1ood | i'm here all week :-) | 16:39 |
| ssam2 | I can look tomorrow at making it possible to disable temporary build branches, it should be a small job | 16:40 |
| paulsher1ood | cool | 16:40 |
| richard_maw | let's try disabling the adding uncommitted changes first | 16:40 |
| ssam2 | richard_maw: oh, that's what I mean actually | 16:40 |
| ssam2 | it's the 'examine the entire working tree' part that is slow | 16:41 |
| ssam2 | thanks for pointing that out | 16:41 |
| Kinnison | ssam2: I think the major work is that we currently do an in-place visit of everything and expect it to fill in as we go. If we could do a pre-traverse and gather all the questions, ask them in one go, annotate, and then do the in-place traversal, it might work without changing too much | 16:41 |
| Kinnison | ssam2: alternatively we have to rework the core loop, but I'd not be upset if it were slightly less coupled :-) | 16:41 |
| ssam2 | Kinnison: some of the loop is still written as if strata can be in multiple repos, probably | 16:42 |
| Kinnison | Probably | 16:42 |
| ssam2 | since they're all in definitions, it should be possible to look at every stratum, build a graph, then query ref and build-system for every chunk in one go | 16:42 |
| * Kinnison still wants that data cached in the definitions commit but appreciates that might be hard | 16:44 | |
| persia | The main issue is that it's really hard to enforce it remains valid. | 16:44 |
| Kinnison | It can be made so that it'll always be valid, but not always be useful | 16:45 |
| persia | Yes, but unless it's useful, it's not even interesting! We want both valid and useful, or we require a network operation. | 16:46 |
| Kinnison | Well, when I said "not always be useful" I meant that it may not always be *complete* | 16:47 |
| Kinnison | e.g. if you change a ref for a chunk and commit that without letting morph update the cache, it may not be complete | 16:47 |
| persia | I meant that with "valid" :) | 16:48 |
| Kinnison | I'm now very confused :-) | 16:48 |
| Kinnison | But I am shattered and slightly unwell, which will be contributing to that | 16:48 |
| persia | But I can also commit a change involving causing new chunks to appear or old chunks to be removed, which causes greater grief. | 16:48 |
| pedroalvarez | paulsher1ood: is there a branch for the patch that I'm supposed to merge? | 16:50 |
| Kinnison | 16:42 < paulsher1ood> Kinnison: while you are on, please could you consider adopting/merging my baserock/ps/tidy-jetson patch? | 16:51 |
| Kinnison | pedroalvarez: ^^^ | 16:51 |
| pedroalvarez | Kinnison: ta | 16:51 |
| pedroalvarez | but that branch is not pushed | 16:53 |
| pedroalvarez | urrghh | 16:53 |
| pedroalvarez | it is | 16:53 |
| paulsher1ood | http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/log/?h=baserock/ps/tidy-jetson | 16:53 |
| pedroalvarez | apologies | 16:53 |
| paulsher1ood | it's been a long day. beer o'clocl | 16:53 |
| * straycat likes these nova tools | 16:54 | |
| richard_maw | OpenStack is pretty cool | 16:54 |
| paulsher1ood | +1 | 16:54 |
| paulsher1ood | OpenStack + Baserock is extra cool :) | 16:54 |
| * richard_maw wonders if the pun in Nova's name is a refernce to Short Circuit | 16:54 | |
| pedroalvarez | i will provide some instructions about how to create a distbuild network of many workers and a controller in openstack with only one image :) | 16:56 |
| paulsher1ood | pedroalvarez: i'll happily try them :) | 16:57 |
| pedroalvarez | paulsher1ood: baserock/ps/tidy-jetson merged | 16:57 |
| paulsher1ood | yay! | 16:58 |
| straycat | If you're doing that could you write them as separate docs, the docs we have now are getting a bit long. | 16:58 |
| paulsher1ood | straycat: it's time we separated out openstack i think | 16:59 |
| * straycat nods | 16:59 | |
| pedroalvarez | straycat: yeah, the one with info is pretty big now | 16:59 |
| pedroalvarez | with trove deployment info, I mean | 16:59 |
| straycat | paulsher1ood, planning to split that up quite soon | 17:00 |
| paulsher1ood | great :-) | 17:00 |
| straycat | the original trove doc I wrote wasn't too good in the first place and could use some improvement | 17:00 |
| *** liw-orc [~liw@access.ducie-dc1.codethink.co.uk] has joined #baserock | 17:01 | |
| *** ctgriffiths [~quassel@ec2-54-72-26-210.eu-west-1.compute.amazonaws.com] has joined #baserock | 17:01 | |
| *** ratmice_______ [bosshog@nightfall.forlorn.net] has joined #baserock | 17:01 | |
| * paulsher1ood volunteered to write docs ages ago, then failed to actually do so | 17:02 | |
| pedroalvarez | I'd like to have a versioned documentation | 17:02 |
| ssam2 | petefoth likes writing docs ;) | 17:02 |
| paulsher1ood | heh | 17:02 |
| pedroalvarez | with versioned documentation I mean, diferent documetnation for every release | 17:03 |
| paulsher1ood | interesting idea | 17:04 |
| paulsher1ood | first step, put the wiki doc onto gbo | 17:04 |
| pedroalvarez | many projects do that | 17:04 |
| *** violeta [~violeta@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 272 seconds] | 17:07 | |
| *** tpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Ex-Chat] | 17:15 | |
| *** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection] | 17:22 | |
| *** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving] | 17:28 | |
| *** nemos [~user@astra.technisat-digital.de] has quit [Ping timeout: 258 seconds] | 18:22 | |
| pedroalvarez | regarding my idea of "versioned documentation". This is an example: http://read-the-docs.readthedocs.org/en/latest/install.html | 18:38 |
| pedroalvarez | django also uses read-the-docs | 18:39 |
| pedroalvarez | I think it takes the documentation from git repositories. | 18:39 |
| pedroalvarez | Anybody has opinions on this? | 18:40 |
| Kinnison | It's an interesting idea. django's docs are usually good, but how much of that is readthedocs and how much is just django's team I don't know :-) | 18:41 |
| * paulsher1ood is in favour of exploring this | 18:56 | |
| persia | readthedocs is just a nice sphinx browser, so give credit to the team for the content, and readthedocs for the presentation | 19:13 |
| persia | WIth versioned docs, one usually wants to maintain them in parallel trees, so that one can apply patches sanely when docs for an older version would benefit from a fix. | 19:14 |
| persia | Note that this also requires a versioned release of the thing documented, so we'd want to have the conversation about morph releases again. | 19:15 |
| paulsher1ood | again? :) | 19:31 |
| persia | Yep. We'll probably keep having it until ether 1) we decide to have stable/development branches of morph with releases or 2) we give up on versioning anything | 19:34 |
| pedroalvarez | I'm probably going to take a look at the integration of this read the docs. | 19:42 |
| pedroalvarez | just for fun (for now) | 19:42 |
| pedroalvarez | and yeah, it only makes sense if we do versioned releases of moprh | 19:43 |
| pedroalvarez | or if we do versioned releases of baserock devel systems | 19:43 |
| pedroalvarez | I've raised this idea because in some places of the wiki we have documentation for latest morph, in other places documentation for an old morph, and everything mixed | 19:45 |
| persia | My memory is that it goes the other way around: the project exports sphinx, which readthedocs integrates, but I may be mistaken | 19:52 |
| persia | I'm still opposed to versioned releases of baserock devel systems, because it's rare that whatever gets released is useful to me, and I don't believe it possible to make a release of a devel system that is useful to several different folk. | 19:53 |
| pedroalvarez | I kind of agree, but sometimes we add some chunks needed to use morph, but yes you can also build it and deploy it to your system. But how do you get your first devel system? | 20:09 |
| * paulsher1ood can't remember what caused the 'Failed to open frame buffer device' error http://imgur.com/2nidKhe | 20:09 | |
| pedroalvarez | KERNEL_ARGS in your cluster morphology | 20:09 |
| pedroalvarez | one sec | 20:09 |
| paulsher1ood | gah! :) | 20:09 |
| paulsher1ood | tvm - i'll find it | 20:10 |
| persia | pedroalvarez: First devel system should be whatever would be built from HEAD of definitions, in my mind | 20:10 |
| pedroalvarez | paulsher1ood: I think the clusters/release.morph has an example of how to deploy genivi baseline, (it needs that kernel arg too to make framebuffer work) | 20:11 |
| paulsher1ood | thanks | 20:15 |
| * paulsher1ood gives jjardon a +1 and hopes someone else will follow suit :) | 20:21 | |
| rjek | +1 for Django's docs. It's easy to navigate and switch between releases. The actual text varies in quality, though. | 20:46 |
| pedroalvarez | Announcement: Public mason IP has changed. In my attempt of releasing the old IP and allocate the same IP for a fresh deployed mason instance, I lost it. | 21:34 |
| pedroalvarez | New ip: http://85.199.252.101/index.html | 21:34 |
| pedroalvarez | Again, this is a "work in progress", but is nice to have it in public. | 21:35 |
| paulsher1ood | cool! | 21:45 |
| paulsher1ood | nice to see a PASS to start, too :) | 21:46 |
| rjek | This is why we have DNS with low TTL :) | 22:07 |
| persia | rjek: The issue is that we don't seem to have a good API to update the DNS | 22:20 |
| rjek | Does Mythic Beasts not provide an API? | 22:22 |
| * rjek tsks. | 22:22 | |
| * rjek also creates a card on the Pepperfish kanban to provide an API for updating DNS. | 22:22 | |
| persia | Pepperfish manages baserock DNS? | 22:23 |
| rjek | (Although the URL and post scheme for the web interface is trivial to drive from scripts.) | 22:23 |
| rjek | persia: No. But it does host its mailing list. | 22:23 |
| rjek | Mythic Beasts handle Baserock.org's DNS IIRC | 22:23 |
| rjek | WHOIS confirms this. | 22:23 |
| * persia reads http://blog.mythic-beasts.com/2013/09/21/dns-api-implementing-dynamic-dns/ | 22:23 | |
| rjek | Looks like Mythic Beasts have an API :) | 22:24 |
| persia | Ugh. Passwords. | 22:24 |
| rjek | Over SSL at least. | 22:25 |
| persia | Yeah, but it means we either need to distribute the password, or have some service with rational authentication that proxies. | 22:25 |
| * rjek makes a note to rip that API off, but instead of a password allow users to generate and revoke API keys from the human web interface. | 22:25 | |
| rjek | Service credentials and public version control are always a headache. | 22:26 |
| persia | Doesn't solve the secrets problem for situations where the dynamic IP is hosted on untrusted infrastructure | 22:26 |
| rjek | It does let you limit what they can access, though | 22:27 |
| persia | True. | 22:27 |
| persia | There's a few projects out there for this sort of thing. Designate, for example. | 22:27 |
| rjek | And if you're doing things on untrusted infrastructure, you may have already lost. | 22:27 |
| * persia almost always does stuff on untrusted infrastructure: trusted infrastructure is expensive and doesn't have high-grade internet access | 22:28 | |
| rjek | Actually, a solution would be to generate a token that is actually a shared secret, and you hash your modification request with the token. | 22:28 |
| rjek | (ie, poor man's digital signature) | 22:28 |
| rjek | Stops sniffing, at least. | 22:29 |
| persia | Hash won't do it: you need reversible encryption. | 22:29 |
| rjek | Gets more complex if you want to prevent replay, obviously. | 22:29 |
| rjek | yeah, the shared secret is still on the box you don't trust that's also building software you run as root :) | 22:29 |
| persia | Presumably one isn't planning on running that software on trusted infrastructure | 22:30 |
| persia | Or one would have gone to the effort of constructing everything in a trusted manner in the first place, and this would not be a problem (as one could safely share the disposable secret inside the trust domain) | 22:30 |
| persia | Better docs on Mythic Beasts DNS API: https://www.mythic-beasts.com/support/api/primary | 22:36 |
| persia | pedroalvarez: Dunno if you have the password, but you might want to put together something that lets you set DNS entries, rather than needing bare numbers, and then chase whoever has the password to run it. | 22:37 |
| pedroalvarez | Sooner or later we will have to create dns entries | 22:42 |
| pedroalvarez | I'm not sure if this has to happen automatically | 22:44 |
| persia | I don't think it can happen automatically right now, but I think that if there is a script to do it, the person who knows the password might be more willing to do it. | 22:48 |
| persia | Of course, this presumes there exists someone who knows the password :) | 22:48 |
| radiofree | lots of assumptions | 23:38 |
| radiofree | though i'm not best qualified to comment on this considering i think baserock systems should have a default password | 23:39 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!