IRC logs for #baserock for Tuesday, 2014-09-16

*** thecorconian [~thecorcon@136.1.1.102] has quit [Remote host closed the connection]01:06
*** thecorconian [~thecorcon@136.1.1.102] has joined #baserock02:07
*** thecorconian [~thecorcon@136.1.1.102] has quit []02:43
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock06:54
*** dutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock07:20
*** dutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Read error: Connection reset by peer]07:20
*** ct-dutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock07:21
*** tiagogomes [~tiagogome@213.15.255.100] has joined #baserock07:33
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock07:47
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]07:49
* ssam2 is just finishing up talking in "The GENIVI Baselines" webcast: http://automotive.linuxfoundation.org/webinars07:58
ssam2there's another one at 15:00 UTC if anyone is interested07:59
ssam2wait, 16:00 UTC07:59
ssam2timezones are hard07:59
paulsher1oodhow did it go?08:00
ssam2first question: ""Native-compile only (right now)" -- any tentative timeline we can expect a cross compilation?"08:01
paulsher1oodwho asked that?08:01
ssam2went well though I think, about 15 attendees08:01
ssam2one of the attendees08:01
paulsher1oodwhat did you tell him/her?08:01
ssam2I said we don't have a timeline at this point08:01
ssam2although we're interested in it08:01
paulsher1ood'when hell freezes over, and the moon falls fron the sky'08:02
ssam2you never know :)08:02
paulsher1oodi replied to anand. i'm slightly worried that his patch didn't apply cleanly08:02
ssam2Thanks for that08:03
ssam2were you applying on rc3 or rc5 ?08:03
paulsher1oodrc508:03
ssam2hmm, weird. could be that his patch is against the btrfs-next tree or something08:03
paulsher1oodmaybe. hopefully he'll notice the difference in my reply08:05
pedroalvarezhi all08:17
pedroalvarezyesterday I finally found the problem with delta/gusb08:17
pedroalvarez(is not lorrying08:17
pedroalvarez)08:17
Kinnisonoh?08:18
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:18
pedroalvarezIs failing in this assert: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/lorry.git/tree/lorry#n28608:18
pedroalvarezand I think I know why and how to reproduce it08:19
KinnisonOh, did it splorf during initial clone?08:19
Kinnisonmariadb seems to not have completed either :-(08:20
KinnisonDespite completing its conversion08:20
pedroalvarezKinnison: we were trying to clone it with an ssh url. So the repository was created but empty.08:23
pedroalvarezthen we fixed it: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/local-config/lorries.git/commit/?id=c437b37edfe39068853e4bb06b4259a06ca542b908:23
Kinnisonpedroalvarez: Hmm08:23
Kinnisonpedroalvarez: it can be fixed by a trove admin, but bleurgh08:24
Kinnisonpedroalvarez: Do you want to fix it or shall I?08:24
pedroalvarezHm.. I'm not sure about how to fix it. Delete the repo?08:24
Kinnisonin the lorry working area, yes08:25
KinnisonAlso, that assert is probably removable08:25
pedroalvarezis what I was going to suggest.08:25
Kinnisonit seems over-pernickety08:25
pedroalvarezIf there is not source code, why do you want to do a backup?08:26
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:35
ssam2what is going on with http://85.199.252.95/ ?08:40
ssam2seems to have encountered a reproducible error in definitions ref 0a8fa2da3197a43a220e8d68a3287a66f90b210408:40
ssam2but then the build failure has magically fixed itself08:40
ssam2without the definitions ref changing08:40
* ssam2 very much hopes this is a parallel make issue08:42
KinnisonQuite probably08:43
Kinnisonthose kinds of failures in the past tended to be parallel build issues08:43
persiaEven so, if the build process isn't behaving deterministically, it bodes ill for reproduciblity08:43
ssam2persia: agreed.08:44
ssam2I'm not sure what we can do, other than either force 'max-jobs: 1' everywhere08:44
ssam2or stress test all of the chunks so that we discover any broken makefiles08:44
ssam2perhaps fixing up these errors as we see them will be enough08:44
Kinnisonpersia: s'a missing build-dep in the makefile08:44
paulsher1oodssam2: well in this case, force max-jobs for the breaking chunk08:44
paulsher1ood?08:44
ssam2or fix the makefile, yeah08:45
Kinnisonmax-jobs 1 will potentially help08:45
Kinnisonfixing the makefile is the right approach08:45
ssam2but the general problem is 'something that built successfully once may not build successfully again'08:45
KinnisonThere's nothing we can do for that beyond either forcing non-parallel builds, or fixing the issues as we find them08:46
KinnisonIt's very sad, but many upstreams simply don't care about making their makefiles robust08:46
persiaAnother alternative is patching make to accept an input file describing the specific acceptable parallelism, but that's likely to be more painful.08:47
Kinnisonpersia: the issue is a missing dependency08:48
Kinnisonpersia: the acceptable parallelism in any missing deps situation is, erm, one at a time08:48
persiaWell. no.08:48
*** violeta [~violeta@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:48
persiaIt really depends on how fast things take: if something is slow, and other things happening theoretically later have missing deps, the build fails.08:48
persiaIf it's fast the next time (using ccache, for example), then the build succeeds.08:49
KinnisonIn this case, I think ccache exacerbates the issue08:49
persiaAlthough it can also happen as a result of changes in build hardware (more cores, faster cores, etc.).08:51
Kinnisonyep08:52
persiaFor the mechanism that populates the artifact cache: is there a facility to stage chunk artifacts in such a way that it is possible to test if they misbuilt, or built significantly differently?08:53
* persia is pondering building everything a few times, and using consensus to determine which is "correct"08:53
KinnisonUnfortunately unclear08:54
Kinnison(how to achieve that)08:54
persiaWhy?08:54
KinnisonWe don't have any useful bit-for-bit comparison stuff yet, so how would you determine if two chunks are equivalent in any automatic way?08:55
persiaYou just build it several times, and compare the results: the content that matches the plurality of builds is probably correct.08:55
KinnisonHmm08:56
KinnisonThat'd slow distbuilds down somewhat08:56
persiabit-for-bit fails.  Capture the list of build artifacts, and some relevant aspects of them (symbols definitions for binaries, text match for text, etc.)08:56
pedroalvarezdelta/gusb fixed08:56
Kinnisonpedroalvarez: cool08:56
pedroalvarezjjardon_: ^08:56
persiaBy at least a factor of the number of builds participating in consensus, yes.08:56
persiaBut if a given definitions may build or not build non-deterministically, it's one way to deal with that without actually fixing the problem.08:57
ssam2for e2fsprogs, http://git.baserock.org/cgi-bin/cgit.cgi/delta/e2fsprogs.git/commit?h=5c15bf5f978bae01f1ca3cbe6414ab1d355a6adf seems to fix the issue08:57
ssam2so we just need to upgrade the version in use. I'll do a patch08:57
Kinnisonssam2: yep, that should fix it08:58
Kinnisonssam2: specifically the inclusion of prof_err.h in the deps08:58
ssam2sadly that commit doesn't seem to be in a stable release tag yet08:58
ssam2no, I'm lying08:59
ssam2(confused by 1.42.12 being sorted before 1.42.9 :)08:59
Kinnisonheh09:00
ridgerun1er is now known as ridgerunner09:23
*** ridgerunner [~robjones@access.ducie-dc1.codethink.co.uk] has quit [Quit: leaving]09:30
ssam2http://wiki.baserock.org/genivi/ says that one needs latest Morph to build the x86_64 GENIVI baseline. I think that's no longer true. Anyone know for sure ?09:46
ssam2also, I guess we can replace the deploy instructions on that page which give an explicit cluster with 'morph deploy clusters/release.morph genivi-baseline-system-x86_64-generic'09:47
pedroalvarezssam2: hmm,,, I think you are right about the "latest Morph"09:48
ssam2did 14.29 support chunks in definitions? I forget now !09:48
ssam2morph NEWS is only up to date for 14.28 anyway09:49
ssam2I guess we should still recommend to use latest morph, then09:49
franredssam2, I think you need the latest morph to build anything from morph in definitions09:50
ssam2right :)09:50
ssam2someone asked the disk space and CPU requirements for building the baseline too, so I'd like to put that on http://wiki.baserock.org/genivi/. Any rough guesses? I'd put 4GB RAM, 20GB disk space and at least 1GHz CPU, as a best-guess09:50
*** ridgerunner [~robjones@access.ducie-dc1.codethink.co.uk] has joined #baserock09:52
richard_mawyou probably won't need a full 4GB, since it's possible to build a 32-bit baseline09:56
Kinnisonssam2: Is there a branch somewhere with the fix for the 3.17 btrfs issue which I could use to test on my Jetson?10:02
ssam2Kinnison: I don't have one, paulsherwood may have10:07
Kinnisonpaulsher1ood: ?10:08
ssam2do we have a public artifact cache I can use? I remember there was one, but I forgot the link10:28
ssam2an artifact cache with up to date builds of master, I mean10:28
pedroalvarezssam2: yeah!10:29
persiaCan that be added to baserock.org DNS?  e.g. artifacts.baserock.org?10:29
persiaAnd can morph default to that?10:29
pedroalvarezhttp://http://85.199.252.93:8080/10:29
rjekI put an http in your http.10:29
pedroalvarezmeh10:30
pedroalvarezchrome...10:30
pedroalvarezto use it, add the follwing in morph.conf10:30
pedroalvarezartifact-cache-server = http://85.199.252.93:8080/10:30
ssam2persia: that might be a good solution10:31
persiaSaves everyone trying to remember annoying numbers10:31
pedroalvarezpersia: I had that suggestion in my notes10:31
ssam2I think we've always thought that trove.baserock.org should be doing that job, but the current system at trove.baserock.org doesn't have enough disk space to contain continuous builds of the artifacts10:31
ssam2if we had a separate subdomain for artifacts, we could be a bit more flexible10:31
persiaI'm not convinced I want the same backing hardware for git repos and an artifact cache.10:32
pedroalvarezI agree with persia 10:32
ssam2logical10:32
ssam2i've been brain damaged I guess, because 'everything in one box' is convenient for a developer10:33
persiaAbsolutely.  Makes code easy.  Causes extreme stress to sysadmins.10:33
ssam2the artifact cache isn't working, sadly, and Morph doesn't give me sufficient output to diagnose why10:34
ssam2I could have given it the wrong URL or whatever. I'll see about improving the error logging to find out why.10:34
persiaWhat do you mean by "isn't working"?10:34
ssam2ah, perhaps it is working10:34
pedroalvarezpersia: the thing is that this artifact-cache server may be temporary, and redeployed10:35
ssam2the problem is actually that the CI isn't building the GENIVI systems, so I have to build them locally10:35
persiaIsn't it behind NAT anyway?  Can't the NAT be adjusted?10:35
persiassam2: Yeah, I think we need elastic build workers before we can build all systems defined in definitions.10:36
jjardon_ is now known as jjardon10:36
jjardonpedroalvarez: thanks! and sorry for sending the incorrect url the first time10:38
paulsher1oodKinnison: http://git.baserock.org/cgi-bin/cgit.cgi/delta/linux.git/log/?h=baserock/ps/btrfs-fix10:40
tlsassam2: I did leave a mason instance building GENIVI systems, and uploading artifacts to ct-mcr-110:41
tlsassam2: only for x86-64 though10:41
paulsher1oodtlsa: that's a private trove10:42
tlsait was tlsa-controller on ct-stack-210:42
tlsaoh, I see10:42
tlsayes indeed10:42
ssam2I've switched http://http://85.199.252.95/ to building release.morph so we can publicise GENIVI artifacts10:42
tlsacool10:43
ssam2I guess this means feedback might be slower.10:43
ssam2but more valuable, as we'll notice if GENIVI breaks :)10:43
persiaMy worry is that it might miss a commit.10:43
ssam2how does including the GENIVI system make it possible for it to miss a commit ?10:43
persiaBut then again, it's better to miss some commits than all of them (which was the case before that was present).10:44
tlsamaybe that mason instance could be redeployed with several distbuild workers10:44
tlsato speed up cycle time10:44
persiaUnless I missed something in the backscroll I have yet to process, there's a timed poll, so that if two commits to definitions.git happen in sufficiently close succession, one may be missed.10:45
persiaSlowing down the build time increases the window of time in which this can happen.10:45
tlsapersia: that's correct10:45
persiatlsa: What happens if the prior build isn't yet complete when the next poll occurs?10:45
persiaOr does it only poll when it's not building?10:46
tlsait doesn't poll until a cycle is complete10:46
pedroalvarezpersia: hm... the same think will happens when we add tests for the deployed image in mason10:46
persiapedroalvarez: yes.10:46
pedroalvarezI wonder if we can force mason to just pull one commit10:46
persiaThe solution to these problems is to do things elastically: deploy separate build and test nodes for each commit.10:46
persiaWhich means decomposing the current solution into a controller and workers (each of which workers manage several build/test workers, etc.).10:47
paulsher1oodpersia: with docker :)10:49
* richard_maw grumbles about buzzwords10:50
persiapaulsher1ood: Why?  I'd use a cloud, personally.10:51
petefotheringhamI thought our preferred eventaul solution involved workers registering with the controiller and saying 'Gis a job!'10:57
paulsher1oodpersia: maybe i'm over-optimising :) i was imagining cloud vms with docker containers :)10:57
persiaAdding more levels of virtualisation is rarely described as "optimisation"10:58
persiapetefotheringham: That's even better :)10:58
petefotheringham is now known as petefoth10:59
* petefoth wonders where the petefotherignam came from!10:59
* paulsher1ood notices an odd message during a `morph deploy` - http://fpaste.org/133843/11:02
richard_mawpaulsher1ood: that's new from Lars' work to reduce the amount of data transferred when uploading images11:04
paulsher1oodis it a bug?11:05
richard_mawI'll let you know after I've investigated mroe11:06
richard_maws/mroe/more/11:06
* paulsher1ood updates morph in the meantime11:06
richard_mawthat won't fix it11:06
richard_mawin this case, it's because the mktemp on your mac behaves differently to those available on Linux11:07
paulsher1oodurgh :/11:07
richard_mawthe filename template is optional on Linux, but mandatory on a Mac11:08
persiamorph runs on a mac now?11:09
richard_mawno, the replacement for rsyncing data across runs a shell script on the remote end to convert the input stream into a disk image sparsely11:10
paulsher1oodas justin said on the ml, probably we should try to fix rsync?11:10
persiaAh, yeah, that needs to be POSIX clean, because we never know what lives on the other end.11:10
persiapaulsher1ood: How is a userspace tool supposed to know how the OS decided to represent a file?11:11
richard_mawI fear the dd command is the show-stopper for Mac support11:12
persiaOr is the suggestion that rsync grow a flag to cause a written file to be sparse?11:12
paulsher1oodhttp://listmaster.pepperfish.net/pipermail/baserock-dev-baserock.org/2014-September/007906.html11:13
persiaOh, heh, yes, that would be a bug :)11:13
* paulsher1ood is a bug magnet11:13
paulsher1oodso this deploy is just sitting there. i guess i need to rollback morph to before th sparse changes?11:15
richard_mawif the dd command on your Mac supports iflag=fullblock, then it's possible to fix the script, which would be better than rolling it back11:16
pedroalvarezpaulsher1ood: `dd --help`11:17
paulsher1oodno help on mac :) it has man, though11:17
richard_mawhttps://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dd.1.html11:18
richard_mawno mention of iflag11:18
paulsher1oodhttp://fpaste.org/133846/11:19
paulsher1oodrichard_maw beat me to it :)11:19
* pedroalvarez wonders what happens if the host is running windows11:20
richard_mawnot a lot11:20
ssam2presumably there's no ssh server running on Windows anyway, unless the user has Cygwin11:20
persiarichard_maw: Shouldn't `...conv=sparse...` do the right thing?11:20
pedroalvarezssam2: true :)11:21
richard_mawpaulsher1ood: in which case the simplest thing would be to either roll back, or try `git revert -m1 ef6a4743aaaada781685ed6988917f299dbcfcda`11:21
persiassam2: There are other ssh servers for windows, but most windows doesn't have the POSIX compat stuff loaded, so can be safely ignored.11:21
richard_mawpersia: coreutils dd doesn't have conv=sparse, and it doesn't help having to send a long sequence of 0s in the input11:22
persiarichard_maw: my install of coreutils contains a dd.1.gz that claims to have conv=sparse11:23
persiaIt's 8.2011:23
richard_mawah, I've got 8.2011:24
richard_maw8.13 I mean11:24
persiaYou're certainly not alone: I suspect that (or older) to be installed for many folk who don't upgrade often.11:25
richard_mawI'm on Debian Wheezy11:25
persiaWhich is a sensible place (current stable).  Ubuntu 12.04 is also 8.1311:28
richard_mawfixing rsync would be the most useful solution, but I have no idea what would be involved, and I don't have time to look at it11:28
richard_mawI just remember being terrified the last time I looked at rsync's codebase11:29
pedroalvarezrichard_maw: meh, we sent almost the same reply11:38
pedroalvarezrichard_maw: I like your "*64" suggestion.11:39
richard_mawI'm not sure I do, but I can't adequately express why.11:39
pedroalvarezrichard_maw: what about `uname -m` vs $MORPH_ARCH11:41
Kinnisonpaulsher1ood: thanks for the linux link11:43
paulsher1oodKinnison: yw11:44
paulsher1ooddoes it work?11:44
KinnisonI thought you'd tried it11:45
paulsher1oodnot on jetson11:45
* Kinnison is still trying to work out if 3.17 will work on the K111:45
paulsher1oodit works on x8611:45
KinnisonIt seems to have stuff targetted at K111:45
Kinnisonbut I'm unsure if enough of nvidia's stuff is in11:45
Kinnisonpaulsher1ood: btw, Codethink is cited on http://en.wikipedia.org/wiki/Tegra#Tegra_K111:47
KinnisonThere's no mention in mach-tegra on master of the GK20A family of SoCs11:47
Kinnisonso I'm guessing it's not merged up yet anyway11:47
Kinnisonwhich makes me super-sad11:48
franredpedroalvarez, richard_maw, thanks for the fast review, Im trying the "*64" and "$MORPH_ARCH" version of the case statement (this does not include aarch64 yet, tough)11:51
Kinnison*64 is probably not good enough11:52
Kinnisongiven it'll likely be aarch64le if we do it11:52
persiaIsn't there already ppc64?11:53
paulsher1oodKinnison: check state of play  in #tegra?11:54
Kinnisonpersia: Yes, we have ppc6411:55
KinnisonI'm just downloading the current tegra kernel source to see how it manifests in there11:56
franredKinnison, ok, then I will leave it as richard_maw's 1st suggestion x86_64|ppc64) and when aarch64 comes we can modify or I can add *64*) but I think this is very weak regular expression11:56
KinnisonIt's a shell case pattern, but yes :-)11:57
Kinnisonradiofree: Okay, I give up, what's your recommendation for Tegra kernels then?12:03
Kinnisonradiofree: because you have 3.15 and 3.16 branches, but nvidia seem to be shipping 3.1012:12
paulsher1oodKinnison: we should aim for mainline, with minimum patch set, i believe12:13
paulsher1oodradiofree had things working on 3.16 but compilations were unstable iirc.12:14
* paulsher1ood hopes things have improved in the intervening time12:15
Kinnisonpaulsher1ood: I am, of course, aiming at mainline, but I'm having difficulty tracking everything, I've found a git tree to poke, but if I get completely lost I'll probably go chat to #tegra12:15
KinnisonSpecifically I'm just poking through https://git.kernel.org/cgit/linux/kernel/git/tegra/linux.git/12:17
KinnisonHmm, the tegra 124 family might be the k112:26
KinnisonI hate marketing terms vs. engineering terms issues12:27
KinnisonOkay, I'mma try paulsher1ood's branch, with radiofree's kernel config12:30
Kinnisonlet's see how explodey this is12:30
*** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 258 seconds]12:31
*** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock12:43
* paulsher1ood crosses his fingers12:44
KinnisonToday I have very little brain12:46
Kinnisonit's running again with a defconfig and dtb which might actually exist :-)12:46
*** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 250 seconds]12:51
*** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock12:59
*** thecorconian [~thecorcon@eccvpn1.ford.com] has joined #baserock13:00
KinnisonDoes morph deploy not update git repos if it needs to?13:14
KinnisonBecause I had an odd issue13:14
paulsher1oodwhy would it need to?13:14
Kinnisonwhere a ref was on my trove, and distbuild could build with it, but it wasn't in my local git cache13:15
Kinnisonso when I ran deploy, it failed because the ref was "bad"13:15
SotKI'm pretty sure that deploy never updates git repos now13:15
ssam2that's a bug introduced when we made `morph deploy` never update git repos13:15
KinnisonRight, well that means I ran face-first into an obvious workflow issue13:15
paulsher1oodnor should it, imo. it should only deploy a built system?13:15
ssam2we should revert that change now that Morph doesn't blindly update every ref on each run13:15
ssam2*every repo13:15
Kinnisonpaulsher1ood: You're missing the point, *I* didn't build system system, distbuild did13:15
Kinnisonand I never had that SHA locally because I never needed it locally13:16
persiaWhy do you need it locally at deploy time?13:16
Kinnisonpersia: because morph needs to calculate the artifact IDs, and to do that it needs tree SHAs etc13:16
Kinnisonpersia: and if we have a repo locally, we use that rather than take network time to talk to the trove13:16
Kinnisonpersia: but we're missing a fallback to say "if the ref isn't in the cache, and I've not tried updating the cache, {update the cache and retry,use the remote git resolver}"13:17
persiaWhy does it need to calculate any artifact IDs other than that of the system being deployed?13:17
paulsher1oodi disagree, i think. morph shouldn't need to be calculating all the time. distbuild di the calcualtion already, can't morph deploy benefit from that?13:17
Kinnisonpaulsher1ood: that calculation happened on the distbuild controller13:18
paulsher1oodKinnison: i know13:18
Kinnisonpersia: erm, to calculate the system artifact needs all the strata artifact ids which needs all the chunk ids13:18
Kinnisonpersia: it's a heirarchy13:18
Kinnisonpaulsher1ood: My point is, without examining the graph to know if anything changed, deploy cannot know if it's the same artifact graph as distbuild built anyway13:18
paulsher1oodKinnison: the output of the calculation is auseful articatc - we could just publish/cache it?13:19
persiaNo, Kinnison is right here.13:19
paulsher1oodbah. typing13:19
paulsher1oodi still disagree.13:19
persiaThe alternative is to never build anything that isn't committed, at which point we can rely on the definitions tree ID to confirm things haven't changd.13:19
persiaBut that's not how morph works.13:19
ssam2we could potentially cache the cache key of each system morph in a given SHA1 of definitions13:20
Kinnisonpersia: *iff* all the refs happen to be SHA1s13:20
ssam2I think it'd be more trouble than it's worth, and calculating the build graph only takes seconds anyway13:20
Kinnisonwhich, in a tree where I have run 'morph edit' is current not the case13:20
persiaAh, right.13:20
paulsher1ooddistbuild only works on committed stuffs13:20
ssam2it'd be much faster if we batched requests to the remote git cache, too13:20
Kinnisonssam2: I believe we modified distbuild to batch some operations.  we could batch more13:20
Kinnisonspecifically the "are these artifacts built?" query is batchable I think13:21
ssam2Morph resolves refs one by one13:21
Kinnisonaye13:21
Kinnisonthose could usefully be batched13:21
paulsher1oodssam2: yes. but why do we do it every time?13:21
ssam2paulsher1ood: if it took a second, why wouldn't we?13:22
Kinnisonbut it'd require a reasonably large rework of the graph walker iirc13:22
paulsher1oodit takes minutes13:22
* Kinnison would much rather make it fast to calculate, than add another potentially out-of-date cache13:22
ssam2paulsherwood: Not for me. There may be some other performance issue on your system, or an existing performance problem is greatly exacerbated for you13:22
ssam2so we should fix that, not add extra code to cache stuff13:22
persiaThere are already so many caches that there is a significant performance impact maintaining cache consistency13:23
paulsher1ood:)13:38
Kinnisonpaulsher1ood: fyi that kernel and dtb built, I'm just waiting on my superslow laptop drive to deploy it :-)13:40
* ssam2 worries that the build instructions for the ARM Genivi baseline begin: "wget http://download.baserock.org/baserock/baserock-11-devel-system-armv7-versatile.img.gz"13:42
ssam2any thoughts on what we should change this to?13:43
ssam2on http://wiki.baserock.org/genivi/13:43
ssam2maybe '1. Buy a Jetson' ?13:43
paulsher1oodKinnison: i wonder whether the whole process would have been faster using a jetson  as devel, and upgrading self13:43
Kinnisonpaulsher1ood: possibly, but then I wouldn't have my comfortable dev environment13:44
* Kinnison is odd13:44
pedroalvarez"buy a jetson board"13:44
Kinnisonit'd be faster if my laptop had an SSD :-)13:44
* pedroalvarez is lagging13:44
*** genii [~quassel@ubuntu/member/genii] has joined #baserock13:49
straycatIs there an analogue of ATTACH_DISK for openstack deployments?14:06
richard_mawstraycat: no14:07
straycatOkay14:07
pedroalvarezstraycat: also, the openstack deployment only creates the image. To attach a volume (disk) is at instantiation time.14:08
straycatI was going to ask what might be involved in writing an analogue for ATTACH_DISK, but I guess for now it's a case of running the right commands when you instantiate the thing.14:10
pedroalvarezrichard_maw: Can I use then "if [ $DISTBUILD_GENERIC = True ]", or would you prefer to stick with  "if [ -n $DISTBUILD_GENERIC ]"14:10
pedroalvarez?14:10
pedroalvarezhm... Am I the only one who doesn't like "openstack-app" for the name of the stratum that only has gerrit at the moment?14:15
richard_mawpedroalvarez: so long as you quote $DISTBUILD_GENERIC in the code, I don't mind whether you compare it to True or or check whether it's set14:17
* persia thinks gerrit ought be in a "gerrit" stratum, unless there's some strong expectation to deploy systems containing gerrit *and* something else.14:18
petefothpersia: what an *odd* idea :)14:20
* franred though that we aim to add gerrit + zuul + turbo hipster + storyBoard... but maybe Im wrong14:21
franredpetefoth, persia, pedroalvarez ^^14:22
persiafranred: Wouldn't one typically deploy those as separate systems, rather than trying to stuff it all in one place?14:22
paulsher1oodfranred: persia is probably right. he often is :)14:23
persia"often" is an operative word here: it's typically a good idea to validate my claims, rather than just believing them as gospel :)14:23
pedroalvarezjust to validate persia's claims: I raised the point because I had the same opinion14:25
petefothMy dfeeling is that these apps should be in sepearate chunks whcih can b combined in differet ways into different systems14:25
petefothSo our CI Pipeline might consist of a number of co-operating systems. Others might mak a systme where all of the tools are in the same system14:26
pedroalvarezpetefoth: did you mean "separate strata"? </google>14:26
petefothpedroalvarez: I almost certainly did :)14:26
* petefoth is not yet fluent in the Baserock language14:26
franredok, I will change the name of the stratum from openstack-app to gerrit14:27
paulsher1oodcan we set a standard for naming of stratum 'foo and its dependencies' - i for one would rather not have it called just 'foo'14:29
franredpaulsher1ood, this stratum only depends on itself - the system which includes this stratum is called gerrit_x86_64.morph and its cluster openstack-gerrit.morph14:31
persia" and its dependencies" is long though, and some strata may sensibly only contain a single chunk.14:31
paulsher1oodi wasn't actually suggesting that as a name, just noting the use-case. what about 'gerrit-stuff' as a specific example14:32
persia"-stuff" is less bad.14:32
Kinnisonrichard_maw: can we stop morph doing the disk space check on distbuild operations, it's very annoying :-)14:33
paulsher1oodgerrit-set ?14:33
franredwhat stuff means in this case?14:33
persia"gerrit" is actually bad, because it causes name collision (same name for chunk and stratum).14:33
paulsher1oodthat's why i raised this14:33
* paulsher1ood is obvbiously not making much sense today14:33
franredchunck is called gerrit-installation-binaries14:33
persia"patch-manager" might also work, where gerrit is one chunk needed, but others may be useful.14:33
franredso no collision in this case14:34
persiafranred: Yes, but the chunk should be called "gerrit" once it is being built from source.14:34
richard_mawKinnison: I guess14:34
franredpersia, code-review-manager?14:35
persiafranred: That also works, although less terse.14:36
* persia fails at grammar14:36
persias/terse/tersely/14:36
franredpersia, ok, patch-manager then14:37
franredpedroalvarez, ^^ any thoughs against patch-manager?14:38
persiaeither is fine.  It's not like it's something that gets typed often, and it tab-completes now14:38
pedroalvarezfranred: better :)14:39
rjekMinix is now cross-buildable from Linux.  Here's a Debian package definition for it: https://build.opensuse.org/package/show/home:beng-nl/Minix314:39
rjekIt could be an interesting thing to build with Baserock instead, given its aim for high-reliability embedded systems.14:40
rjek(It also now has a working ARM port)14:41
richard_mawrjek: ARM for which hardware?14:45
rjekA8 and Beagle* boards atm.14:45
petefothI'm concerned that the name 'patch-manager' is too generic. 'patch-manager-gerrit' makes clear this is a specific patch manager14:46
ssam2paulsherwood, persia: interesting point about stratum names14:49
ssam2petefoth: I think you've just described the systemd unit naming policy, too :)14:49
petefothssam2: is that a good thing?14:50
ssam2I think it's sensible, yeah14:50
ssam2https://fedoraproject.org/wiki/Packaging:Systemd#Naming14:50
ssam2"Unit files should be named after the software implementation that they support as opposed to the generic type of software. So, a good name would be apache-httpd.service and bad names would be httpd.service or apache.service as there are multiple httpd implementations and multiple projects produced by the apache foundation."14:50
persiaIn classic distros, one only needs a few systems, expecting users to customise to needs.  For a Baserock distro, one ends up wanting a system for every intended use, which can lead to some confusion when also wanting somewhat generic strata.14:51
persiaEspecially in the case of application-centric systems: as an example, what might be sensible strata/system names for a nagios controller?14:52
*** ct-dutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Quit]14:54
*** dutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock14:54
*** tiagogomes [~tiagogome@213.15.255.100] has quit [Quit: Leaving]15:48
*** dutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Quit]16:15
straycatGood news, you can format ephemeral disks when you create an instance16:35
straycatBad news, you can't specify a volume label16:36
straycatGood news, you can attach a volume when you bring up an instance instead16:36
straycatBad news, gbo is 500 again16:36
straycatDoes anyone have any more of an idea what's up with it, I noticed on the weekend that one of the lighttpd services was using around 5GB of memory.16:37
straycatWe can just restart the process again, but that doesn't help us much in the long run.16:39
*** mwilliams_ct [~mikewilli@access.ducie-dc1.codethink.co.uk] has joined #baserock16:44
*** tpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock16:48
straycatFor the sake of the docs it's probably simpler not to use labels.16:56
pedroalvarezfranred: wrt gerrit, my previous +1 is still valid16:59
persiaLabels are useful in a libvirt environment where one is sharing volumes between VMs, but not so much in other contexts16:59
richard_mawis it safe to have multiple VMs use the same volume?16:59
franredpedroalvarez, cheers, I will merge into master16:59
pedroalvarezrichard_maw: I think is not allowed17:00
persiarichard_maw: Yes, if they aren't running at the same time.17:01
persiaIf you need consistency between two hosts, you need another layer of abstraction.17:02
persiaErr, two concurrently running hosts, that is.17:02
richard_mawprobably in the form of a networked filesystem with good locking semantics17:02
persiaThe other common solution is the object store abstraction, without traditional FS semantics17:03
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]17:04
pedroalvarezurgh... I wonder if we broke this with the automake upgrade;; http://pastebin.com/w5z3ApM417:12
persiaDoes it build correctly if you reset automake to the old ref?17:13
radiofreewon't that require a rebuild of most things?17:14
pedroalvarezno if i have all the artifacts cached :)17:14
*** pdar [~patrickda@access.ducie-dc1.codethink.co.uk] has joined #baserock17:15
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]17:22
*** flatmush [~flatmush@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 272 seconds]17:29
*** flatmush [~flatmush@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock17:32
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]17:35
*** violeta [~violeta@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 255 seconds]17:43
pedroalvarezreverting automake, libevent built19:00
pedroalvarezreverting the automake upgrade19:00
pedroalvarezhm.. upstream has fixed the problem but there are not stable releases with that fix on it19:09
pedroalvarezthis is the fix: http://git.baserock.org/cgi-bin/cgit.cgi/delta/libevent.git/commit/?id=a55514eeed96b9bf9a16fbed1a709dfcce5a608019:10
tlsacan you cherry pick it?19:12
pedroalvarezgiven that is the makefile of the 'test' folder I was wondering if I can disable them19:12
tlsaah19:13
pedroalvarezso I don't have to create a branch for it19:13
pedroalvareztlsa: but yes, that is also a good idea19:14
pedroalvareztlsa: git cherry-pick worked. I'm going to opt for that solution.19:23
tlsacool19:23
paulsher1oodgbo is down...20:24
straycatagain?20:31
straycat:/20:31
straycatLooks like the same problem20:32
*** benbrown_ [~benbrown@access.ducie-dc1.codethink.co.uk] has quit [Ping timeout: 245 seconds]20:38
*** benbrown_ [~benbrown@access.ducie-dc1.codethink.co.uk] has joined #baserock20:39
*** bjdooks [~ben@trinity.fluff.org] has quit [Ping timeout: 250 seconds]22:12
*** thecorconian [~thecorcon@eccvpn1.ford.com] has quit []23:19
*** Kinnison [~dsilvers@gateway/shell/pepperfish/x-pgoajfzccsllvrvs] has quit [Ping timeout: 260 seconds]23:19
*** Kinnison [~dsilvers@gateway/shell/pepperfish/x-ytsfmzvonwqarpnd] has joined #baserock23:20

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!