IRC logs for #baserock for Wednesday, 2015-05-20

paulsherwoodrdale: http://paste.baserock.org/iyivunonop04:25
paulsherwoodweird04:25
*** zoli__ has joined #baserock05:39
*** zoli__ has quit IRC06:26
*** zoli__ has joined #baserock06:54
*** a1exhughe5 has joined #baserock07:00
*** mike has joined #baserock07:03
*** mike is now known as Guest7523607:04
*** Albert has joined #baserock07:38
*** gary_perkins has joined #baserock07:48
*** franred has joined #baserock07:56
*** jonathanmaw has joined #baserock07:59
*** bashrc has joined #baserock08:03
*** mariaderidder has joined #baserock08:34
wdutchbaserock-chroot refers to the directory where the chroot lives in /opt/baserock/chroot as 'name' in some places and 'tag' in others. I'd like to sort this out and use the same term everywhere. I think refering to the 'name' of a chroot makes the most sense, what do people think?08:40
*** Guest75236 has quit IRC08:45
radiofreecan you also fix it so you don't have to install in /opt?08:47
DavePageradiofree: You'd like to opt out of that?08:47
* wdutch reads the makefile08:48
radiofreeDavePage: http://i.imgur.com/DUyiCbE.gif08:49
wdutch/opt is hardcoded, this needs fixing08:49
pedroalvarezguys, we have too many branches in definitions.git. I've just cleaned those that were under my namespace, can others do the same and clean the repo a bit today?08:54
pedroalvarezIt's really relaxing, I promise :)08:55
tlsapedroalvarez: on git.b.o or on gerrit, or both?08:58
pedroalvarezg.b.o08:59
pedroalvarezhttp://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/refs/heads08:59
pedroalvareztlsa: you only have 2 :) well done08:59
*** Guest75236 has joined #baserock08:59
pedroalvareztlsa: well, you have some more under mikedrake and michaeldrake09:00
tlsapedroalvarez: 2 under tlsa, but a whole load under michaeldrake09:00
tlsaI'll attend to them later09:00
pedroalvareztlsa: note that you can keep whatever you want09:00
tlsamm09:00
pedroalvarezthis is more like: remove the branches that were merged09:01
tlsayep09:01
* pedroalvarez just removed 30 or so09:01
*** ssam2 has joined #baserock09:02
*** ChanServ sets mode: +v ssam209:02
SotKI'm seeing an error from Babel when I try to get a list of the running jobs from Zuul: http://paste.baserock.org/calukuluza09:08
SotKthe advice it gives doesn't work, as that tries to download a zip file from somewhere09:09
SotKdoes anyone have a suggestion on the neatest way to fix this?09:09
pedroalvarezhm..09:10
pedroalvarezpreviously we have done:09:11
pedroalvarez- Run the command, and commit&push the result to a branch09:11
pedroalvarez- Build from tarball if the tarball has those data files09:11
*** edcragg has joined #baserock09:12
*** Krin has joined #baserock09:12
ssam2SotK: yeah, it depends on the nature of the .zip file09:12
ssam2if it's something that rarely changes, it'd be ok in my opinion to commit it on a branch in the babel.git repo09:12
ssam2if it changes a lot, and is generated somehow, it's probably worth the effort to find out how it's meant to be generated09:13
jmacsDoes either morph or cache.baserock.org keep statistics on cache hits & misses?09:16
radiofreefranred looks like the worst offender09:17
* SotK investigates further09:17
radiofreeor maybe javier09:17
franredradiofree, me? ahhh regarding the number of branches in definitions?09:17
radiofreeyep :)09:18
* SotK also has many, I'll clean up later09:19
* richard_maw cleaned up all but his sshfs branch, which he'd like to revive some day09:20
ssam2jmacs: neither, I think09:21
ssam2jmacs: there may be something going on involving 'atime' that i'm forgetting about09:21
jmacsssam2: OK, I'll think about adding some then!09:21
ssam2jmacs: `morph gc` has some kind of heuristic for detecting what chunks are least used, that's about it09:22
ssam2jmacs: nice!09:22
pedroalvarezjmacs: cache.baserock.org has a log from were we can extrac that info I believe09:22
jmacsJust the httpd logs would do the job, I think - but don't worry about it for now09:25
pedroalvarezok, let me know if you want to take a look at those logs09:27
*** pacon has joined #baserock09:47
wdutchare branches always necessary for gerrit? I only want to change one word in an echo command09:48
jmacsfranred: Java is mentioned on the "Doing stuff with Baserock" page. I assume you mean OpenJDK? Any particular version?09:48
ssam2jmacs: I don't see franred today. It was actually me that added him to that page as he mentioned it in the past ;)09:49
richard_mawAIUI a complicated circular dependency bootstrapping problem prevented us having a proper java bootstrap09:50
ssam2jmacs: the hardest part is bootstrapping it, as all Java implementations seem to be implemented in Java, indeed.09:50
richard_mawbut they're supposed to have fixed that with the latest major version09:50
ssam2jmacs: is OpenJDK 8 the "best" available Java? I don't know much about it09:51
SotKssam2: I pushed my mason branch: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/log/?h=baserock/adamcoldrick/mason-201509:53
jmacsssam2: That's what I'm looking at now - the build instructions for OpenJDK seem to require the Oracle commercial JDK to bootstrap it.09:54
franredjmacs, I was having a look time ago, and as ssam2 said I was looking at OpenJDK 8 but when I looked at it I think it still required some java binaries09:54
ssam2jmacs: right.09:55
richard_mawKinnison did a lot of research into this, perhaps you could ask him for more details?09:55
ssam2jmacs: which of course is possible, but if we have to have the JDK binary available in order to bootstrap a Baserock system, it kind of defeats the point a bit for me09:55
ssam2jmacs: since one can also just download the JDK binary and use that directly, as we do now for https://gerrit.baserock.org/09:55
ssam2shall I create a storyboard story with this info?09:56
*** flatmush has quit IRC09:56
jmacsOh, storyboard, forgot about that09:56
ssam2a wiki page is fine if you prefer, just something that can be linked to from the 'doing stuff with baserock' page so this info isn't lost09:57
*** flatmush has joined #baserock09:57
jmacsYep09:58
*** robtaylo1 is now known as robtaylor10:04
* straycat will delete everything under baserock/richardipsum10:09
pedroalvarezdoesn't have to be everything, just whatever has been merged10:10
pedroalvarezthanks guys! It's getting cleaner :)10:10
ssam2SotK: I've made a bunch of changes to https://storyboard.baserock.org/#!/story/8 ("continuous build and test")10:14
pedroalvarezI wonder how would be better a patch to upgrade all the components needed for openstack kilo10:15
* straycat does the same for morph while he's there10:16
pedroalvarezgreat thanks!10:16
ssam2SotK: i've added it to http://wiki.baserock.org/doing-stuff-with-baserock/ as well10:16
SotKssam2: thanks, looks good10:17
pedroalvarezwould be acceptable to upgrade 30 chunks in one comit?10:17
richard_mawit would be ok IMO10:18
pedroalvarezright10:18
pedroalvarezI think I'd prefer that rather than 30 commits upgrading 1 chunk10:19
* pedroalvarez continues squashing10:19
* SotK would definitely prefer 1 commit to 30 commits10:22
* SotK is reminded of duck sized horses vs horse sized ducks10:22
* richard_maw now has the duck song stuck in his head10:23
straycatwhich duck song?10:23
Kinnisonbetter than the llama song10:23
*** ssam2 has left #baserock10:24
richard_mawstraycat: https://www.youtube.com/watch?v=MtN1YnoL46Q10:27
mwilliams_cthow do people even find videos like that?10:28
Zarathrough you?10:29
straycatoh i've seen that one before actually10:30
*** ssam2 has joined #baserock10:30
*** ChanServ sets mode: +v ssam210:30
radiofreeis it safe to delete factory?10:31
radiofreei'll still be able to do upgrades right?10:31
* Kinnison is actually unsure -- worth checking the upgrade path to see if you can specify a basis version?10:32
richard_mawI think it uses the current version these days, rather than the factory version as the basis10:32
pedroalvarezradiofree: I removed factory yesterday and upgraded the system later10:32
radiofreethanks10:32
* radiofree moves his / to the ssd to avoid this in the future10:33
ssam2i've created an initial story for 'firehose' as well: https://storyboard.baserock.org/#!/story/4410:35
mwilliams_ctchatting to jjardon yesterday, I believe he was working on a write extension for jffs2. he's away for a few days, so I thought i'd have a look at where he was up to. Does anybody know where he's likely to have pushed any work he's done?10:36
ssam2mwilliams_ct: no idea, but you could look through the branches with 'jjardon' in their name that are in definitions.git and morph.git10:39
mwilliams_cton g.b.o?10:39
ssam2I know that he uses storyboard so that might give you a clue (but it doesn't track branch names unless you post them in a comment, so maybe it won't be helpful)10:39
ssam2mwilliams_ct: yes, probably10:39
mwilliams_ctStoryboard is a good idea too, thanks ssam2 :)10:40
jmacsKinnison: Did you look into OpenJDK? Were there any results I could add to the baserock wiki?10:41
Kinnisonjmacs: I started to try and work out how to bootstrap OpenJDK10:41
Kinnisonjmacs: there's approaches involving eclipse's compiler and some other stuff, but bootstrapping from zero java support I hadn't gotten to10:42
Kinnisonjmacs: there are guides out there, but I'd not had enough time10:45
jmacsIndeed, I'm sure I'll get to them10:46
KinnisonSadly, similarly to Haskell, they expect you to have a previous JDK10:46
richard_mawwhat was it you needed to do to see more than 10 stories on storyboard?10:50
Kinnisonssam2: Any chance you can get the proper SSL certificate chain onto storyboard?10:51
richard_mawoh, page size preferences10:51
Kinnisonssam2: Also, isn't the last task on https://storyboard.baserock.org/#!/story/38 done?  (isolating mason) ?10:54
ssam2Kinnison: I want to redo storyboard completely, the current one is a hack10:58
ssam2is the wrong SSL certificate causing you a problem?10:58
ssam2good spot on story #38, thanks10:58
Kinnisonssam2: it may be related to the auto-errors10:59
Kinnisonin story 210:59
ssam2richard_maw: on http://storyboard.openstack.org there's a setting above the list of stories for how many you want to see, so I think updating storyboard would help there10:59
ssam2Kinnison: good point10:59
Kinnisonssam2: also it's annoying :)10:59
Kinnisonbut a re-do seems a sane approach11:00
Kinnisonclean deployment ftw11:00
*** pacon has quit IRC11:24
*** Albert has quit IRC11:24
*** pacon has joined #baserock11:25
SotKI'm thinking about how we would want to set up a Zuul Mason against our Gerrit. Does it make sense to have a few different criteria it votes on (for definitions at least) rather than just "Verified"?11:26
SotKFor example "build" and "tests"?11:26
SotKI'm mostly thinking of trying to do that because getting feedback from a full test cycle may take a while, eg. if the devel system has to build itself from scratch with no cache.11:27
ssam2I don't think it'll be possible to run a comprehensive set of tests for each branch in Gerrit11:28
ssam2I think we should aim to provide quick quick feedback on branches in Gerrit, and then do a thorough test of 'master' once per week or so11:28
Zarahi, I'm looking at section 4a here: http://wiki.baserock.org/guides/how-to-cross-bootstrap/ . I'm wondering: do I run the first two steps inside a chroot? I'm also not sure about what the second step entails, but I guess I'll ask more about that when I get to it.11:29
pedroalvarezSotK, ssam2: indeed, quick feedback sounds ok to me11:29
ssam2Sotk: with that in mind, I think just 'verified' would be OK in Gerrit. Although providing extra feedback in comments is important too, e.g. 'I just started building this, should be done in 5 minutes'11:30
SotKssam2, pedroalvarez: so just checking that stuff in clusters/ci.morph still build then?11:30
pedroalvarezZara: looking at it11:30
SotKthat makes sense11:30
ssam2yeah, that sort of thing11:30
pedroalvarezSotK: we can figure out better ways later11:31
ssam2and run `morph certify`11:31
pedroalvarezmaybe in the future, a quick first feedback would be to build only the strata that has changed11:31
pedroalvarezbut for now, only build ci.morph would be brillian11:32
pedroalvarezt11:32
pedroalvarezZara: 4 is confusing11:32
pedroalvarezZara: all of it is confusing, although maybe not that much given that you managed to get there11:33
ssam2thanks for cleaning up stale branches in definitions.git everyone!!!!11:33
pedroalvarez:D ;D11:33
pedroalvarezZara: I think you can skip 4a, 4b and go to 4c11:34
franredssam2, pedroalvarez, do we have to do it in the gerrit repo too?11:34
pedroalvarezI don't think so11:35
pedroalvarezwell, I'm not sure11:35
franredpedroalvarez, I've checked and all the branches that were in gbo are in gerrit11:35
franredunless gerrit sync branches...we will have them stalled in gerrit mirror11:36
Zarapedroalvarez: I thought I needed that step because, when I tested with 'linux-user-chroot / /bin/sh' I got an odd message that looked like an error: 'mount(/, MS_PRIVATE | MS_REC): Invalid argument'11:37
pedroalvarezaha11:37
pedroalvarezZara: you are right11:38
Zara:D11:38
ssam2franred: it's up to lorry / lorry-controller to delete branches in gerrit that have been deleted in git.baserock.org11:39
ssam2franred: I'm not sure whether it will or not, good point11:40
franredssam2, I imagine they will stay, don't we have a policy/rule in lorry/lorry-controller for not removing branches? (if we treat our repos as the normal repos this shouldn't happen)11:43
ssam2franred: I can't find such a rule documented anywhere11:44
ssam2not in http://wiki.baserock.org/Trove/reference/ or lorry's README anyway11:44
ssam2so I'm looking through the code so see what it does11:45
ssam2seems that 'lorry' will run `git remote update --prune` in the repo in its working area, so branches get deleted in that repo11:45
pedroalvarezZara: I've modified 4a a bit11:46
Zaraah, thanks. :)11:47
ssam2franred: but then to update the actual repo in gerrit, it just does 'git push ssh://... +refs/heads/* +refs/tags/*'11:48
edcraggthis kernel seems to be building with '-dirty' :(11:48
ssam2franred: which I think won't remove any branches that are no longer present in lorry's copy11:48
ssam2edcragg: I think nowster knows why that happens11:48
nowsteryes11:49
nowsterI do.11:49
ssam2franred: so I guess we need a patch to 'lorry' to make it handle the case of deleting branches no longer present upstream11:49
edcraggnowster: ssam2: i've read about uncommitted files in git, but not sure how that would happen in baserock build11:49
nowsterAdd a "git status" above your "make xxxx_defconfig"11:50
nowsterThe git tree is in an inconsistent state at that point.11:50
nowsterIt's a morph bug, I suspect.11:51
franredssam2, do we want to delete them or keep them? (Just thinking in the case we are building from a branch)11:51
edcraggnowster: ok, very nice :) odd, this the first time i've seen this11:51
ssam2franred: delete them11:54
ssam2franred: they wouldn't have been deleted in git.baserock.org if they were useful11:54
franredssam2, yeah, I was thinking in other repos (git, linux...etch)11:55
franreds/etch/etc/11:55
ssam2franred: ah. that goes back to the problem we've discussed a bunch of times of consuming upstream commits11:56
ssam2franred: i'd be happy if we deleted branches whenever upstream deleted them, and used `morph anchor` to ensure this didn't cause source code for released images of baserock to be lost... but others may disagree11:56
Zaramost of the shiny new 4a instructions work fine for me, but I get an odd message when I run 'ldconfig' (this also cropped up several times when I was running the native-bootstrap script; not sure how significant it is): http://paste.baserock.org/ezogixocak11:57
*** pacon has quit IRC11:59
franredssam2, yeah, that sounds sensible to me too11:59
*** pacon has joined #baserock12:00
* SotK tries to figure out how Zuul's timer trigger works whilst waiting for his Mason test to finish12:02
*** Albert has joined #baserock12:08
pedroalvarezZara: first result in google: https://gcc.gnu.org/ml/gcc-help/2014-08/msg00053.html12:09
pedroalvarezZara: looks like they might be just warnings12:09
SotKZara: I'm pretty sure I've seen that every time I've run ldconfig in Baserock12:09
edcraggssam2: nowster: the built zImage seems to have a modification date of Aug 24, 1991 not sure if that's a sign of something going a bit squiffy (the system's clock is correct)12:10
Zarahehe :) I wanted to check in case baserock was unusual, glad I can ignore it12:11
radiofreedoes anyone know why journalctl fails to work on genivi images?12:11
radiofreeif you do systemctl restart systemd-journald it'll work for that session, however when you restart it's gone again12:11
nowsteredcragg: same as all of mine!12:12
SotKedcragg: I think the mtime of everything in a chunk gets set to that date during the build, but I may be wrong12:12
nowsterAug 25  1991 vmlinux-4.0.0-rc7-be12:12
ssam2radiofree: could be https://storyboard.baserock.org/#!/story/43 perhaps12:12
pedroalvarezradiofree: that has been failing for months12:14
SotKnowster, edcragg: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/tree/morphlib/bins.py#n9012:14
radiofreeoh12:14
pedroalvarezradiofree: looks like the systemd service that waits untils /var is mounted to flush all the early logs, depends on /var being mounted, but looks like that is failing in baserock12:15
radiofreethis makes it hard to debug systemd services12:15
edcraggSotK: ok, i guess that's not it, then12:16
* straycat thinks ybd could possibly use a little arg validation12:17
straycatmaybe using argparse?12:17
paulsherwoodstraycat: you're right, i never got round to that12:19
paulsherwoodbut in general i'm aiming for no commandline args... i want all input from a yaml file12:19
rjekHow will it know which one?12:20
rjekOne of the problems I'm having with ansible is that /too/ much can only be specified via a yaml file12:20
paulsherwoodrjek: ok, only one arg.12:20
rjekMaking it difficult to do ad-hoc experiments.12:20
paulsherwoodrjek: currently ybd has two. the second is $arch. i intend that it and all the other settings it needs come from yaml12:21
paulsherwood(and/or env variables)12:22
pedroalvarezpaulsherwood: $arch should come from your system12:22
paulsherwoodpedroalvarez: it can't12:22
pedroalvarezpaulsherwood: and in the yaml maybe we should add the valid archs for that yaml12:22
paulsherwoodmy system can build various $arch12:22
pedroalvarezpaulsherwood: I mean, I think we should aim for non-specific-arch systems12:23
paulsherwoodagreed. but we still need to specify a target arch somehow12:23
pedroalvarezpaulsherwood: given that you can't crosscompile, the target arch is your host arch12:24
paulsherwoodnot necessarily12:24
paulsherwoodif i run ybd in a cluster, and all the artifacts i need exist, ybd should pull all the artifacts, integrate them, deploy them12:25
pedroalvarezI see12:25
* paulsherwood has done *a lot* of thinking about this12:25
paulsherwoodalso, ybd can be targeted to build a 'chunk' - which won't normally be arch specific, so we need to specify it when invoking ybd12:26
pedroalvarezpaulsherwood: then, I think that to build something you don't need the $arch, to deploy something you might need it12:27
pedroalvarezs/might //12:27
paulsherwood$arch is part of the cache_key, so i think i do need it12:27
pedroalvarezthis is a really big architecture discussion12:28
paulsherwoodyup.12:28
paulsherwoodactually now you mention it, might be better to have arch explicitly out of the cache_key12:29
pedroalvarezwe need to start thinking about this more and more12:29
pedroalvarezmm12:29
paulsherwoodwho is 'we'? i can't think about it much more than i already do :)12:29
pedroalvarezit should go in, i think12:30
ssam2how could 'arch' be out of the cache key? a build of GCC for ARM is totally different to a build of GCC for x8612:30
pedroalvarezwe = developers and users of Baserock12:30
ssam2unless we had a 'cache per architecture', but I think that'd get confusing when sharing artifacts12:30
paulsherwoodssam2: systemd@e7f13e3f05fcfc054bc90a340f640d4083c873fcf5a9d646ea0824ee829ca39b.x86_6412:30
richard_maweww12:30
ssam2paulsherwood: oh, that seems reasonable12:31
paulsherwoodybd currenly does systemd@e7f13e3f05fcfc054bc90a340f640d4083c873fcf5a9d646ea0824ee829ca39b12:31
ssam2richard_maw: are you suggesting that such a cache ID is worse than what we have in Morph now? to me, it looks a bit clearer12:31
nowstermips32 and mips64 might be in the same cache...12:32
paulsherwoodnowster: how?12:32
richard_mawssam2: IMO we shouldn't have any identifying information in those file names12:32
nowsterbecause you can build the same on same machine without rebooting12:32
Zaraokay, I'm now on the 'move the bootstrap filesystem to the top level of another disk' instruction. am confused; I originally extracted and built this on the disk where I thought there would be room for it, and now I'm not sure where it's suitable to move it to. I have a jetson attached to an ssd; the bootstrap filesystem is on the ssd.12:33
Kinnisonx86_64 and armv7blh can be on the same system in the same cache12:33
* paulsherwood strongly disagrees with richard_maw's comment since paulsherwood is a human being, and needs to debug cache_keys from time to time12:33
Kinnisonbecause you can deploy any architecture from any other12:33
paulsherwoodah, i misunderstood. i thought nowster meant that mips32 and mips64 might be in the same cache *artifact*12:34
* SotK isn't a fan of the @ but otherwise agrees that it is clearer12:34
pedroalvarezI assume that richard_maw  means that we shouldnt need to look at the cache artifacts ever12:34
paulsherwoodi'm not a fan of the @ either, am thinking of going for '.'12:34
richard_mawpedroalvarez: either that, or if we need to work out what the artifact is, we look at the metadata inside it12:35
paulsherwood-112:35
SotKpaulsherwood: I would like '.'12:35
paulsherwoodanyway, this is the cache_key for ybd. i get to dictate it. morph can do what it likes :)12:36
pedroalvarezhaha12:36
SotKthe first iteration of the OSTree patch had the cached artifacts ref be just the cache key, and I thought it was really annoying12:36
richard_mawif we don't have tooling to inspect the cache then every time someone needs to inspect it, they hand-roll their own with varying degrees of success, often with false positives or negatives12:38
ssam2richard_maw: I don't think ugly and unhelpful cache keys is a sensible way of encouraging the development of tooling to inspect the cache, though12:38
paulsherwoodrichard_maw: i still disagree12:38
richard_mawI still think the cache should be an implementation detail, hence we shouldn't need to inspect it unless something has gone wrong, at which point it doesn't need to look pretty, we just need a way to get the information we need out, and the file name is a poor place for structured data.12:40
paulsherwoodnoted  :)12:40
straycati think ybd could use morph-cache-server regardless, as far as i can tell the cache-server doesn't place any requirements on the format of the cache key12:45
paulsherwoodreally? that would be *awesome*12:53
paulsherwoodhow hard is it to spin up a local morph-cache-server? then i could try my 'crazytown' distbuild idea? :)12:54
*** pacon has quit IRC12:56
*** zoli__ has quit IRC12:57
* SotK wonders what "'crazytown' distbuild" is12:58
SotKpaulsherwood: morph's test suite sets up a couple, so I think its probably fairly easy12:59
paulsherwoodSotK: thanks12:59
SotKpaulsherwood: specifically the distbuild yarn12:59
paulsherwood"'crazytown' distbuild": run the same build on several instances of ybd at once, where they all share the same cache server13:00
paulsherwoodthat's it :)13:00
* paulsherwood waits for the flood of reasons why it won't work13:00
pedroalvarezit's software13:01
rjek:)13:01
paulsherwoodheh13:01
rjekIt sounds racy, anyway13:01
* paulsherwood considers tweeting 'The more i think about code, the less i like it'13:01
paulsherwoodrjek: yup. don't care13:01
persiaIf we trust the reproducibility story, racy doesn't matter.13:02
rjekI mean, in that, you may end up with things being built multiple times because it wasn't in the cache before they started13:02
paulsherwoodyup13:02
rjekAnd we don't need any help to make builds slower.13:02
persiaThe part I don't like is the non-elastic nature of it, along with no obvious means to schedule jobs onto a set of workers (whether a fixed set or an elastic set)13:03
rjekELB!13:03
paulsherwoodpersia: i think that's just 'cloud' once i prove the concept13:03
persiaload-balancing is a decadent subset of scheduling.13:04
paulsherwoodheh13:04
persiapaulsherwood: At this level, 'cloud' has no concrete meaning.13:04
Kinnisonconcrete clouds wouldn't float very well in air13:05
Kinnisonperhaps their seafaring clouds13:05
Kinnisontheir? their?!13:05
rjekthey're13:05
Kinnisonthey're!13:05
Kinnisondamnit13:05
paulsherwoodpersia: know. but no point in thinking about how to fix the problem you've described if the basic premise doesn't even work, so i need to try that first13:05
persiaRight, which is why one needs to unfuzzy the details before actually deploying to clouds.13:05
persiapaulsherwood: Fair, although I don't see any reason the premise doesn't work excepting the race condition (where all the nodes simultaneously build gcc, because it takes a long time)13:06
SotKpersia: +113:06
rjekAnd gcc is one of the first things to get built, so it's going to be a massive slow down13:06
SotKI see no reason why sharing a cache server would be a problem?13:07
persiaIt doesn't slow final delivery much in clock time: it just wastes resources during the duplicate build.13:07
* persia has been imagining a shared cache server for this application.13:07
paulsherwoodpersia: are you imagining something other than morph-cache-server?13:09
* rjek does not want a portal into persia's imagination.13:09
persiaIn my imagination, any file server would work.  Could just use Swift or Ceph as an object store, could use WebDAV, etc.13:09
rjekYou'd want locks, though?  Or would you do something like Maildir where only rename needs to be atomic?13:10
persiaSomething like Maildir, excepting that the name ought be determined by the cache key, so there is never a need to rename.13:10
paulsherwoodatomic rename would be my favourite13:10
persia(excepting the publication moment)13:11
paulsherwood+113:11
rjekpersia: You need the hostname of the builder who put it there in the filename, no?13:11
rjekAh, right13:11
paulsherwoodnope13:11
persiaThat said, Ceph or Swift handle this entirely differently, so atomic rename shouldn't be a requirement, so much as atomic availability.13:11
rjekotherwise two builders building the same thing would end up trying to create the same cache key entry?13:11
paulsherwoodone fails,  so what?13:12
persiaThat's fine.  Make the cache server prevent overwrites, and make the builder gracefully fail if it cannot write to cache because of a cache key conflict.13:12
rjekI suppose you just call creat() and have it fail if the file already exists13:12
paulsherwood:)13:12
persiapaulsherwood: Failure management needs to be specific: you want to retry in the case where the failure was for something other than overwrite rejection.13:12
paulsherwooddetails, details :)13:13
persiaThis is where the devil lives.13:13
rjekActually, does creat(..., O_EXCL) even work on network file systems?13:13
persiarjek: Depends on the NFS in question.13:13
rjek"On NFS, O_EXCL is only supported when using NFSv3 or later on kernel 2.6 or later.  In NFS environments where O_EXCL support is not provided, programs that rely on it for performing locking tasks will  contain  a  race condition"13:13
rjekRight13:13
Kinnisonrjek: NFS before that supports exclusive mkdir()13:14
Kinnisonrjek: hence most NFS-safe software older than perhaps 10-12 years uses mkdir for its lock generation13:14
rjeknod13:15
tlsapedroalvarez: removed a bunch of my old personal branches13:16
pedroalvareztlsa: ta!13:17
*** zoli__ has joined #baserock13:22
*** sambishop has quit IRC13:34
*** sambishop has joined #baserock13:34
* SotK fails to find a way to customise the message Zuul's Gerrit reporter sends when starting through layout.yaml :(13:37
persiaSotK ask upstream (#openstack-infra)13:41
persiaMost of them will be at conference now, so UTC-7.13:42
straycatahh so morph supports cpan, wonderful13:52
rjekSpamAssassin time!13:53
SotKrichard_maw: thanks for the reviews/merge!14:14
jmacsOpenJDK8 makes no sense at all - I've spent several hours now just trying to build it on Ubuntu, let alone baserock14:26
ssam2wow14:28
*** petefoth has quit IRC15:02
Zara(am still confused about step 2 in 4a here: http://wiki.baserock.org/guides/how-to-cross-bootstrap/ . details in backscroll at 13:33, not much has changed. might be worth mentioning that, as far as I can tell, my bootstrap filesystem ended up in the same directory as the extracted tarball.)15:05
Zara(jic that necessitates an extra step)15:06
* paulsherwood has updated ybd readme https://github.com/devcurmudgeon/ybd15:07
*** Guest75236 has quit IRC15:11
straycat"Currently YBD generates a lot of log output, which hopefully helps to explain what is happening." might make sense to switch to python's logging module and then just change the log level, rather than implementing that ourselves15:12
pedroalvarezZara: that step means "move everything (bootstrap filesystem, with all the things that the tarball had)  to another disk (usb stick, hard drive..)"15:12
SotKZara: That page suggests that doing one of either the bind mount or the "move everything" should make it work to me. Is that not the case? (disclaimer: I've never done a cross-bootstrap)15:13
ZaraSotK: I tried the bind mount step and didn't get an error at that step, but did get an error with the subsequent linux-user-chroot step, so I assume they're both necessary.15:15
SotK:(15:15
*** mariaderidder has quit IRC15:15
*** a1exhughe5 has quit IRC15:15
Zarapedroalvarez: Okay, that's quite awkward for me rn; maybe could do with a note at the start of the page warning people!15:16
pedroalvarezZara: if 1 doesn't work, try 2,  not sure if both are needed for 215:16
pedroalvarezWarning: don't do this ever!15:16
pedroalvarezZara: what kind of warning :)15:17
* SotK guesses "Warning, you may need two disks for this"15:17
pedroalvarezhm15:17
straycathas anyone tried upgrading from 15.10 to current?15:17
pedroalvarezstraycat: I think that yesterday I upgraded from something older15:18
pedroalvarezdidn't hit any problem15:18
straycathrm15:18
pedroalvarez(I might be wrong though)15:18
pedroalvarezZara: maybe we can suggest to use a different disk to put the uncompressed-tarball at the start?15:19
pedroalvarez(given that it's likely that people hit this problem)15:19
pedroalvarezstraycat: is it failing for you?15:20
straycatseemed to be, i'll see if i can reproduce it elsewhere15:20
Zarapedroalvarez: yeah, so if I've followed it right you need to extract the tarball and run the native-bootstrap script on one disk, then transfer it all to another disk (bearing in mind that both disks need to be big enough!)? (I personally quite like 'warning: don't do this ever' option)15:24
*** sambishop has quit IRC15:27
pedroalvarez:)15:27
pedroalvarezthing is that it might not be needed15:27
pedroalvarezI had it working once with the bind mount15:28
*** sambishop has joined #baserock15:28
pedroalvarezbut later it stopped working and I had to move it to a disk15:28
pedroalvarezis just annoying15:28
SotKZara: can you do the first step on the Jetson's built-in flash and then transfer the result onto an SSD?15:28
*** mariaderidder has joined #baserock15:29
*** a1exhughe5 has joined #baserock15:29
SotKI think the built-in flash is 16G so should be big enough?15:30
radiofreei wouldn't recommending resizing the btrfs partition on the emmc to 16G15:31
radiofree10G seems to be ok15:31
jmacsFinally, jdk8 seems to be building (touch wood)15:31
ZaraSotK: I'm not totally sure, haha, I built it in the bigger space to be on the safe side (and this jetson has been used before so I'd need to check how much of that 16G is usable)15:32
SotKZara: I'd imagine there will be enough free for the (I assume small) bootstrap tarballs15:32
ZaraI think there would be room for the tarballs, but I thought it needed to be moved after extraction and build? the total of the extracted stuff and the native bootstrap system seems to be around 18G, unless I've misinterpreted df -h . will pastebin the output because maybe I'm doing something weird.15:36
Zarahttp://paste.baserock.org/lukohuqoce15:37
Zarathe extracted stuff + built bootstrap system is in /src/armv515:37
SotK18G seems excessive15:42
SotKZara: what's the output of `du -sh /src/armv5`?15:43
Zaraa long pause, so far15:43
ZaraI'll keep you posted :)15:43
SotKyeah, it takes a while if the directory is big15:44
SotKactually, maybe `du -sh /src/armv5/*` would have been better so we can see which bits are so big :315:45
ZaraSotK: heh, it loaded anyway, and now I am more confused: / # du -sh /src/armv515:50
Zara8.4G/src/armv515:50
Zara# du -sh /src/armv515:50
Zara8.4G/src/armv515:50
SotKI imagine there is other stuff in /src which is making the df -h output confusing15:50
SotKlooks like it will be able to fit on the emmc anyway15:51
SotKthough from the looks of df -h you'll need to do something like `btrfs filesystem resize / 10G`15:52
paulsherwoodstraycat: +1 on logging, if it's in python by default. i've been learning python on the way :)15:52
SotK(maybe / and 10G should be the other way around)15:52
*** jonathanmaw has quit IRC15:58
*** a1exhughe5 has quit IRC15:58
ZaraSotK: thanks :)16:00
paulsherwoodrdale: i'm consistently failing to build your systemd. any ideas/16:00
rdaleis that using ybd?16:01
paulsherwoodyup16:01
rdalethe openwrt system doesn't have systemd, and so i'm not sure what is going on with your build16:01
paulsherwoodinteresting! :)16:01
paulsherwoodthanks. i'll investigate16:01
paulsherwoodrdale: have you produced a different foundation, then?16:02
rdalei created a clean system today, and built it from scratch and it worked. i needed to do a local checkout of 'iwinfo' and that was all, after i fixed the two things you found last night16:02
paulsherwoodrdale: did/could you push your working branch?16:02
rdaleit uses build-essential with a custom busybox config, core-wrt and cpe-wrt16:02
rdalei pushed the two changes to the baserock/rdale/openwrt branch, and pushed the commits that were on github into the baserock busybox git repo16:03
paulsherwood2015-05-19 20:25:04 [linux-user-chroot] WARNING: multiple definitions of build-depends16:04
paulsherwood2015-05-19 20:25:04 [linux-user-chroot] ['strata/core.morph', 'strata/python-core.morph'] | ['strata/foundation.morph']16:04
rdalethat's wrong16:05
paulsherwoodi daresay :)16:05
rdalenone of those strata are in the openwrt system16:05
paulsherwoodok16:05
paulsherwood2015-05-19 20:25:05 [iptables] WARNING: multiple definitions of build-depends16:06
paulsherwood2015-05-19 20:25:05 [iptables] ['strata/foundation.morph'] | ['strata/core-wrt.morph']16:06
rdalecore-wrt.morph is in, foundation.morph isn't16:06
paulsherwoodyup, understood. thanks16:07
rdaleiptables is in both connectivity.morph and cpe-wrt.morph16:08
paulsherwoodyup16:09
*** zoli__ has quit IRC16:23
nowstera lot of stuff moved from foundation to core recently16:26
*** wdutch has quit IRC16:44
*** wdutch has joined #baserock16:46
*** Krin has quit IRC16:54
*** ssam2 has quit IRC16:55
*** bashrc has quit IRC17:00
*** CTtpollard has quit IRC17:14
*** Albert has quit IRC17:18
*** mariaderidder has quit IRC17:20
*** gary_perkins has quit IRC17:45
*** edcragg has quit IRC17:50
*** zoli__ has joined #baserock18:02
*** franred has quit IRC18:43
*** edcragg has joined #baserock19:34
*** edcragg has quit IRC19:50
*** edcragg has joined #baserock20:13
*** zoli__ has quit IRC20:37
*** zoli__ has joined #baserock20:52
*** edcragg has quit IRC21:15

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!