IRC logs for #baserock for Monday, 2015-01-12

*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]01:03
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock01:03
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]01:04
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock01:04
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]05:21
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock05:22
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]05:22
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock05:23
*** zoli__ [] has joined #baserock05:27
*** zoli_ [~zoli_@linaro/zoli] has quit [Read error: Connection reset by peer]05:27
*** zoli__ [] has quit [Remote host closed the connection]07:15
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock07:34
*** mike [] has joined #baserock07:45
mike is now known as Guest9350907:45
*** wdutch [] has joined #baserock08:10
*** rdale [] has joined #baserock08:36
*** bashrc_ [] has joined #baserock09:00
paulsherwood+1 for patchwork09:01
*** bashrc_ [] has quit [Client Quit]09:02
*** bashrc_ [] has joined #baserock09:04
pedroalvarezI would have said +1 some time ago, but now.. Weren't we planning to move to gerrit?09:13
*** CTtpollard [] has joined #baserock09:19
*** jonathanmaw [] has joined #baserock09:21
*** tiagogomes [] has joined #baserock09:25
persiaEither is better than what we have today.09:34
mwilliams_ctdo we have any naming conventions for feature branches?09:45
straycatmwilliams_ct, what are you pushing to?09:46
straycattrove/name/feature_foo is common09:46
mwilliams_ctstraycat: github, as per
straycatit's not just a convention though, but something defined by trove's gitano rulesets09:46
straycatOh, I'm not aware of any particular conventions for pushing to random git repos that aren't on trove09:47
rjekIs convention that is enforced by rules tradition? :)09:47
mwilliams_ctstraycat: ok, I'll go with name/feature in that case. ta :)09:48
*** sambishop [] has quit [Read error: Connection reset by peer]09:50
*** sambishop [] has joined #baserock09:52
*** Krin [] has joined #baserock09:53
pedroalvarezHi Zara_! did my suggestions to install node-npm and node-semver work?09:59
*** jonathanmaw [] has quit [Ping timeout: 264 seconds]10:03
Zara_Using 'npm install' doesn't work there, unfortunately (I tested it on Friday just in case I'd misremembered from December). :< It tries to do extra stuff that we don't need, and then it gives a bunch of errors and fails. 'npm build' does just the relevant part of 'npm install'.10:03
Zara_moving build and install commands to the relevant sections makes sense, though :)10:04
Zara_(I templated the morphologies off those in sam's branch, which I think were templated off ones the tool generated, so that may explain the oddness. I don't know exactly how morph build does its thing; I tested the chunks and they worked, so I didn't change more than the minimum! :P10:06
*** mariaderidder [] has joined #baserock10:08
pedroalvarezZara_: maybe you could do a `npm build` and then `cp` only the things that you need (a binary?)10:13
persiaThat would require a means to autodiscover the set of things needed, would it not?10:13
Zara_it's very possible that the cp -a * just snuck in there and is doing nothing useful. I'll test a version without it (just because I'm cautious) and the other changes, and if it works I'll send an updated patch round later today.10:16
pedroalvarezZara_: I don't think that you can avoid the `cp -a *` in your version10:18
pedroalvarezpersia: yes, I have to say that I don't know anything about node, just saying that I'd prefer another way to install this node-npm thingy10:19
pedroalvarezhm.. npm source has a Makefile10:21
*** ssam2 [] has joined #baserock10:29
Mode #baserock +v ssam2 by ChanServ10:29
*** jonathanmaw [] has joined #baserock10:30
*** jmalk [] has quit [Quit: Lost terminal]10:35
*** jmalk [] has joined #baserock10:36
Zara_tbh, I don't like submitting patches that I don't really understand, so I'll read up on morph today.10:38
*** locallycompact [] has joined #baserock10:40
pedroalvarezZara_: I think is more a matter of understanding how to install the thing that you want to install10:55
pedroalvarezalthough understanding how morph executes the commands in the chunk morphology is interesting as well10:56
Zara_heh, okay. I think npm is different from other nodejs packages (makefiles aren't the norm), and I didn't handle the shell-scripting side of the import tool anyway, so my understanding here is hazy at best.11:03
radiofreepaulsherwood: patchwork is excellent12:13
radiofreeeven has a cli app!12:13
radiofreealthough maybe it's not what baserock needs/wants. Was pretty elegant and simple though12:14
persiaIt's a delightful tool.  I'd like to see it used.  I think gerrit is better, and has more useful features and more CLI tools, but *anything* is better than what we have.12:15
persiaBaserock desparately needs a patch tracker.  While I'm not part of Baserock Ops, I suspect anyone offering a deployable system that did patch tracking would see it deployed.12:15
paulsherwoodisn't there a deployable gerrit now?12:17
persiaI think so: I think testgerrit was directly deployed.12:17
persiaBut it seemed to need a fair degree of post-deploy config, and I'm not sure if that ever got committed to automation.12:18
petefothIs the project still hoping / intending to use the same tools as OpenStack, or have we moved on?12:18
persiaThe last statement of intent I saw from the operations team was to use the same tools.12:20
persiaI don't think we can do better than them in tool selection, because there are more of them and they have been doing it longer.12:20
petefothpersia: thanks for clarifying12:20
persiaThat said, we still don't have a patch tracker or a bug tracker, which is much more annoying that whether we have a specific one.12:21
persiapetefoth: I'm not authoritative on this: I'm just reporting what I've read.12:21
ssam2getting storyboard going is still on my list, but there's still a couple of problems to solve before it's usable12:23
ssam2patch review and security fixes take priority so I can't make any promises about it being done, sadly12:25
ssam2the main problems are: fixing the open ID provider to take a full name (currently storyboard shows up all users from our openID provider  as "")12:25
ssam2and backing up the database server (although this doesn't actually block it)12:26
persiaThat sounds really close.  Thanks for the summary.12:26
persiaIn that vein, do you know what is required to have a patch tracker?12:27
ssam2a few approaches were discussed on the mailing list in December12:28
ssam2first thing we need to do is pick one. And then do it :)12:29
*** mariaderidder [] has quit [Ping timeout: 264 seconds]12:29
ssam2i don't really have any idea which approach is best, so if it's left to me I will probably pick whichever seems easiest12:29
*** mariaderidder [] has joined #baserock12:30
straycatProbably about time I started doing this >.>12:37
*** wdutch [] has quit [Ping timeout: 245 seconds]13:01
*** wdutch [] has joined #baserock13:10
franredhi, I've create some system-integration commands for my strata: and Im having this error:, are there any way to debug if my system has already the group 25 in use?14:00
persiaI believe system artifacts are tarballs, so you can unpack and inspect /etc/group14:02
paulsherwoodyes they are14:05
franredpersia, true14:07
persiaAlternately, take a look at the logic in adduser and addgroup, and perhaps include some of that to autoselect a sane group number.14:07
pedroalvarezfranred: is this using latest morph?14:13
franredpedroalvarez, using: 1bed7a3732e7d6158613609a57fb1f77ec99de1e14:14
pedroalvarezfranred: there have been some changes in system-integration since then, but I think they are not critical14:16
pedroalvarezworth a try though14:16
pedroalvarezalso, unpacking the system artifact would work if you manage to build it (removing the system-integration commands?). Otherwise it won't produce any artifact14:17
franredpedroalvarez, so it runs the system-integration commands before creating the rootfs package?14:21
persiaYes, it does.  It runs those against a staging area constructed from the included strata immediately before creating the tarball.14:22
persiaApologies for my timing confusion.14:23
*** lachlanmackenzie [] has joined #baserock14:30
mwilliams_ctpedroalvarez: thanks for the review!14:41
pedroalvarezyou are welcome! :)14:42
pedroalvarezso I've done a little research and I've found that you actually need the kernel headers installed to build it14:42
mwilliams_ctthat's right, I knew there was a logic!14:42
pedroalvarezI thought that linux-api-headers (in build-essentials.morph) was installing also that, but I think is another different thing14:43
persiaWhich are you installing with kernel headers?14:43
pedroalvarezthis is installing ZFS14:44
mwilliams_ctpersia: ZFS and its dependency SPL14:44
Zara_(skeletal import tool quickstart tutorial here; hoping will reduce duplication of wiki content: . Not yet linked in 'guides' because it could do with a bit of fleshing out and tidying.)14:44
mwilliams_ctpedroalvarez: so is there currently a better way around this?14:45
persiaIsn't ZFS CDDL?  Just be careful not to cause a binary artifact to exist that contains a single executable with both GPL and CDDL implemented symbols: that can't go through CI.14:45
* pedroalvarez doesn't know about this14:47
pedroalvarezmwilliams_ct: I was thinking that maybe the kernel headers could go in a different chunk outside the bsp?14:48
mwilliams_ctpersia: I would think it's OK to add to definitions as it doesn't produce images or binaries?14:49
mwilliams_ctpedroalvarez: to clarify, a chunk would live in eg spl-generic/ ?14:49
pedroalvarezmwilliams_ct: maybe... bsp-generic/? Then the bsp's that need the kernel-headers can depend on it, and also zfs an spl14:50
* Kinnison notes that the generated kernel headers / build-system includes the .config, thusly you can't easily separate it from the bsp14:51
Kinnisonit needs to be a chunk artifact generated from the kernel build14:52
persiamwilliams_ct: Only if it doesn't get used for CI, because that creates published binaries.14:52
persiaAs a result, it is not in the interest of the project to accept something that can cause such binaries to appear.  That said, it is possible to create CDDL artifacts that are distributable, so can be CI.14:53
pedroalvarezpersia: thanks for raising that important point.14:53
persiaOne just needs to create something that only contains the kernel modules (and not the kernel).  The system artifact is a "collection", making it permissible to have both.14:53
persiaYou then need some magic to cause the CDDL modules to be loaded in the preboot environment in your cluster: this is usually done with an initramfs.14:53
*** lachlanmackenzie [] has quit [Ping timeout: 244 seconds]14:54
*** lachlanmackenzie [] has joined #baserock14:58
pedroalvarezKinnison: I didn't know that thanks. But in this case mwilliams_ct is using the files installed by:, which I don;t think need anything built.14:58
pedroalvarezBut yes, maybe we can't call those files "kernel-headers"14:58
* Kinnison knows that the .config is present in what Debian installs into the kbuild dir14:58
pedroalvarezmwilliams_ct: I take that your system doesn't build without using the --with-liinux option14:59
mwilliams_ctpedroalvarez: was just trying to test that :D15:00
pedroalvarezthanks for cheking :)15:00
rdaleif i've generated a TEST system using the script and i've changed stuff in /etc and /var, will those changes be lost if i rebuild the TEST system again?15:14
persiaI believe system configuration is maintained.  I very strongly believe that to include /etc ,I'm less certain about /var.15:16
persiaMaybe take a backup, then try?15:16
paulsherwoodrdale: why not create a different system next time? then compare both?15:18
paulsherwood(unless you don't have space for that)15:18
richard_mawAIUI /etc, /var, /opt and /home are preserved, but changes made to /etc are sync'd between updates, so if TEST+1 adds a config file in /etc, it gets added to /etc of the system you run, but if TEST+1 adds files to /var you won't, as it contains only the files that were available when you first installed the system15:18
paulsherwoodrdale: note you can specify a name as extra param for cycle, instead of the default TEST15:19
rdaleok, i'll take a backup of what i know i've changed, rerun a build and see what happens15:19
rdaleTEST does describe what i'm doing, but i might try changing it in the future15:19
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]15:20
paulsherwoodrdale: my point is that if you re-run foo bar NEWTEST, it ssohuld create NEWTEST while leaving TEST unchanged (so no need to backup TEST)15:21
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock15:21
rdaleah right - thanks15:21
*** Kinnison [~dsilvers@gateway/shell/pepperfish/x-jvfjkxmehqmxbxuk] has quit [Remote host closed the connection]15:21
paulsherwoodmodulo what richard_maw said, though15:21
*** Kinnison [~dsilvers@gateway/shell/pepperfish/x-bpqnxwtyshrwtriw] has joined #baserock15:24
*** zoli_ [~zoli_@linaro/zoli] has quit [Ping timeout: 264 seconds]15:26
bwhKinnison pedroalvarez: For building OOT modules you need at least arch/$arch/include, include, scripts, .config, Module.symvers. scripts/package/builddeb is probably the simplest example of that15:50
pedroalvarezoh! so this ZFS is actually building a kernel module?15:52
pedroalvarezand that's why it needs the kernel-headers?15:53
pedroalvarezI see15:53
pedroalvarezin that case, mwilliams_ct, I don't think you will be able to build them without addding --with-linux16:00
mwilliams_ctpedroalvarez: no I don't think so either. Still giving it a go though ;)16:00
pedroalvareztaking too long uh?16:01
persiaJust take care for the licensing of the resulting artifacts.  ZFS upstream has good guidance on what is OK, and what is not OK.16:01
mwilliams_ctyep, currently on build 81/208. but doing other stuff so it's not too bad16:01
pedroalvarezmwilliams_ct: hm.. does that mean that this change cause other things to rebuild?16:02
mwilliams_ctpedroalvarez: possibly, but I think it's more likely that I typed in something wrong with morph16:02
mwilliams_ctStill getting used to Baserock16:02
persiamwilliams_ct: What is rebuilding?  Unless you're really knowledgeable about Baserock or very lucky, you shouldn't be able to cause spurious rebuilds by calling morph differently.16:03
pedroalvarezalso, might help to avoid build things16:04
persiaIsn't that default now?16:05
mwilliams_ctpersia: two things I did that may have confused matters. firstly, I got confused between git branch and morph checkout (there is some UI confusion there and it's not well documented) and secondly, busybox was referring to a branch called my-new-system that I changed to master16:05
paulsherwoodpersia: only if it's set in morph.conf16:05
mwilliams_ctpedroalvarez: Think I'm using the cache in morph.conf16:05
persiamwilliams_ct: Ah.  Yes.  morph checkout can be confusing.16:06
SotKmwilliams_ct: changing the ref of busybox would cause such a rebuild16:07
mwilliams_ctSotK: that's what I figured :\16:07
paulsherwoodmwilliams_ct: do you have artifact-cache-server =
SotKadding "artifact-cache-server =" should reduce the time rebuilds take16:08
SotKs,should,to /etc/morph.conf should,16:09
mwilliams_ctpaulsherwood, SotK: Yep that's in there16:09
paulsherwoodmwilliams_ct: maybe rebase your stuff against master definitions? (if that's an option)?16:11
paulsherwood(and use latest morph)16:11
mwilliams_ctpaulsherwood: I think definitions is up to date, I pulled and rebased earlier today to submit my patch. Morph won't be the absolute latest, but it should be modern enough to use the cache server. at any rate, this isn't slowing me down much as I am doing other things while waiting. As pedroalvarez has said, we think this is likely to fail anyway16:12
pedroalvarezmwilliams_ct: given that I'm the only one annoyed by the architectre-dependent strata of zfs and spl, I can wait until other people review the patch.16:14
pedroalvarezI'm not too annoyed, I just think it could be better16:15
persia_I thought the entire point of OpenZFS was platform-independence16:17
*** einonm [~einonm@] has joined #baserock16:20
mwilliams_ctpersia_: maybe so, but currently they don't consider 32 bit stable enoug to run anyway16:21
persiaAh, if you're just restricting to 64-bit environments, that's fine, as long as your restriction test doesn't involve a string comparison between the architecture string and "x86_64".16:22
persiaHow are you testing processor width?16:22
mwilliams_ctusing a ruler?16:22
mwilliams_ctsorry I'm not sure what you mean by processor width in this context16:23
mwilliams_ctgive me a second to google16:23
persiahow many bits wide is the instruction queue?16:23
mwilliams_ctpersia: I don't know the answer, sorry16:23
persiaCommon values are 8, 16, 24, 29, 31, 32, 60, 64, 80, 128, and 256.16:23
jmacsInstruction queue?16:24
* persia needs better words.16:24
persiaHow are you testing the architecture?16:24
persiaAnd as such, how are you filtering between 64-bit and !64-bit?16:25
mwilliams_ctI'm not, that's a good point. Should that be in the strata? At the moment I'm just assuming that the user will guess x86_64 won't work on !64 bit16:25
persiaMy point is more that I can think of 5 64-bit architectures off the top of my head, so testing for "x86_64" is not a good test to meet the criteria demanded upstream.16:26
rjekpersia: what is an instruction queue?  Typically, a CPU is 64 bit if its registers are 64 bit wide; instructions don't tend to factor16:26
jmacsNeither ARMv8 or x86_64 have 64-bit instructions16:26
persiarjek: The result of cognitive dissonane.16:26
rjekjmacs: Nor MIPS6416:26
jmacsPOWER does, iirc.16:27
rjekI don't think there's a hard-and-fast rule.16:27
persiaSPARC64 can, optionally.16:27
persiaI think POWER is optional as well.16:27
rjeksizeof(int)==8 isn't perfect16:27
persiappc64 is *definitely* optional.16:27
mwilliams_ctI don't know enough about computer science to provide an educated comment on this. pedroalvarez is suggesting we should be able to build without concern for the architecture, which is what I'm working on so hopefully this will be irrelevant16:27
persiamwilliams_ct: Excellent.  If it doesn't work, just make sure you have a switch statement that allows it to work for several architectures.16:28
rjekThe only reliable solution I suspect is to hard-code a list of architectures considered to be 64 bit. :-/16:28
persiaI think the only 64-bit architectures we have working in Baserock today are x86_64 and POWER.16:28
rjeksizeof(void*) probably isn't enough to know, either16:28
robtaylorthere are some good tests in autoconf16:29
persiasizeof(int*) might work though.16:29
robtaylorif that's appropriate16:29
rjekpersia: There's a horrible ABI which runs AMD64 in long mode, but with 32 bit pointers.16:29
straycatWhy would there be a difference between sizeof(void*) and sizeof(int*) ?16:29
rjekstraycat: Yes, I'm wondering that :)16:29
persiastraycat: Some compilers process void* in special ways, for annoying historical reasons.16:30
*** mariaderidder [] has quit [Quit: Ex-Chat]16:52
*** mariaderidder [] has joined #baserock16:56
*** Krin [] has quit [Remote host closed the connection]17:07
juergbirjek: I assume you're referring to x32? Why would that cause additional issues (compared to x86-32, e.g.)?17:16
juergbi(and I don't think that ABI is such a bad idea)17:16
flatmushsizeof(void*) is not guaranteed to be the same as sizeof(char*) for platforms that aren't byte addressable17:16
juergbibaserock supports non-byte-addressable platforms?17:17
flatmushI imagine on such a platform given that a void* could point to a char* then the void* would be sized differently to int*17:17
flatmushbut you're very unlikely to run into that now, given all the major platforms support byte addressing17:17
rjekjuergbi: The discussion was how to programatically detect if a CPU/architecture was 64 bit; x32 was an example of one that would confuse the issue if you just checked the size of pointers17:17
persiajuergbi: The issue with x32 is that because of the narrow address space, it isn't a good target for ZFS.  In other contexts, I'd agree that it is not an unreasonable ABI.17:17
flatmushsizeof() isn't programmatic anyway, it's determined at compile time17:18
juergbirjek: it's just a userspace ABI. just like you can run x86-32 userspace with a x86-64 kernel17:18
rjekjuergbi: Yes, I know.17:18
juergbii.e., not different to x86-32 from this point of view17:19
juergbias for the check, would CONFIG_64BIT from include/generated/autoconf.h be suitable?17:20
*** wdutch [] has quit [Quit: Quit]17:21
juergbiand as long as you use the kernel compiler, userspace ABIs shouldn't interfere, i.e., sizeof(void *) or also sizeof(long) should work for all linux platforms, afaik17:21
*** jonathanmaw [] has quit [Quit: Leaving]17:25
*** sambishop [] has quit [Ping timeout: 264 seconds]17:26
*** straycat [] has quit [Ping timeout: 244 seconds]17:26
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock17:27
persiajuergbi: That's a good test indeed.17:29
persiamwilliams_ct: If you can't make it architecture-agnostic, you might use that.17:30
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]17:31
*** straycat [] has joined #baserock17:41
*** ssam2 [] has quit [Quit: Leaving]17:43
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock17:45
*** jmalk [] has quit [Quit: Leaving]17:48
jmacs"ERROR: Git directory /src/workspace/jmac-merged-branch/ has no commit at ref refs/heads/baserock/builds/590358d577..."17:54
jmacsI'm not sure what morph is telling me17:54
richard_mawcould this be related to the recent change allowing you to avoid creating a temporary build branch?17:55
pedroalvarezhas it been merged?17:56
jmacsI doubt I have that change17:56
*** locallycompact [] has quit [Ping timeout: 240 seconds]17:58
*** bashrc_ [] has quit [Quit: Lost terminal]18:00
jmacsJust deleted my working directory and did a 'morph checkout' again and I'm getting the same error18:20
*** Guest93509 [] has quit [Quit: Leaving]18:27
* jmacs tears hair out18:31
*** straycat_ [] has joined #baserock18:41
*** mariaderidder [] has quit [Quit: Ex-Chat]18:42
*** tiagogomes [] has quit [Ping timeout: 245 seconds]18:44
*** straycat [] has quit [Quit: Lost terminal]19:11
straycat_ is now known as straycat19:11
*** lachlanmackenzie [] has quit [Ping timeout: 255 seconds]19:22
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]19:48
*** locallycompact [] has joined #baserock22:15

Generated by 2.15.3 by Marius Gedminas - find it at!