IRC logs for #baserock for Tuesday, 2015-01-20

*** petefoth [~petefoth@host-92-11-21-246.as43234.net] has quit [Ping timeout: 276 seconds]00:25
*** 17WAAZDA2 [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Remote host closed the connection]00:37
*** rdale [~quassel@82.Red-83-47-22.dynamicIP.rima-tde.net] has joined #baserock03:30
*** rdale__ [~quassel@187.Red-2-138-205.dynamicIP.rima-tde.net] has quit [Ping timeout: 276 seconds]03:34
*** mike [~mike@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock07:25
mike is now known as Guest6856907:25
*** petefoth [~petefoth@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock07:31
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock08:04
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:13
*** pedroalvarez [~quassel@ec2-54-191-112-126.us-west-2.compute.amazonaws.com] has joined #baserock08:51
Mode #baserock +v pedroalvarez by ChanServ08:51
Mode #baserock +nt by verne.freenode.net08:51
Mode #baserock +o pedroalvarez by ChanServ08:55
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:00
*** bashrc [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:06
*** sambishop [~sambishop@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:12
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:21
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]09:23
bashrcgreetings earthlings09:27
bashrcdoes anyone know how to get out of "emergency mode" on boot?09:27
bashrcI'm getting the error "failed to start network service"09:27
paulsherwoodfor what device?09:28
bashrcit's a VM09:29
paulsherwooddo you have a console?09:29
bashrcyes, but not a usable one09:29
bashrcI'm just instructed to predd Control-D to continue, perpetually09:29
bashrcs/predd/press09:29
paulsherwoodhow did you get to this, if i may ask?09:30
bashrchttp://wiki.baserock.org/guides/vm-script/09:30
paulsherwoodam i right in thinking you wrote that page?09:31
bashrcyes09:31
bashrcif there isn't any way of getting out of emergency mode then I'll try again. It does take a while to install though09:32
persiaIt depends on the problem.09:33
persiaCheck your logs.09:33
bashrclooks like a network problem09:33
persiaGenerally that happens when the initrd can't load the filesystem properly.09:33
bashrcah, I got in09:34
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:35
Mode #baserock +v ssam2 by ChanServ09:35
bashrcit does some hanging and then you can enter a root password09:35
bashrcshould there be an /etc/resolv.conf ?09:35
KinnisonA lack of resolv.conf should not prevent boot09:35
pedroalvarezi think that I've seen once the network service failing just because it failed to mount a device listed in fstab before, could this be the case?09:38
pedroalvarezs/it/the system/09:39
persiafailing to process fstab should cause the system to enter recovery mode, whereas failing to start network should not: perhaps the network failure, although the last error message, is a side-effect of an earlier error?09:43
*** Krin [~mikesmith@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:48
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:48
pedroalvarezbashrc: ^^09:49
bashrcI'm looking at the logs. Because there's no ssh access, it's awkward09:52
ssam2pedroalvarez: what's the deal with git.baserock.org now? DavePage was saying something yesterday about it needing migrating again09:52
pedroalvarezI'm happy to do it again.09:53
DavePageWell, I still have Plans for the host it's currently using, once we migrate g.b.o back to DC09:53
pedroalvarezthen, let's move it again to DC, so we can move forward09:54
franredummm, and if DC goes down again which machine are we going to use to have a back up of g.b.o?09:54
franredpedroalvarez, DavePage ^^ ?09:54
DavePagefranred: Possibly, to the same Hetzner box which has been reinstalled ;)09:55
franredsounds good to me09:56
bashrclooks like the first error is probably "systemd-networkd.service has no holdoff/stopping network service"09:57
bashrc"holdoff time"09:57
ssam2seems testgerrit.baserock.org is down because Java ran out of memory again09:58
ssam2i'll restart it09:58
Kinnisonbashrc: Is this a fresh VM made from a download from wiki.baserock.org ?09:58
bashrcit was made yesterday from download.baserock.org/baserock-current-build-system-x86_64.img09:59
franredssam2, pedroalvarez, we should think at some point how to use hetzner as a back up which we can swap in few seconds/minutes if g.b.o goes down at some point in the future and try to automatize the migration that we are going to do for second time10:00
bashrcI could scrap it and try again. Might be useful to know why it broke though10:00
persia+1 for automated failover10:01
DavePagefranred: Trouble is distributed / HA / failover git is Hard without proprietary software10:01
straycatDid we reach a decision on creating a new devtools stratum for things without dependants, like vim, that we can change without needing to rebuild a third of the system?10:02
persiastraycat: A few people +1'd the idea, but I don't know that there was a patch that was discussed for actual application.10:02
persiaI don't remember anyone expressing opposition10:03
franredDavePage, I still think that we can use the troves synchronized and swap troves main functions from mirror the repos from one trove to mirror the repos from the proper repositories10:03
straycatpersia, Okay I'll send one then10:03
franredbut maybe pedroalvarez and ssam2 know if that is possible without an horrific amount of development10:05
ssam2franred: baserock isn't at the stage where we need to keep downtime within seconds10:05
ssam2franred: but within minutes or hours would be good10:05
ssam2franred: if we back up git.baserock.org's /home to hetzner, it seems logical we could set it up in such a way that we can also spin in a mirror git.baserock.org in afew minutes10:06
DavePagefranred: Basically there's some political / business discussions that need to happen between Baserock and Codethink as Baserock's primary infrastructure donors; lots of things are technically possible but may not be desireable for Baserock's long-term objectives.10:06
franredDavePage, good point... I missed it, was just focused in technical stuff sorry10:09
*** lachlanmackenzie [~lachlan@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:14
ssam2from the baserock-mason-x86-64 (new Zuul-based Mason) logs:10:16
ssam2Jan 20 04:59:15 baserock-mason-x86-64 sshd[10095]: Accepted publickey for root from 82.70.136.246 port 54824 ssh210:16
ssam2preceeded by several segfaults in libc.so10:17
ssam2i've turned off the machine10:17
persiaGood choice.10:17
persiaIs there anything manual on there, or can it just be redeployed?10:18
ssam2it can be redeployed, but presumably would be compromised again10:18
ssam2actually, it was only given an internet-facing IP for debugging purposes10:19
ssam2so could be redeployed without one10:19
DavePageI would suggest that if it was *preceded* by segfaults, then that probably wasn't the cause of the rooting.10:19
DavePageSomebody rooting by guessing a private key is hugely unlikely.10:19
persiassam2: redeploying without one sounds best.10:20
persiaIf the attack vector can be identified, and the relevant bit patched, even better.10:20
ssam2I can give someone access to the image if they'd like to investigate.10:21
persiaDavePage: My interpretation was overflows that granted filesystem write access that granted ssh, but you could be right.10:21
DavePageHang on10:21
persiaThat said, I don't really want to explore attack vectors in this channel :)10:21
DavePagehttp://www.whatismyip.com/10:21
DavePageI suggest people in the CT office click ^^^10:21
Kinnisonwhy?10:22
ssam2hmm, that does indeed suggest that the login was from the ct office :)10:22
DavePageKinnison: You might see an IP address which is eerily familiar10:22
KinnisonDavePage: It's a zen IP10:22
Kinnisonand a misspelled attribution10:22
Kinnisoninetnum:        82.70.136.240 - 82.70.136.24710:22
Kinnisonnetname:        ZEN00009066310:22
Kinnisondescr:          Code Think Ltd10:22
Kinnisontsk10:22
DavePageKinnison: But specifically, it's the IP address in ssam2's log message which he thought implied a VM being rooted.10:23
Kinnisonoh10:23
persiaI'm not made that much more comfortable by a login from Codethink if the source isn't identified further.10:23
Kinnison"""Accepted publickey"""10:23
DavePagepersia: I'm made quite a bit more comfortable, but it was 5am.10:23
Kinnisonimplies it was someone who has a pubkey in root's .ssh/authorized_keys10:23
Kinnisonso I'd start from that list10:23
DavePageKinnison: Yes, I mentioned that previously :)10:23
* Kinnison is slow10:24
* Kinnison is also angered that whatismyip.com doesn't have ipv6 support10:24
Kinnisonif I google for 'what is my ip' I get told 2a02:40:41:7fff:211:acff:feff:ffed which is more fun :-)10:24
pedroalvarezssam2: don't worry about the segfaults10:31
pedroalvarezis caused by someone trying to ssh in to a non-existing user10:31
ssam2ok10:33
ssam2I've brought the machine back up but removed its floating IP10:33
*** grahamfinney [~grahamfin@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:33
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 256 seconds]10:38
grahamfinneyOK, new on  Baserock. Where is the best place to start with the documentation?10:41
Zara_http://wiki.baserock.org/ is what I use most10:41
ssam2SotK: can I have your help debugging a zuul-mason issue?10:42
ssam2i'm trying to trigger a job but I see in the debug logs this:10:42
ssam22015-01-20 10:40:21,243 DEBUG zuul.IndependentPipelineManager: Considering adding change <Change 0x7f9084112390 134,1>10:42
ssam22015-01-20 10:40:21,243 ERROR zuul.IndependentPipelineManager: Unable to find change queue for project None10:42
persiaIt's massively undercomplete, so ask here when you get stuck, and please help fix any pages that caused confusion once you have the answer.10:42
paulsherwoodgrahamfinney: please shout up here if you are confused or misled by anything on wiki.baserock.org10:43
SotKssam2: what is the command you are using to trigger the job?10:44
ssam2zuul -c /etc/zuul.conf enqueue --trigger gerrit --pipeline check --project definitions --change 134,110:44
ssam2trying to use new mason to test paulsherwood's morph-arch-config branch :) http://testgerrit.baserock.org:8080/#/c/134/10:44
SotKssam2: can you paste /etc/zuul-layout.yaml somewhere?10:45
*** mariaderidder [~maria@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:45
* paulsherwood is excited to see his patch getting such attention10:46
ssam2SotK: http://paste.baserock.org/puvuxeyuko10:46
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:47
ssam2it's the zuul-mason we deployed before Christmas10:47
ssam2I've not redeployed it since10:47
SotKOK, the project on testgerrit looks to be "baserock/baserock/definitions" but that config file and your zuul enqueue command are expecting it to be called "definitions"10:48
ssam2ah, ok10:48
SotKjust s,definitions,baserock/baserock/definitions,g should work I think10:49
grahamfinneyThanks paulsherwood. Been looking at the wiki. I'll continue wading. 10:49
ssam2SotK: morph is running now. thanks for your help!10:51
SotKssam2: no problem10:51
bashrcso, is the latest baserock 3.15.2 ?10:58
ssam2versioning scheme is YY.WW (last 2 digits of year . week number)10:59
ssam2the latest version is 15.0210:59
paulsherwood3.15.2?10:59
bashrcit's what someone was just saying11:00
bashrcI don't know where the 3 comes from11:00
kejiahuuname -a in baserock gives 3 digits version number11:02
bashrc3.18.011:03
* bashrc is now officially confused11:04
* SotK believes that is the kernel version11:04
persiakejiahu: Why so it does.  Is this a problem for some tooling?11:04
* SotK could be wrong11:04
franredbashrc, kejiahu , that is the linux kernel version11:04
bashrcah, of course11:04
kejiahuoh, thanks for clarify11:05
franredlatest tag of morph and definitions I think is 15.0211:05
franredmorph --version will give you the version of morph you are running11:06
franredif your system has morph on it ;)11:06
persiaThere's no good way to identify the version of Baserock as a number.  One can check the definitions hash used to create the system one is running, and investigate nearby tags, but that's about it.11:07
persia(this is an intentional side effect of Baserock being a set of tools to generate systems, rather than being a software distribution)11:07
tiagogomes_can't we put the version of Baserock in /etc/os-release?11:07
kejiahumorph --version only give commit tag isn't it? it's not obvious which one is newer11:08
DavePageA more human-readable form of persia's description in /etc/baserock-provenance or something? Listing the hashes of components therewithin?11:08
tiagogomes_Also, you can look at /baserock/deployment.meta11:09
DavePageAh OK11:10
persiakejiahu: To check ordering of tags, you need to use git, or a git visualisation tool.11:10
persias/tags/commits/11:10
richard_mawtiagogomes_, DavePage: Those don't belong in /etc; /etc is for configuration11:11
richard_mawos-release has been moved to /usr/lib IIRC11:12
kejiahupersia, thanks11:12
nowsterrichard_maw: is that so that /usr can be ro  and /etcb be rw ?11:14
nowsters/etcb/etc/11:14
richard_mawnowster: that's one aspect of it, but the general idea is that /etc is the domain of the system administrator, /usr is the domain of the distribution vendor, hence for a traditional distro, only the package manager should be able to fiddle with /usr (except /usr/local)11:15
ssam2tiagogomes: I'd like us to put the version of Baserock in /usr/lib/os-release in the reference systems we release11:19
ssam2for systems users build themselves, we should provide that facility, but it's impossible for Morph to automatically guess what versioning scheme a user considers it to be11:20
ssam2or to have11:20
ssam2if there was some way to express 'this system is based off Baserock 15.02 but could be wildly different' it would be nice to do that11:20
persiaI don't think morph should even try to do that.11:24
persiaIf we want to provide a version file for reference systems, we should do so using file injection, and store the file to be injected in definitions.11:24
ssam2sure, better for it to be an extension to morph than morph itself11:25
ssam2using the install-files extension and maintaining the /usr/lib/os-release in definitions would be great actually11:27
ssam2except that install-files.configure sucks. but we need to fix that11:27
persiaI wouldn't complain about it implemented that way :)11:31
persiaThere will be a little confusion, as users won't always update the os-release file when changing definitions, but that happens no matter how we do it.11:31
persiaAnd lots of derivatives of lots of image production systems in the past had the same issue: groups sharing more widely were usually happy to change.11:32
tiagogomes_I dislike that morph moves failed builds from the staging dir to failed, making me having to clean everything and re-run configure11:42
richard_mawyou should be able to run `morph gc` to get morph to clean up the artifacts it doesn't need and failed builds11:43
persiatiagogomes_: While I dislike the move as well, I'm not sure I understand your specific reasoning: are you unhappy that morph leaves the dirty build directory?11:44
tiagogomes_richard_maw you misunderstood, I don't want to remove the failed build, I want  to debug why it failed11:44
ssam2tiagogomes_: oh, you're debugging builds that failed in bootstrap mode, right ?11:46
ssam2so the paths change when it moves stuff to /src/tmp/failed and you have to rerun configure11:46
ssam2you can always move it back again :)11:46
tiagogomes_persia, I just would want to cd to the directory and run `make` to see what is the build failure, as sometimes morph seems to swallow some of the error output11:46
persiaI actually prefer the dirty directory, as I like to see what artifacts were produced.11:47
tiagogomes_yep ssam2, build essential11:47
persiaJust cd'ing won't replicate the buid environment, so one can have very different results doing that (because with cd, one ends up using hte host tools, rather than the staging area tools)11:48
tiagogomes_Because the other run in a chroot I don't need to re-run configure after chroot11:48
ssam2persia: in bootstrap mode things are a bit different, because you *are* using the hosts tools11:48
tiagogomes_persia, and for the other ones I chroot11:49
ssam2and often your build takes 20 minutes or more before failing so you get very angry about having to reconfigure it again :)11:49
persiassam2: True11:49
persiaDoes ./configure have to be run again because the paths are different?11:49
tiagogomes_It would be nice that at the end of a build failure, morph printed out a command that would cd to right directory and would  set up same environment as when it failed11:50
persiaIf that is too complicated, I'd settle for morph providing a command to deal with it: e.g. `morph debug foo`11:51
tiagogomes_persia, yes, different paths11:55
persiaLets just not move it then.11:55
persiaIn the event that we run a new morph-managed build, it will be overwritten anyway, and it makes debugging (and discovery) easier.11:55
tiagogomes_+111:56
tiagogomes_persia it will not be overwritten, it will be in a different random temporary directory11:56
jmalkpersia: I'm looking at cross-bootstrapping aarch64 (http://wiki.baserock.org/guides/how-to-cross-bootstrap/) and see that you've addid it here - http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/tree/morphlib/util.py#n470 - but not to the list of valid architectures in http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/tree/morphlib/__init__.py#n40 . Is there a reason you didn't, should I do so, will11:56
persiatiagogomes_: That works just as well.  I think the main issue is it makes the `morph gc` logic harder.11:57
Kinnisonjmalk: whatever you were saying was cut off after ', will'11:57
persiajmalk: I never completed that, so if you're finding issues, you should just fix them.11:57
jmalks/, will/, will it cause headaches?/11:58
persiajmalk: Essentially, the change I proposed was enough to be able to build armv7lhf binaries on an armv8 host from an armv7lhf chroot, but may not be enough to do it right.11:58
jmalkpersia: ok thanks for the info11:58
* Kinnison doesn't like line 470 there -- I'd prefer morph called it 'aarch64l'11:59
tiagogomes_persia I don't think so, chunks build successfully are kept in a chunks folder, so everything in the staging folder will be failed builds11:59
persiajmalk: Also, there's a patch for util-linux that needs to be applied.  I've been sitting on it, but will clean it up and rebase to upstream and give you a link, so you don't have to repeat my investigations.11:59
jmalkpersia: more thanks!11:59
persiatiagogomes_: Assuming morph isn't actually running in parallel, yes :)11:59
*** rdale_ [~quassel@34.Red-83-45-185.dynamicIP.rima-tde.net] has joined #baserock12:00
tiagogomes_persia not supported :)12:01
persiatiagogomes_: I know, but that was part of the previous discussion :)  A patch to drop "failed" is probably the best solution to this class of issues.12:02
*** rdale [~quassel@82.Red-83-47-22.dynamicIP.rima-tde.net] has quit [Ping timeout: 246 seconds]12:03
*** rdale [~quassel@243.red-80-26-163.adsl.dynamic.ccgg.telefonica.net] has joined #baserock12:03
ssam2Kinnison: aarch64l because we may also have aarch64b ? makes sense to me. hopefully jmalk can fix that12:05
Kinnisonaye12:05
*** rdale_ [~quassel@34.Red-83-45-185.dynamicIP.rima-tde.net] has quit [Ping timeout: 252 seconds]12:06
persiaPlease also support arm8l and arm8b, if adding architectures, so `linux32` has sensible targets.12:07
*** rdale_ [~quassel@185.Red-2-136-65.dynamicIP.rima-tde.net] has joined #baserock12:08
ssam2would arm8l be the same as aarch64l ?12:08
ssam2we should pick one for Morph and stick to it, I think12:09
persiaNo.  aarch64l is 64-bit, arm8l is 32-bit12:09
ssam2right12:09
*** rdale [~quassel@243.red-80-26-163.adsl.dynamic.ccgg.telefonica.net] has quit [Ping timeout: 252 seconds]12:09
persiaI've only worked with that hardware in LE environments, but I was able to switch between "aarch64" and "armv8" using `linux32`12:10
ssam2tiagogomes: a good solution for allowing `morph gc` to run at the same time as `morph build` would be to add a lock file to staging areas that are currently being used12:10
ssam2so `morph gc` could remove anything in /src/tmp/staging that didn't have a lock file12:10
paulsherwoodwhy does morph gc need to run at the same time? i've never seen a need forthat?12:11
ssam2program defensively12:11
ssam2if there's a way to make morph break it's own builds, we're stupid12:11
ssam2its own builds12:12
*** rdale [~quassel@248.Red-83-61-44.dynamicIP.rima-tde.net] has joined #baserock12:13
*** rdale_ [~quassel@185.Red-2-136-65.dynamicIP.rima-tde.net] has quit [Ping timeout: 264 seconds]12:16
*** rdale [~quassel@248.Red-83-61-44.dynamicIP.rima-tde.net] has quit [Read error: Connection reset by peer]12:16
*** rdale__ [~quassel@77.Red-83-52-107.dynamicIP.rima-tde.net] has joined #baserock12:16
kejiahuquestion: how to change size of tmpfs in basercok12:18
paulsherwoodssam2: that was an answer to me?12:18
paulsherwoodif so, i repeat my previous suggestion. run morph gc at the start of every build. less complexity. more useful12:19
persiakejiahu: Why do you need to expand it?12:19
* straycat discovers a couple more duplicate lorries :o12:19
persiaThe most common reason is best resolved by configuring morph to use a different temp directory, but if it is something else, editing fstab ought help.12:20
ssam2paulsherwood: that was an answer to your question. removing the 'gc' command and running it at the start of every build would be a simpler solution, good point12:20
paulsherwoodssam2: unless there is a real use-case for running multiple morph builds on one machine simultaneously?12:20
ssam2paulsherwood: although then people would complain that morph deleted their failed staging areas12:20
kejiahupersia, I am trying to morph build an image, and it complaints /tmp too small12:20
franredstraycat, there are some python duplicated lorries for openstack AFAIK12:20
paulsherwoodssam2: no, they wouldn't12:20
ssam2paulsherwood: I often run multiple morphs12:20
persiakejiahu: Right.  Reconfigure morph to use a non /tmp empfs.12:20
persiaErr, non /tmp temporary directory12:20
ssam2I develop morph quite a bit, so my use cases aren't "normal"12:20
straycatfranred, Really, where?12:21
persiaI often find myself waiting for morph, although I've only run in parallel a couple times.  I can well imagine workflows that involve significant parallel usage other than developing morph.12:21
paulsherwoodmutiple morph builds? given it spends most of its time hard at work, running multiples in parallel seems like a bad idea12:21
kejiahupersia, ok, will try it thanks.12:21
persiapaulsherwood: Depends on the environment: if you have good CPU but bad IO, multiples are more tempting.12:22
ssam2in general, writing programs that can't be run in parallel (unless they are system daemons) seems like very bad workmanship to me12:23
persiaI'd agree.12:23
franredstraycat, they are in my list of checking, I can't point you to them right now12:23
persiaAnother use case for parallel morph is when one starts something that is bandwidth constrained, and wants to run something local (or mostly using local cache) in parallel.12:23
straycatfranred, I ask because I'm writing a check script for the lorries repo so we can detect this sort of thing, and it didn't find those lorries12:24
franredummm, interesting, I think I've seen some duplicated12:24
paulsherwoodssam2: fine, except that most folks then get into adding complexity, and don't actually fully test the parallelism.12:26
persiaLet's not conflate things.12:27
persiamorph *should* be capable of parallelism.  morph should clean up after itself.12:28
persiaThere is presumably a solution that meets both these criteria, regardless of who might try to do what else to the code.12:28
paulsherwood:)12:28
*** rdale [~quassel@164.Red-83-50-81.dynamicIP.rima-tde.net] has joined #baserock12:29
paulsherwoodssam2: i notice elsewhere you are considering using ostree with baserock?12:30
*** rdale__ [~quassel@77.Red-83-52-107.dynamicIP.rima-tde.net] has quit [Read error: Connection reset by peer]12:31
ssam2paulsherwood: yeah, hopefully SotK will be looking at that12:31
ssam2I have a meeting now, sorry12:31
tiagogomes_our libtool package really needs to be updated, it is from 200912:39
tiagogomes_and it strips out --sysroot from the flags12:39
persiajmalk: prepress version: http://paste.baserock.org/olojehireb12:41
jmacsHaving an odd problem since switching back from mime-types to ruby-gems/mime-types lorry: http://pastebin.com/rZvXiqPr12:41
jmacsSeems to be trying to fetch the url without ruby_gems12:42
tiagogomes_also, build commands fail if the *last* command executed had a non zero exit status. It would be better that failed if any command had a non zero exit status no?12:42
persiatiagogomes_: That would make more sense to me: users can add "|| true" if they don't want to exit on failure.12:43
jmalkpersia: thanks12:43
tiagogomes_persia, yep12:43
pedroalvarezANNOUNCEMENT: git.baserock.org will be down for 15 min aprox, starting at 14:00 (UK time). We will move it back to Datacentred in this time frame12:44
persiaAlthough it isn't different at this time of year, I'd prefer times announced in UTC :)12:45
pedroalvarezheh, I'll do that in futre announces12:45
pedroalvarezannouncements*12:46
Zara_jmacs: this looks like the issue in chapter 5 here; the repo field looks like it won't work to me: http://wiki.baserock.org/guides/import-tool/rubygems/12:50
jmacsI don't understand the use of the file:// protocol there12:54
jmacsSurely the source needs to come from a trove somehow. I won't have run the ruby import tool locally.12:56
nowsterfile:///  with three slashes?12:57
straycatfile with as many slashes as you like13:05
Zara_jmacs: if you haven't used the import tool, then I don't know how to fix it. If you have, then the repo field should point to the checkout of the package concerned, on your baserock system.13:05
straycatjmacs, the source doesn't need to come from a trove13:06
Zara_eg, for me, the repo for the npm 'accepts' pkg is: repo: file:///src/workspace/here/checkouts/npm_accepts13:07
jmacsutterly confused now13:07
jmacsare you suggesting that every time someone wants to build a system with a ruby gem they have to run the import tool13:08
jmacs?13:08
straycatno?13:08
straycati'm suggesting that for testing purposes you don't need to lorry stuff into a trove13:08
jmacsthis isn't for testing purposes13:08
straycatyou can just build from your local sources, ensure everything works, then lorry stuff into the trove13:08
jmacsmime-types is already lorried and I want to use it13:08
straycatOkay, in that case don't pass --use-local-sources13:09
jmacsstraycat: I'm not passing that13:09
Zara_(I'm not suggesting that either, I'm saying that my knowledge of this only extends as far as the tool, and I can't be helpful otherwise)13:09
ssam2tiagogomes_: is there a new libtool that doesn't strip out --sysroot ?13:10
ssam2that'd be awesome if so13:10
ssam2jmacs: that looks really weird13:11
ssam2you have 'repo: upstream:ruby-gems/mime-types' and yet morph is looking at 'upstream:mime-types'13:12
jmacsIndeed13:12
ssam2if you 'git grep upstream:mime-types' in the commit you have checked out, are there any results ?13:12
tiagogomes_ssam2, http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=a8f549c1348c0bdedf3c01e6bed6c1e1aa5e885513:14
ssam2AWESOME!13:14
ssam2hmm, that's from 2010 ??13:14
tiagogomes_yes, the one that we are using reports 200913:15
ssam2WHAT13:15
ssam2there's a fair amount of faff in the build-essential chunk morphs to work around the fact that libtool eats --sysroot13:15
robtaylorgnng13:16
ssam2stuff like setting LD='gcc --sysroot=$STAGE2_SYSROOT' if I recall correctly13:16
ssam2robtaylor: gnng ?13:16
tiagogomes_ah wait, it is more recent13:16
tiagogomes_but the libtool script in binutils reports 200913:17
ssam2ah..... because we build from tarballs during bootstrap13:17
KinnisonDo we need to just force-upgrade the libtool in that branch then?13:17
ssam2ok. so maybe you can rebuild the bintutils tarball with a newer libtool13:17
ssam2that may be harder than maintaining the current faff, but it's worth a try I think13:18
ssam2it's hard because there are many combinations of autotools that don't work with each other13:18
tiagogomes_I am using the latest binutils tarball, but they seem to still use an older libtool13:18
tiagogomes_tell me about13:18
robtaylorssam2: working around stuff when there was already a fix //13:19
robtaylorah i see13:20
jmacsssam2 straycat _Zara: Problem solved; I'd changed to a different definitions branch, and morph build doesn't understand that13:21
paulsherwoodyes, morph assumes the branch named at 'morph checkout' or 'morph branch' time is the one it should build13:22
tiagogomes_we just need the bintutils libtool for stage3, if stage 1 and stage 2 bintutils used the host libtool we would be fine, as in stage 3 we don't use sysroot support13:22
Zara_jmacs: Glad you solved it. :)13:22
paulsherwoodi've personally settled on just having one morph branch and putting it into the state i want using git reset --HARD foo13:23
jmacspaulsherwood: Does that work OK if you force push your original branch?13:24
paulsherwoodjmacs: force push where?13:24
jmacsSo if I've done a morph checkout of jmac-merged-branch-5 and then force pushed that from elsewhere13:24
paulsherwoodgit branch -v should show that you have checked out jmac-merged-branch-5. you can force reset that branch to any content/state you like, and morph will work as you hope13:26
jmacsSounds good13:26
paulsherwoodbut if you check out any other branch there, morph will still believe it should work with jmac-merged-branch-513:26
*** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds]13:30
*** rdale_ [~quassel@100.red-80-26-165.adsl.dynamic.ccgg.telefonica.net] has joined #baserock13:31
*** rdale [~quassel@164.Red-83-50-81.dynamicIP.rima-tde.net] has quit [Ping timeout: 276 seconds]13:35
persiaI've had success with using `git checkout` to different branches of definitions and morph still doing the right local thing.  I've no idea how this interacts with morph push or distbuild.13:48
*** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock13:49
* paulsherwood found too many dark corners13:49
persiaEven without using `morph edit` or `morph push`?13:52
paulsherwoodi never ever use morph push. the problem jmacs described above is a problem irrespective of morph edit13:53
persiaOdd.  Worked for me before.13:55
bashrcis there any way to check the current installed version of baserock?13:57
paulsherwoodbashrc: metafiles in /baserock tell you all versions of everything13:57
bashrcso I think I now have a working devel image. Better save it before I break anything14:00
* paulsherwood wonders why it was so hard14:00
persiaAre the components of devel not stored at cache.baserock.org?14:00
* persia would think `morph build devel` would just download it, and `morph deploy` would just work14:00
nowstermanpage morph.1 could do with some fixing. The .RI macro doesn't work how the author thought it did.14:07
pedroalvarezDavePage: could we point git.baserock.org and trove.baserock.org to 185.43.218.18314:10
DavePagepedroalvarez: Now?14:10
nowstereg. .RI "" CLUSTER  [ DEPLOYMENT ...] [ SYSTEM . KEY = VALUE ] ""14:10
nowstershould be .ΙR CLUSTER " " "" [ DEPLOYMENT ...] " " [ SYSTEM . KEY = VALUE ]  (or split)14:10
pedroalvarezDavePage: if possible, yes14:10
persianowster: I've often seen patches that small +1'd on IRC, but needs a commiter.  +1 from me anyway (I'm not a committer): if it doesn't get a committer, you might need to mail the list.14:12
nowsterpersia: there are multiple instances of this14:12
Kinnisonnowster: the manpage is created ,at least in part,14:13
Kinnisonnowster: programmatically14:13
persiaIn that case, a proper patch to the mailing list is probably the best way to sort it.14:13
persiaKinnison: Maybe a bug in the generator?14:13
Kinnisonmaybe14:13
DavePagepedroalvarez: DNS is updated, give it ~60s to propogate14:14
pedroalvarezDavePage: great! thanks! I hope this time is definitive the change :)14:14
jmalkDavePage, pedroalvarez: I'm having problems pulling from git.baserock.org - is this because of the change or should it be unaffected?14:16
DavePagejmalk: Might be worth trying again?14:16
pedroalvarezjmalk: is because of the change. It should be up soon14:16
jmalkthanks14:16
wdutchfollowing the cross-bootstrap x86 64 to x86 32 example here: http://wiki.baserock.org/x86_64_to_x86_32/ I've been getting the following14:18
wdutchERROR: Nothing to cross-compile. Only chunks built in 'bootstrap' mode can be cross-compiled.14:18
nowsterthe manpage thing could be a bug in cliapp14:18
pedroalvarezANNOUNCEMENT: git.baserock.org is UP again, thank you for your patience :)14:19
bashrcwdutch: I was getting that earlier14:20
wdutchbashrc, I noticed but build-essentials is in the morph14:22
bashrcyes, I think that's what's recommended on the wiki14:22
* bashrc should build smaller images next time. Copying them is very sluggish14:23
jmalkI'm looking into making changes to morph, following http://wiki.baserock.org/contributing/#index1h2, but morph is a sibling of workspace, so not included in baserock/my-branch - what is the procedure for developing morph?14:36
richard_mawjmalk: I usually follow the steps for using latest morph, then work in the git repository of that14:37
jmalkrichard_maw: thanks, I'll take a look at that14:37
persiaAnother option is to git clone morph, hack it, update definitions to the SHA of your hack commit, and use cycle.sh or similar to update to new morph.14:38
persiaDepending on your deploy/update speed, this method might be annoying though.  On the other hand, if you use latest morph, it's a good idea to test with cycle.sh once before submitting the patch, just to make sure it works when installed normally.14:39
wdutchbashrc, did  you find any way around it?14:43
jmalkpersia: I understand most of that but am unsure how to update definitions to SHA of the morph hack commit14:43
SotKjmalk: edit the "ref:" field in the "morph" chunk in "strata/morph-utils.morph" to your SHA14:44
bashrcwdutch: no I didn't, but am trying with 6G of RAM14:44
jmalkSotK: ok thanks14:44
SotKjmalk: if you aren't pushing your morph repo, or you are pushing it somewhere other than git.baserock.org, you'll want to change the "repo" field to point to your repository too14:45
SotKjmalk: for example, "repo: file:///src/morph"14:45
jmalkSotK: ok. plan is to push to github following http://wiki.baserock.org/contributing/#index2h214:46
jmalkSotK: file:// being a placeholder or literally what I would have to type in?14:46
persiajmalk: URLs can have lots of shapes.  "http://servername/pathname" is a common pattern.  "file:///pathname" is another, and references something on the local filesystem.14:47
richard_mawliterally what you have to type in, see https://en.wikipedia.org/wiki/File_URI_scheme14:48
bashrcwdutch: indeed, I'm still getting that same error14:48
bashrceven after having installed the development system14:48
persiaJust remember to have three slashes.14:48
richard_mawpython accepts just 1 slash, but I believe that's an extension to the standard14:48
SotKjmalk: if its on github, just point to the github repo URL instead14:49
jmalkrichard_maw: thanks. going back to your earlier suggestion of the git repo in morph, do you still follow the baserock/username/feature-description convention?14:49
jmalkSotK: thanks very much14:49
persiarichard_maw: It is indeed an extension.14:49
richard_mawjmalk: I always do14:49
jmalkrichard_maw: great, thanks14:50
richard_mawpersia: yeah, it wasn't until today that I learned that the file URI scheme had a hostname portion between the first two slashes and the third14:50
SotKin fact, I think we have a github repo-alias by default so you could just do 'github:your-repo'14:51
persiarichard_maw: It isn't so important anymore, but it used to be meaningful for clusters, back when /usr/ was typically a shared NFS mount, and /usr/share another one that served multiple clusters with different architectures.14:51
richard_mawpersia: does that mean that the file:// URI was initially for NFS?14:52
persiaNo, that was nfs://14:52
persiaBut in the days when everyone used NFS as I describe, and didn't like NFS, there was constant talk about more transparent filesystem access mechanisms to be implemented at the OS layer.14:54
persiaI believe the file:/// URI scheme was, in part, expecting that to come to fruition14:54
richard_mawIt's fun to learn historical perspective!14:54
bashrcso, assuming my problem has something to do with build-essential, essentially how would I build build-essential?14:56
KinnisonYou don't14:56
Kinnisonyou only build systems14:57
bashrcah14:57
wdutchi tried to build a system with only build essentials but had the same error14:59
wdutchs/build/cross-boostrap/14:59
KinnisonThere are cross-bootstrap systems14:59
Kinnisonyou should use those14:59
wdutchwell I had slightly more success with cross-bootstrap-system-x86_64-generic.morph than cross-bootstrap-system-x86_32-generic.morph15:06
wdutchnow I'm getting ERROR: Failed to copy upstream:bash into /tmp/morph_tmp/staging/tmprVwLJI/stage2-fake-bash.build15:06
Kinnisonyou should be aiming for a system which has the right architecture for your goal15:07
Kinnisonif you'r eon x86_64 and trying to cross to x86_32 you should be cross-bootstrapping an x86_32 system15:07
wdutchI'm just trying to see how it works, I don't have access to any architecture other than this x86_64 machine15:09
ssam2wdutch: you should have a /src/morph.log file that might let you see whatever error occurred in more detail15:12
ssam2if you don't have /src/morph.log, set up a /etc/morph.conf file as described in http://wiki.baserock.org/quick-start/#index4h215:12
*** tiagogomes_ [~tiagogome@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]15:13
*** tiagogomes_ [~tiagogome@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock15:13
ssam2wdutch: in fact, I'd guess the problem is you are using the default morph tempdir of /tmp, which is too small to be useful15:14
ssam2(making /tmp not a sensible default tempdir)15:14
wdutchokay15:14
wdutchssam2, tempdir = /src/tmp and src is 30GB15:19
wdutchI don't think that's the problem15:19
ssam2ok, then why did your error message refer to /tmp/morph_tmp/staging/tmprVwLJI/stage2-fake-bash.build ?15:19
ssam2it should be /src/tmp/staging/...15:19
ssam2(not a trick question, might be a bug, or anything)15:20
* wdutch wonders if he forgot to symlink his morph.conf15:20
wdutchyes he did15:21
wdutchdoh15:21
wdutchthanks ssam2 :)15:22
*** bashrc [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Read error: Connection reset by peer]15:24
*** bashrc [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock15:26
persiarichard_maw: "There is clearly a danger of confusion that a link made to a local15:33
persia   file should be followed by someone on a different system, with15:33
persia   unexpected and possibly harmful results.  Therefore, the convention15:33
persia   is that even a "file" URL is provided with a host part.  This allows15:33
persia   a client on another system to know that it cannot access the file15:33
persia   system, or perhaps to use some other local mecahnism to access the15:33
persia   file."15:33
persiaSo for multiuser scenarios, this was a mechanism to identify the namespace of the host that actually had the file.15:33
* persia had the details incorrect, and is happy to have learned the real reason15:34
richard_mawah, so if the hostname portion doesn't match any of the hosts your machine is known as, you shouldn't attempt to read the file15:34
persiaRight.15:35
persiaAnd "localhost" or "" are wildcards.15:35
persiaI wonder if * is supported, such that one could use "file:///*.baserock.org/foo" to identify path foo that was only available on baserock infrastructure machines.15:36
* persia suspects nobody ever got around to adding support for that sort of thing in the various libraries we use today15:36
richard_mawgiven different machines may resolve hosts entirely differently, it's probably only safe to follow the URI when the host is a wildcard address anyway15:36
bashrcgetting a new cross-bootstrap error in a different VM: ERROR: Couldn't find morphology: strata/build-essential/stage2-linux-api-headers.morph15:37
persiaAh, good point.  More modern RFCs about file: URIs may have removed the hostname specification part, only leaving the format for historical compatibility.15:37
persiabashrc: Does that file exist in your branch of definitions?15:38
bashrcindeed it does15:40
bashrcwithin strata/build-essential15:41
paulsherwoodssam2: are those firehose merge fails on linux master? is that actually possible?15:45
paulsherwood(i'm wondering if it's a sign of a bug)15:46
perrylthat's embarrassing, it seems to be bringing in a lot more than just definitions updates to me15:47
perryllooking at the contents15:47
ssam2yeah, that firehose instance seems to create its candidate branch on an outdated commit of definitions, so it has a large diff against master15:52
ssam2when the diff against master should just be one line15:52
ssam2so it is a bug in firehose15:53
perryl\o/ now, how to fix this bug?15:53
ssam2i don't know, but I'd suspect it has a local clone of definitions which it never updates15:53
ssam2or it updates the remote-tracking refs, but bases its candidate branches off 'master' instead of 'origin/master'15:54
bwhpersia: ISTR some browsers on Windows treat file://host/path as a UNC path15:54
perrylindeed; the first three in december passed mason with minimal changes, the fourth on jan 2nd passed but with a massive diff15:54
persiabwh: That is a microsoft extension, so they should do so to maintain OS compliance (if not RFC compliance).15:55
bwhpersia: and there were some Unix systems where //foo/bar was different from /foo/bar (don't think that's allowed by POSIX though)15:55
persiabwh: For those, one would use file://hostname//foo/bar15:56
persiaI believe it not to be permitted by SUS, but I'm not sure about POSIX15:56
perrylKinnison: in the firehose prototype, is there a section that restricts it to only looking at SHA differences in 'ref's?16:10
Kinnisondo you mean follow-tip ?16:10
perryli wonder if that's the issue? i'm interested as to why firehose is bringing in definitions changes that aren't just ref values16:12
ssam2perryl: remember that gerrit shows the diff between what firehose pushed, and what Gerrit thinks 'master' is16:12
ssam2so if firehose and gerrit have different opinions of what ref 'master' points to, it might cause this sort of confusion16:13
perrylah! then yes it could be not updating definitions...which needs to be put in16:13
persiaIt is probably best to pull "master" from gerrit, rather than from anywhere else, as the potential for confusion can be very large indeed.16:14
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]16:21
nowsterstage2-fake-bash failing with: /src/tmp/staging/tmpXfoFif/tools/bin/sh: line 4: syntax error: unexpected ")"16:22
nowsterhas anyone seen that ^^ before?16:22
ssam2no, that's creative16:22
ssam2could you pastebin some context somewhere?16:23
nowsterI assume it's something coming from the install script.16:23
paulsherwoodi think i've seen it when my ybd attempts have not been progressing from stage1 to stage2 properly, ie it's not getting the  expected paths/tools16:23
persiaIs bootstrap something Mason is testing?16:24
nowsterssam2: https://admin.codethink.co.uk/pnopaste/?219616:25
paulsherwoodpersia: no, i don't think so.16:26
SotKpersia: nope, just build and deploy16:26
paulsherwoodcan anyone explain/remember why boostrap starts differently from normal systems?16:26
Kinnisonnowster: My recent patch fixed that16:26
persiaSotK: Would it be possible to add something that verified build-essential still builds as well, or does caching mean that can never work?16:26
Kinnisonnowster: and sam merged it16:26
Kinnisonnowster: So you might just need to pull changes from master (of definitions)16:26
paulsherwoodKinnison saves the day :-)16:27
persiaOr rebase your definitions over master16:27
nowsterKinnison: when did that change to definitions go in?16:27
Kinnisonumm, yesterday morning I think16:27
KinnisonCheck if stage2-fake-bash has a build-dep on stage2-busybox16:27
Kinnisonif it does, you need to update16:27
Kinnisonbecause that b-d is wrong16:27
nowsterright16:28
ssam2nowster: http://paste.baserock.org is a better pastebin for here by the way because it's not private16:28
* nowster rebases16:29
nowsterok16:29
nowsterKinnison: Cheers! That fixed it.16:30
persia\o/16:30
Kinnisonnowster: yw16:31
ssam2persia, paulsherwood, SotK: Mason tests bootstrap each time we change something in the bootstrap16:31
ssam2it doesn't test cross-bootstrap at all16:31
persiassam2: Ah, so the key thing is really just that we need to get around to dropping cross-bootstrap already?16:31
KinnisonI hope not16:31
Kinnisonnot being able to cross-bootstrap systems would make me sad16:32
ssam2if one codepath could satisfy both use cases that'd be great16:32
Kinnisonssam2: Sadly the issue was masked by the fact that in a native build, stage2's busybox can be run on the BUILD machine16:32
ssam2although just because the same codepath is used for both doesn't mean it can't work for native bootstrap while breaking for cross-bootstrap16:32
ssam2yeah16:32
persiaKinnison: From what pedroalvarez and richard_maw said yesterday, I had the impression that `morph build` should be able to achieve the same goal, using the regular build-essential morphology.16:32
* Kinnison would find that quite surprising, but given the intelligence of those two, not impossible16:33
* Kinnison thought cross-bootstrap did a fundamentally different thing16:33
* richard_maw needs to collect context for a moment16:34
* pedroalvarez too16:34
pedroalvarez:)16:34
richard_mawyeah, you need to run cross-bootstrap, as I recall, I was saying that we don't need a separate cross-bootstrap morphology, not that we don't need a separate cross-bootstrap command16:35
pedroalvarezand I agreed to that16:35
persiaAh.  I thought richard_maw said that we didn't need a separate morphology and that pedroalvarez said he wanted to just use `morph build`.16:35
richard_mawIf we had heterogenous distbuild networks we could do a cross-bootstrap in morph build, but we're not there16:36
pedroalvarezI tried to comunicate that `morph cross-bootstrap` should behave as `morph build`. But talking about the arguments that the command needs to run.16:36
persiaI was thinking using the same two-step model.16:36
Kinnisonrichard_maw: would that work if we're trying to cross-bootstrap to an arch we aren't on yet?16:37
persiapedroalvarez: Ah, I misunderstood.  Thanks for the clarification.16:37
richard_mawKinnison: not without some form of distbuild worker agent on the target. That may not have to be a Baserock system, but if there's not an existing userland from a distribution then we'd always need cross-bootstrap16:38
Kinnisonso let's assume we always need cross-bootstrap16:39
Kinnisonand that cross-bootstrap is always somewhat different from "heterogenous build"16:39
persiaI think there are three potential models:16:40
persia1) I am building for my host16:40
persia2) My host and target differ, and I have distbuild nodes for the target16:41
persia3) My host and target differ, and I don't have distbuild nodes for the target16:41
persia(3) might have (a) and (b), being whether there are any distbuild nodes at all, but that's somewhat separate,16:42
persiaAll of these ought work, except that in case (3), if any of my build commands or install commands need to execute anything native, it all falls down.16:42
Kinnisoncross-bootstrap is not just aobut building chunks though16:42
persiabut that's just the content of definitions, rather than being a morph thing.16:42
Kinnisonit also prepares a native-bootstrap script and a tarball full of appropriately prepared source trees16:43
persiaIs that something that can't be declared as definitions?16:43
KinnisonI'd really rather not try to16:43
Kinnisonit'd be a huge duplication of effort16:43
persiaIn my imagination, I can specify that some sources get included to some paths in a target, so it's just a matter of cross-building the base toolchain, adding the build script, and adding the sources.16:45
persiaMaking my cross-bootstrap definition mostly a matter of including the sources I want and the build-essential bit.16:45
persiaThat said, maybe this is flawed in some way, in which case I'd appreciate correction.16:45
KinnisonThe point is that morph understands the bootstrap chunks (and builds them, preparing the toolchain for $TARGET via crossing) and then assembles those, and the sources to all the non-bootstrap chunks, and a script to build them all as close to how morph would as possible given the limitations, and creates you a tarball to shunt over16:46
KinnisonIf we somehow expected to move that intelligence into chunk definitions, I'd be scared at the complexity involved16:47
persiaWhereas I get scared as soon as you write "morph understands the bootstrap chunks" because of the implications for alternate toolchains, alternate kernels, etc.16:48
Kinnison"understands the bootstrap chunks" was poor wording16:49
Kinnisonmorph understands that for cross-bootstrapping, it should build the bootstrap chunks, and pack up the sources for the rest16:50
persiaAh, so this works for an arbitrary set of bootstrap chunks?16:50
KinnisonIn theory, it should yes16:51
Kinnisonthe critical thing being that bootstrap chunks are prepared in the context of the HOST platform16:51
*** grahamfinney_ [~grahamfin@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock16:51
Kinnisonwhereas the moment you're doing a non-bootstrap chunk you need to be able to execute TARGET code16:52
persiaRight.16:52
*** grahamfinney [~grahamfin@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds]16:54
paulsherwoodKinnison: by non-bootstrap you mean a chunk whose build-mode is not 'bootstrap'?16:55
persiaWith some thought, this is fundamentally different from a generic cross-compile case.16:55
persiaAs a result, we probably do need a separate command, and the logic in morph, despite my misgivings.16:55
*** CTtpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds]16:56
*** grahamfinney_ [~grahamfin@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds]17:00
Kinnisonpaulsherwood: indeed17:00
straycatperryl, How come it also needs to catch AttributeError? My docs say ValueError is raised if the doc is invalid json17:04
perrylstraycat: if you enter a correctly formatted but nonsense JSON string, e.g. {"sha":"abcdef1"} it throws an AttributeError and exits17:05
straycatperryl, from queue_event you mean?17:07
perrylstraycat, looking at the traceback that occurs without, it seems to be in msg = yaml.load(json.loads(line))17:08
Kinnisonwtf17:08
Kinnisonyaml and json?17:08
straycatso it's binary safe17:08
KinnisonOh is this distbuild?17:09
perrylyes17:09
Kinnisonaah17:09
* Kinnison knows nothing and so shuts up17:09
straycatperryl, Ahh I see now that's coming from the yaml.load17:10
perrylstraycat, does yaml.load have issue with unexpected JSON formats? i'm not 100% on distbuild so i'm willing to defer to you on this17:12
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Quit]17:15
straycatperryl, well we're encoding our yaml in json, so when we decode we're expecting the result of json.loads() to be a yaml document17:18
straycatSo yeah I think you're right to catch the AttributeError there17:18
bashrcugh. Network failures again17:19
straycatperryl, I'm not sure what other exceptions could be thrown from a given input though, the safest thing to do may be to just have that one line in the try block, catch any exception that we get from it and pass17:21
perrylstraycat: and put queue_event outside the try: block?17:22
straycatIn fact, we shouldn't be using load at all17:22
straycat"Warning: It is not safe to call yaml.load with any data received from an untrusted source! yaml.load is as powerful as pickle.load and so may call any Python function. Check the yaml.safe_load function though."17:23
straycat(http://pyyaml.org/wiki/PyYAMLDocumentation)17:24
straycatperryl, sorry, yeah17:26
straycatAnd I'd replace load with safe_load17:26
perrylno problem, i'll update my commit. do you want me to change yaml.load to safe_load in a separate commit as well?17:26
straycateither's fine by me :)17:27
perryli think they should be fine in the same commit as they're relatively in the same scope! i'll push a second patch to the ML once i've tested it17:28
straycatcool :)17:28
jmacsSo, can we do pre-commit hooks on g.b.o.? That's potentially interesting17:29
Kinnisonpre-commit is local to the committer17:29
Kinnisondo you mean update hook?17:30
jmacsI do17:30
KinnisonYes, we can do update hooks, but they are constrained in the ways gitano hooks are constrained17:31
jmacsAlthough maybe paulsherwood and straycat did mean something local to the committer, on the mailing list17:31
KinnisonSo it depends on what you want the hook to do17:31
jmacsOriginally we were talking about a script that checked for duplicate lorries17:33
*** Krin [~mikesmith@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]17:33
persiaI think the best time to run that is post-push and pre-merge, but that depends on infrastructure we don't have today :(17:33
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock17:34
KinnisonI think the check is best done as a "try-load" in lorry-controller and then a hook which queries that17:34
jmacspersia: Either that or done as an automatic review on the mailing list.17:34
persiajmacs: Right.  That works just as well.17:35
ssam2jmacs: about your chef patch: shall I merge it with the fixup franred suggested (of adding 'cloud-init' to the list of enabled configure extensions in systems/ceph-service-x86_64-generic.morph) ?17:36
straycatssam2 suggested the script we currently have could be run as a pre-commit hook, but i'm also happy to try out some of the other options17:36
ssam2alternately, if you felt like requesting push access to git.baserock.org i'd +1 your request, if someone else did so too you could then merge it yourself :)17:37
ssam2s/patch/branch/17:37
jmacsssam2: I'll just have a look at fran's suggestion...17:38
jmacsfranred: Is your suggestion to remove cloud-init from ceph-service-x86_64-generic.morph? If so, I'm fine with that17:40
jmacsOr add the configuration extension?17:40
franredjmacs, my suggestion is add the configuration extension if the system can be deployed in openstack if not cloud-init won't work when you enabled in the cluster morphology17:41
franredor if cloud-init is not needed for anything, just remove it from the system17:42
jmacsThis patch series doesn't address openstack17:42
jmacsSo it's probably there by accident17:42
ssam2there's no harm to having cloud-init available, even if you don't care about OpenStack yet17:44
jmacsAh, I was assuming I'd have to add something to the cluster morph, but if I don't need to, then we can just add the configure extension17:45
ssam2you'd also need to add 'CLOUD_JNIT: yes' to the cluster morph *if* you were deploying to OpenStack17:47
ssam2CLOUD_INIT even17:47
ssam2but as a precedent, the build- and devel- systems contain the cloud-init stratum and have the cloud-init configure extension available17:48
jmacsBut we're not17:48
jmacsRight, I see17:48
ssam2which means that when we *do* deploy to OpenStack, it *only* requires a different cluster morph to enable cloud-init17:48
ssam2listing the configuration-extensions in the system morph is a mis-feature17:49
jmacsI'm fine with adding cloud-init to the configuration-extensions section17:50
ssam2they should effectively all be enabled, I think, but before doing that we'd need to vet them all to make sure all of them are no-ops unless enabled with a deploy-time variable17:50
ssam2jmacs: ok, cool. do you want me to do that and merge?17:50
jmacsssam2: Please do17:50
ssam2ok17:50
jmacsThanks ssam2!17:52
jmacsWere the openstack python tools (glance, nova etc) added into devel-system recently?17:58
ssam2they'd been there a while I think17:59
jmacsBut they are in devel?18:00
ssam2in openstack-clients stratum18:00
*** bashrc [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Lost terminal]18:00
franredssam2, jmacs, but they are only the clients18:00
franredand some of them18:00
ssam2since 2014-08-18 at least18:00
jmacsThat's all I need, I think18:00
*** sambishop [~sambishop@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]18:00
jmacsOdd, I should have them then18:01
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]18:01
pedroalvarezhm...18:02
pedroalvarez`which nova` ?18:02
franredjmacs, http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/strata/openstack-clients.morph18:03
lachlanmackenzieHi, I've been trying to follow the instructions contained in http://wiki.baserock.org/guides/vm-script/ - but I don't seem to be able to login to the vm once it's created. Can you confirm that the default password is still 'root'.18:06
franredlachlanmackenzie, the VM does not have password, it has user root, no password18:07
persiaIs there a default password?  I thought default was no password.18:07
franredpersia, default is no password18:08
jmacsYou can only log in on the console and you won't be asked for a password.18:08
lachlanmackenziejmacs: problem is that I can't launch the graphical console - it comes up with an SDL error.18:09
lachlanmackenzieI've tried running with -curses and it simply displays a grey screen.18:11
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]18:13
persiaUgh.  Another solution is to deploy it with ssh injection (I forget the right argument for that), and ssh in.18:13
lachlanmackenziepersia: Thanks, I will have a look at that.18:14
franredlachlanmackenzie, or using virt-manager18:16
jmacslachlanmackenzie: Yep, that's the only way to do it if you can't get the console (unless you have some serial console emulation in your virtualisation software)18:16
jmacsI don't know for sure if baserock has a serial console tbh18:16
lachlanmackenzieI might try setting the vnc port18:16
persiaI don't know of any configuration extension that creates a serial port, but my knowledge isn't encyclopedic.18:17
*** mariaderidder [~maria@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 255 seconds]18:27
*** brlogger [~supybot@host-78-148-28-74.as13285.net] has joined #baserock18:31
*** brlogger [~supybot@host-78-148-28-74.as13285.net] has quit [Quit: Ctrl-C at console.]18:41
pedroalvarezyup.. supybot will be joining #baserock to get irclogs and then publish them.18:59
pedroalvarezNow I have to investigate how to transform the existing logs to the new format19:00
pedroalvarezor how can I get them from my client (quassel)19:00
pedroalvarezso we don't lost any irclog from the past19:00
pedroalvarezMy attempt of converting paste.baserock.org to a baserock definition, is going slower than expected. I thought that with nodejs in a system I could run any node application.  But no. I will have to use the import tool19:03
*** lachlanmackenzie [~lachlan@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 246 seconds]19:05
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]19:09
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock19:09
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Changing host]19:09
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock19:09
*** zoli_ [~zoli_@linaro/zoli] has quit [Ping timeout: 245 seconds]19:14
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock19:16
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Changing host]19:16
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock19:16
*** Guest68569 [~mike@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]19:21
*** rdale [~quassel@50.Red-2-137-7.dynamicIP.rima-tde.net] has joined #baserock19:40
*** rdale_ [~quassel@100.red-80-26-165.adsl.dynamic.ccgg.telefonica.net] has quit [Ping timeout: 264 seconds]19:43
*** JPohlmann [~jannis@xfce/core-developer/JPohlmann] has quit [Quit: Changing server]19:44
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]19:46
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]20:20
persiaWhat magic does the import tool do that can't be done manually?  I thought the import tool was just a mechanism to make it easier to write definitions.20:21
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock20:26
*** rdale_ [~quassel@253.Red-2-136-156.dynamicIP.rima-tde.net] has joined #baserock20:51
*** rdale [~quassel@50.Red-2-137-7.dynamicIP.rima-tde.net] has quit [Ping timeout: 255 seconds]20:54
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]21:05
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock21:06
pedroalvarezThe haste-server depends on 8 or 9 node libraries. Just because of that. 21:28
pedroalvarezSo yes, the only magic that it's supposed to do is to save my time. :) 21:39
persiaAh, excellent.  Tools that save time are lovely.21:43
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]21:56
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock22:07

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!