IRC logs for #baserock for Tuesday, 2015-03-03

*** elevenarms has joined #baserock00:22
*** elevenarms has quit IRC01:15
*** elevenarms has joined #baserock01:16
*** elevenarms has quit IRC01:21
*** elevenarms has joined #baserock01:43
*** zoli__ has joined #baserock06:13
*** zoli__ has quit IRC06:40
*** sherm_ has joined #baserock07:32
*** a1exhughe5 has joined #baserock08:05
*** gfinney has joined #baserock08:06
*** pacon has joined #baserock08:23
*** mdizzle has joined #baserock08:39
*** gfinney has quit IRC08:46
*** gfinney has joined #baserock08:46
*** gfinney has quit IRC08:50
*** gfinney has joined #baserock08:51
*** jonathanmaw has joined #baserock08:52
*** bashrc has joined #baserock09:09
*** bashrc has quit IRC09:10
*** bashrc has joined #baserock09:10
*** gary_perkins has joined #baserock09:18
*** edcragg has joined #baserock09:32
*** tiagogomes has joined #baserock09:35
*** rdale has quit IRC09:44
* Kinnison thinks a lot of our delta against busybox may no longer be needed09:49
Kinnisone.g. we no longer need the systemd units for networking thanks to networkd09:49
*** pacon has quit IRC10:00
*** ssam2 has joined #baserock10:06
*** ChanServ sets mode: +v ssam210:06
*** rdale has joined #baserock10:10
mauricemoss_persia, the binutils patch got send in mid 2014 (not by me), but hasn't been applied yet. http://lists.gnu.org/archive/html/bug-binutils/2014-06/msg00017.html10:20
*** wschaller has joined #baserock10:23
nowsterdoes anyone know why stage2-glibc might be doing this:11:03
nowsterdrwxr-xr-x root/root         0 1991-08-24 23:00:00 lib/11:03
nowsterlrwxrwxrwx root/root         0 1991-08-24 23:00:00 lib/lib -> /tools/lib/11:03
richard_mawlib/lib‽ it sounds like there's a `ln -s` somewhere, which hasn't checked that the target already exists and is a directory11:06
nowsterI can't see it in the definition...11:07
richard_mawme neither, and I even had a poke at stage2-fhs-dirs :¬/11:13
nowsterGot it... stage2-gcc11:13
nowster  install -d "$DESTDIR/lib"11:13
nowster  ln -s "$PREFIX/$libdir/libgcc_s.so" "$DESTDIR/lib/"11:13
nowster  ln -s "$PREFIX/$libdir/libgcc_s.so.1" "$DESTDIR/lib/"11:13
ssam2wow, good catch11:13
nowsterThat probably should be going into tools/lib11:14
richard_mawI saw that, I don't see how it's making a lib/lib -> /tools/lib there11:14
nowsterIt's glibc that's doing that later on, I think,11:14
nowsterbut not sure.11:15
nowsterHmm. False positive?11:16
richard_mawI recall we did need to make a symlink, but I don't recall where.11:16
*** tiagogomes has quit IRC11:25
persiamauricemoss_: Have you replied upstream reporting that it also works for you?11:31
mauricemoss_Not yet, will do.11:33
persiaIf nothing else, it can get the conversation restarted.  As with every project, patches against binutils sometimes don't get reviewed unless you review them :)11:36
nowsterI *think* it's in hardlink_all_files from stagingarea.py11:37
KinnisonSpeaking of patches -- persia: You had a patch for linux32 to support setting an armv7 personality on armv8 systems -- did that ever go anywhere?11:37
nowster        elif stat.S_ISLNK(mode):11:37
nowster            # Copy the symlink.11:37
nowster            if os.path.lexists(destpath):11:37
nowster                os.remove(destpath)11:37
nowster            os.symlink(os.readlink(srcpath), destpath)11:37
nowster...which will fail if there are files in the destpath11:37
persiaKinnison: sadly no.  I'll see if I can find where I put that and do something with it.11:38
persiaI hadn't realised anyone was using it.11:38
*** tiagogomes has joined #baserock11:38
Kinnisonpersia: I've a use-case which might benefit from it, so I'd love to take a look if you do find it11:38
nowster...and if /lib exists with files in it, it'll silently fail the os.remove() and make /lib/lib as a symlink.11:40
pedroalvareznot tags if we use them as refs11:41
* pedroalvarez has answered a message from months ago11:42
persiaKinnison: Sure.11:42
* straycat didn't realise morph transformed cluster variables such as false to False11:42
richard_mawnowster: it won't silently fail the os.remove(), it'll raise an exception11:43
bashrcis there a current devel system image download anywhere? It would save me a lot of messing around11:44
pedroalvarezbashrc: http://download.baserock.org/baserock/11:44
pedroalvarezbashrc: http://wiki.baserock.org/download/11:44
* pedroalvarez also points that devel-system artifacts are in cache.baserock.org11:46
nowsterrichard_maw: are you absolutely sure about that?11:46
petefoththose appear to only be base images, with some work needed to make them into devel systems as described in http://wiki.baserock.org/quick-start/#index7h2 (which should be quick if the artifacts are cached)11:46
ssam2straycat: it's YAML doing so, I think11:47
bashrclatest devel system appears to be 14.4011:47
straycatIs there a way I can do a kvm deploy of the rootfs to an lvm rather than a disk image on my file system?11:47
nowsterhmm11:47
richard_mawnowster: 100%, unless it's been monkey-patched, which I don't believe we have done here11:47
straycatsorry, i mean logical volume, not lvm11:47
ssam2bashrc: 15.02 is latest I think11:47
pedroalvarezbashrc: we stopped releasing devel systems to release build systems11:48
straycatssam2, ahh11:48
ssam2oh yeah11:48
bashrcright, but there's no devel system image for that11:48
straycati still don't understand why we stopped releasing devel systems11:48
ssam2limited resources11:48
straycatssam2, ahh11:48
pedroalvarezthe devel system artifact is almost 2G :/11:48
bashrceek11:48
pedroalvarezso yeah, better to release build systems11:49
bashrcvery well11:49
nowsterrichard_maw: drat11:53
* straycat realises he still hasn't merged the kvm ext checks he added :/11:56
* Kinnison still has somewhere in the region of 400 mails in his baserock-dev mailbox "watched" for activity11:57
nowsterthe only place I can find something similar is in install-files.configure11:57
straycatit might be cool to have a merge/bug squashing party11:58
KinnisonThey work for other projects, so why not?11:59
persiaWe should probably schedule in advance, so people can find time.  Maybe the 14th?12:04
pedroalvarezhm.. genivi-common-api-runtime is failing to build because of a: configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.."12:12
pedroalvarezlooks like autoreconf now generates a configure script that requires one of those files to build12:12
pedroalvarez(I may be wrong)12:12
pedroalvarezit also looks like creating an empty file called "install.sh" fixes the problem, but I'm not sure if this is the right way to solve it12:13
persiaIt isn't12:15
persiaGenerally that should be produced by `automake --add-missing`12:16
bashrcjust an observation based upon previous banter: keeping a current base and devel image for download on the baserock.org is probably more important than having a lot of old base images12:17
persiaIf it doesn't work, then there's something funny about the autoconf installation.12:17
straycat14th works for me12:17
persiabashrc: We don't have any mechanism for continuous push to download.baserock.org on commit yet.  I believe that task is currently waiting on full implementation of pre-merge CI.12:18
*** jmalk has joined #baserock12:19
bashrcI was thinking in terms of disk space, because the devel image is such a monster some older base images could be culled to make room12:20
nowstermorph needs a -no-no-nanette option. ;)12:21
pedroalvarezbashrc: afaik there are not disk-space problems12:21
pedroalvarezbuild-systems are like devel systems but without things like ruby, or nodejs, which are needed for the baserock-import-tool12:22
bashrcI expect in the majority of cases for visitors to the site, they're going to be wanting the devel image rather than the base12:24
persiaWe used to just have devel, but people complained, which is why base was created.12:25
pedroalvarezs/base/build/12:25
bashrcfor a final deployment build makes sense, but if you want to do any amount of development...12:25
persiaDepends on the level of development you want to do.12:26
persiaSome folk use a build image, and only run git and morph, doing all their work in another environment.12:26
persiaOther folk like to have all their tools in a devel environment, and do the work locally.12:26
pedroalvarezalso, it's [well] documented how to upgrade your system to a devel-system12:27
pedroalvarezIn other news, `automake --add-missing` may need some patches in this component to work: http://paste.baserock.org/apafikimek12:27
* persia wonders if the process of downloaded cached chunks should be deserialised, for better use of bandwidth.12:27
ZaraI'd also like devel systems to be downloadable, just to save myself time (I tend to work from a devel system), so if we have the resources for this in the future, it gets my vote.12:28
persiapedroalvarez: Ugh.  Maybe there's another way to cause autoconf to provide those files.12:28
persiaZara: How often do you need to download a *new* devel system?12:29
bashrcI agree with Zara that it's mainly to save time when getting started. Otherwise it's a longwinded process, and that is likely to put off newcomers12:33
bashrcmake the barrier to doing something interesting as low as it can be12:34
Zarapersia: Not that frequently, but more often if I need to do things quickly and I hit weird errors that nobody has come across. In those situations it's quicker for me to make a new devel vm than to troubleshoot (and if I then get the same errors, it narrows the causes down), but it's fiddly.12:34
nowsterstill nibbling away at the stage2-glibc symlink problem.12:35
nowster  *)12:35
nowster    loader=$(basename $(ls "$DESTDIR$PREFIX"/lib/ld-linux*))12:35
nowster    ln -s "$PREFIX/lib/$loader" "$DESTDIR/lib/$loader"12:35
nowster...what if that "ls" doesn't find anything?12:35
nowsterbecause ld-linux is actually ld.so.112:36
persiaZara: I generally just upgrade, or deploy a new devel from my local devel.  To avoid causing issues, I try to only use defined software (no use-local-morph, no local installs), and try to keep a known-good devel image as one of my installed system versions for use with system-version-manager.12:36
Zarapersia: I think this works if you're sure that your vm itself isn't broken in some way (eg: if you have a sudden power outage; my laptop likes to suddenly turn off every so often). I usually do upgrade, and I try to keep a working vm on hand, but the issues come when the image you're upgrading from won't let you upgrade because it's broken in some way.12:40
* nowster tries http://paste.baserock.org/yezejohosi12:40
Zarabashrc: agree that it would lower barrier to newcomers. the only reason I haven't really pushed this before is because I think it serves a dual purpose as a 'building and deploying on baserock' tutorial, where lots of people know what the end result should look like.12:41
persiaZara: You should be able to boot an alternate system without fear if your primary system is corrupt from power loss, assuming you have access to the console to control the boot target, but yes, I concede the point :)12:41
nowsterI suggest http://paste.baserock.org/ubovevixoy  (not a real u-diff)12:44
tiagogomesthe reason why there is a initramfs on clusters/hardware-deployment.morph is it because it was need by the installer, or because it was needed to boot the deployed system in the hardware (e.g. drivers needed)12:50
Zarapersia: :) yeah, I'm sure there are ways around it. I'm just weighing in because I've wished there was a download before, and someone said the images were removed because they were unpopular.12:50
Zaraso now you know where I stand! :P12:50
nowsterpossibly better: http://paste.baserock.org/uxupuqiquv12:50
persiaZara: I think I'll adopt the position that it is a tutorial, and that morph doesn't parallelise cache downloads is a bug.  That lets me cogently argue that not having devel images for download is to save the developer time.12:52
petefothAnd what care we for truth, when we can argue cogently? ;)12:52
* ssam2 still hopes that 'ostree pull' is good enough for artifact fetching12:53
pedroalvareztiagogomes: the installer system doesn't have initramfs, is t he system to install that has initramfs12:53
* petefoth has cause to swear at StroyBoard's half-implmeneted UI :(12:54
persiassam2: Does that parallelise?  The main issue is that the download speed is limited by the latency for each packet, because of how TCP works, so if there are multiple connections with simultaneous traffic, the aggregate time-to-complete is lower.12:54
tiagogomespedroalvarez that's what I needed to know, ta12:55
*** wschaller has quit IRC12:56
ssam2persia: it shares the content of files between the different artifacts12:57
ssam2i suppose that's a separate problem, though12:57
ssam2our artifacts don't really overlap in content...12:57
ssam2hopefully it also parallelises the transfer of those files, but I don't know12:58
pedroalvarezpersia: thanks for your help, Turns out that autoreconf was trying to generate that file but in build-aux/13:03
pedroalvarezand ./configure looks for them in . .. ../..13:04
persiaFrustrating.  I suspect something wrong with configure.ac or similar.13:04
richard_mawnowster: ooh, that does sound plausible if ls is failing to signal failure with `set -e` set.13:05
nowsterbusybox ls doesn't!13:06
persiaIt's running in a subshell.  I think I remember that not carrying the flags (-x, -e, etc.)13:07
nowsterah right!13:07
nowsterof course.13:07
nowsterI was wrong about busybox13:07
nowsterit does DTRT13:08
nowsteryay! It works!13:20
nowsterdrwxr-xr-x root/root         0 1991-08-24 23:00:00 lib/13:20
nowsterlrwxrwxrwx root/root         0 1991-08-24 23:00:00 lib/ld.so.1 -> /tools/lib/ld.so.113:20
richard_maw\o/13:26
*** fay has joined #baserock13:32
*** fay is now known as Guest7505613:32
*** fay_ has quit IRC13:33
pedroalvarezok, weirdest error of the day: http://paste.baserock.org/jahelehixu13:39
pedroalvarezAt first glance I thought that `make` was doing weird things with $DESTDIR, so I decided to remove it before calling make13:40
pedroalvarezbut.. this is building libarchive, and the error says: grep: /attr.inst/usr/lib64/libattr.la: No such file or directory13:40
pedroalvarezok, I think I found the error:13:43
pedroalvarez/usr/lib64/libattr.la:libdir='/attr.inst/usr/lib64'13:43
pedroalvarez/usr/lib64/libacl.la:dependency_libs=' /attr.inst/usr/lib64/libattr.la'13:43
pedroalvarez(running ` grep -r attr.inst /`)13:43
*** wschaller has joined #baserock13:44
richard_mawI've heard it recommended to just delete .la files. It was in the context of relocating them, but apparently it doesn't matter, since you can build fine without them.13:45
*** Guest75056 is now known as fay_13:45
pedroalvarezrichard_maw: do you mean I can just remove that file when installing attr?13:47
richard_mawI've heard that recommended, since you don't need the .la files to find shared libraries. AIUI the libtool stuff is to provide compatibility of build-system between platforms where libraries work less well than on Linux13:49
*** sherm_ has quit IRC13:50
persiaAnother reason is for bundling libraries with applications for binary installation.13:51
* pedroalvarez removes the .la file at the end of the attr.morph definitions13:52
pedroalvarez-s13:52
pedroalvarezoh, weird things are happening in attr13:58
pedroalvarezis installing things in $DESTDIR$DESTDIR13:58
pedroalvarezfixing that will fix this error :)13:58
pedroalvarezbut then I wonder why I don't have a /attr.inst folder in my system :/13:59
pedroalvarezfixed, will send a patch  soon14:04
*** sherm_ has joined #baserock14:14
*** fay_ has quit IRC14:33
*** fay has joined #baserock14:34
*** fay is now known as Guest3007814:34
*** tiagogomes has quit IRC14:41
bashrcany ideas what might be causing this: /tmp/morph_tmp requires 4000000000 bytes free, has 104871936014:43
persiaCheck /etc/morph.conf to make sure you've told morph not to use /tmp as a temporary directory.14:44
persiaOr create a much larger /tmp14:44
KinnisonAlso suggest better wording for that message since you clearly didn't understand it immediately :(14:44
bashrcthere doesn't appear to be an /etc/morph.conf14:44
KinnisonThen you've not yet finished configuring morph according to the recommended process14:45
Kinnisonthe wiki has instructions14:45
persia(but usually people reporting this don't have /etc/morph.conf)14:46
*** Guest30078 is now known as fay_14:47
persiahttp://wiki.baserock.org/quick-start/14:47
persiahttp://wiki.baserock.org/quick-start/#index4h2 , specifically14:48
straycatgiven how often we get this question, we should probably append the question we usually ask the reporter of this problem to the error message14:48
straycat"Have you set tempdir in your morph.conf?"14:48
persia+314:49
KinnisonI'd rather "Is tempdir set appropriately in your morph.conf?"14:51
Kinnisonsince morph can trivially answer the question "have you set it?"14:51
persiaIf we're being nice, maybe suggest creating morph.conf if it is absent?14:51
pedroalvarezbashrc: you had the same problem some time ago: http://irclogs.baserock.org/%23baserock.2014-11-19.log.html :)14:54
Zarastraycat: I'd go for 'Have you set tempdir in /src/morph.conf?'; the path *could* be different, but I think this normally happens to people who are trying to follow the wiki guide and accidentally miss a step14:54
Zaraso the path might be useful... but ymmv.14:55
*** tiagogomes has joined #baserock14:55
*** wschaller has quit IRC14:59
*** wschaller has joined #baserock15:07
nowster /tmp/morph_tmp   is the default15:12
bashrcso I've been setting up morph and one thought which occurs is why isn't this stuff just a command. "morph setup", or something like that15:13
bashrcpossibly a lot of the boilerplate might be similarly consolidated into commands or command options15:14
bashrcwhen trying to init the morph workspace I'm getting: No module named pylru15:19
straycatyou need pylru now15:19
SotKAre you using latest morph or the one that came in your image? PyLRU should be in systems which have that dependency by default.15:20
bashrcprobably the quick start guide needs updating for that15:21
ssam2SotK: we haven't done a release of the 'build' system with PyLRU in it yet15:21
SotKaha15:21
ssam2i tried a couple of times, but the yak kept growing more hairy, so to speak15:21
SotKthe release should still have an old enough morph in that this doesn't happen by default though right?15:21
bashrcSotK: just following the "no frills" guide15:21
ssam2...maybe pedroalvarez can do one this week :)15:21
pedroalvarezmmmmmmkay15:22
pedroalvarez:)15:22
SotKbmottram: oh, that seems to duplicate the content of using-latest-morph15:23
SotKI was unaware of that when I updated the using-latest-morph page to warn of this15:23
* petefoth sighs15:23
* SotK wonders why we have four different "Getting started" guides15:24
pedroalvarezindeed, too many15:24
petefothBecause we have no spec for Baserock docs, anybody can edit the wiki and nobody is tasked with reviwing any changes15:24
petefothSo we will get 'bugs'15:25
SotKtwo of them appear to be instructions on copy/pasting some things into a script and running it... should we not just provide scripts for people to run at this point?15:26
* petefoth walks away, muttering darkly...15:27
pedroalvarezhehe15:27
pedroalvarezpeople should understand what are they doing when setting things up15:27
SotKI agree, but providing a copy/paste page seems just as bad as a script in that regard (to me at least)15:28
petefothpedroalvarez: yes - and they can understand by reading w.b.o... oh wait!15:29
KinnisonIf we ever get to 'curl http://wiki..... | bash' I will explode15:29
bashrcfixed the nofrills15:29
pedroalvarezKinnison: never15:29
SotKbashrc: thanks :)15:29
petefothbashrc: ta!15:30
rjekKinnison: https://twitter.com/dotwaffle/status/57273506639004876915:30
Kinnisonheh15:31
petefothDoes anything need adding to other pages?15:31
SotKthe kvm script page looks ok15:32
petefothWill the steps in http://wiki.baserock.org/quick-start/ work?15:33
SotKyep15:33
SotKonly things which duplicate the content of http://wiki.baserock.org/using-latest-morph/ will be broken15:33
SotK(I didn't think anything did)15:34
petefothSotK: so no need to add anything there abotu pylru?15:34
petefothSotK: OK thanks15:34
SotKnope, I added it to using-latest-morph when I merged the change to morph, and added pylru to morph's dependencies when I updated the morph sha in definitions15:35
*** gary_perkins has quit IRC15:35
*** gary_perkins has joined #baserock15:37
persiaCan we make using-latest-morph harder to find?  I think it causes more problems than it solves, for most users.15:43
SotKI think we should do releases more often if we do that15:45
straycatWe could have a script that periodically moves it to different urls, like the staircases in hogwarts.15:45
petefothpersia: it's a wiki, you can do whatever you want :)15:45
Zarahm, generally when I come on here with an error, the most common fix after 'you need to update your devel vm' is 'you need to update morph'15:46
persiapetefoth: Yes, but if I just remove the page (which is what I want), someone will surely put it back.15:46
persiaZara: Surely that's an indicator that we're doing a bad job: if we need newer morph for something, and we recommend that something, we should be putting that newer morph in latest definitions.15:47
petefothpersia: Maybe, but if you feel stringly and can justfiy your decision, then maybe not15:47
petefoths/stringly/strongly/15:47
persiaThat's why I ask here?15:47
persias/?/./15:48
SotKnewer morph in definitions is useless if its nearly two months since the last images were released I feel15:48
petefothpersia: I would have no obhjection15:48
persiaSotK: Why?  One just upgrades one's devel to latest.15:48
SotKWhat about people who don't have an existing devel machine?15:49
persiaDownload a build machine, build the latest devel machine.15:49
SotKThen people will question why some old bug is in the new machine they just downloaded though maybe?15:51
tlsaa few weeks ago I found it was impossible to build a baserock system from master definitions with the morph supplied in the system available for download on baserock.org15:51
tlsabecause that morph didn't understand armv8be or somesuch15:51
persiatlsa: I thought we were solving that with the definitions version thing, and that we never expected that to happen again.15:51
tlsaperhaps15:52
tlsayou would need to make a new release though15:52
persiaSotK: We can update the build machine occasionally.  Running *any* software that is not from definitions strikes me as not in line with the point of Baserock.15:52
tlsawhcih contains a morph that checks version15:52
persiatlsa: Yes.  One more release, and we can be done.15:52
pedroalvarezha!15:53
SotKpersia: One more release works I guess. And another release each time we change the definitions format?15:53
persiaWhy?15:53
persiaSurely once we have versioned defintions, we'll be able to have a migration path.15:53
SotKDo we intend for old morph to be able to build new definitions?15:53
persiaWell, no.15:54
persiahrm.15:54
tlsapersia: that will improve the error message (your morph is too old for these defintitions, instead of something obscure)15:54
tlsabut not fix the need for updated morph15:54
persiaSo, yes, one release per definitions version ought do it.15:54
* SotK wouldn't like to find Baserock in a year or two's time, download the build system and immediately have to go through a really long migration path before its up to date15:54
tlsaSotK: snap15:54
pedroalvarezpersia: +115:55
SotKpersia: +115:55
pedroalvarezadding a new architecture makes our definitions change its version?15:55
* persia credits SotK with the suggestion15:55
persiapedroalvarez: No, but does that require a release?15:56
SotKCan we not separate architecture from morph?15:56
persiaI'd like morph not to understand architectures15:56
pedroalvarezpersia: well, if we add a new architecture, the current release won't be able to build those systems15:56
pedroalvarezerm15:57
pedroalvarezyeah, that would solve my problem15:57
persiapedroalvarez: Only because we've hardcoded yet another architecture naming scheme in morph.15:57
pedroalvarezindeed15:57
persiaWe can't do it for version 0, but for definitions version 1, I'd hope we could declare architectures in definitions.15:58
* pedroalvarez is imagining a definition file with architecture information, making things easier when adding new architectures15:58
persiaWhen you finish imagining, please send to the list, to help guide the folk who eventually define definitions version 116:00
kejiahuI am trying to net boot baserock on moonshot nodes, but I got the error message while booting http://paste.baserock.org/ujiqihuyiw and it only boot into emergency mode. does anybody have a idea what may cause these happens16:08
SotKMy OSTree work breaks a whole bunch of the cmdtests since they do things like look for chunk artifacts in the artifact cache directory, how should I solve this given that the chunk name is no longer in the artifact "filename" (a ref in an OSTree repo)?16:12
KinnisonI guess you get to convert the useful ones to yarns :)16:13
SotKlucky me! :)16:13
* SotK wonders how to interrogate the cache from outside morph without knowing the cache key16:14
persiaYou also get to purge the useess ones :)16:14
pedroalvarezkejiahu: last time I saw that, the system was trying to mount a disk that didn't exist16:17
pedroalvarezkejiahu: could you check your /etc/fstab ?16:17
kejiahupedroalvarez, that's what I thought at first, but the /etc/fstab seems ok.16:19
kejiahupedroalvarez, I can't show it now, as the machine is un-reachablt for me now16:20
ssam2SotK: perhaps the trick is to add a command to morph that calculates a cache key16:25
ssam2list-artifacts already does that, but maybe gives output in an unhelpful format16:26
pedroalvarezkejiahu: well, we can figure out. Did you set in the cluster morphology of that system any  "FSTAB_*" option?16:27
kejiahupedroalvarez, yes, it was set in the cluster file, /dev/sda2 /srv/distbuild auto defaults,rw,noatime 0 216:31
kejiahupedroalvarez, /dev/sda2 is a local disk with pre-installed system which I can choose to boot from. but I boot into baserock through net booting16:32
Kinnisonkejiahu: can you paste the full boot including all the kernel messages into the pastebin?16:34
edcraggpedroalvarez kejiahu Kinnison this is a more complete log of the situation, http://paste.baserock.org/yehojakiha16:34
Kinnisonthat looks different16:35
kejiahued's log may different from mine, as I was trying to boot a distbuild system, and he is booting a devel baserock system16:35
Kinnisonyour log is a failure to find a disk16:36
Kinnisoned's log seems to contain a segfaulting journald16:36
Kinnisonwhich is quite worrisome in a different way16:36
kejiahuKinnison, mine contains similar systemd failures as ed, I'll get the full log soon16:39
edcraggthis is simultaneously affecting any systems booted from separate rootfs's on the same nfs server16:40
pedroalvarezcould this problem be related to what we tried to fix in nfsboot.configure?16:43
*** sherm_ has quit IRC16:43
rdalei'm getting a syntax error from busy box on a configure script - i'm not sure what to do about that16:43
persiardale: Could you paste the error?16:44
kejiahuhttp://paste.baserock.org/fozabesota here is the full log with kernel messages16:46
pedroalvarezkejiahu: that one looks like "[ TIME ] Timed out waiting for device dev-sda2.device." is the error16:48
pedroalvarezkejiahu: could you try to remove /dev/sda2 from /etc/fstab and see if that fixes it? and after that check why is not mounting /dev/sda2?16:49
CTtpollardthat seemed to fix my sda2.device timeout error the other day16:50
rdalepersia: this the part of the shell script with the error: http://paste.baserock.org/wagohosina16:51
kejiahupedroalvarez, yes, I am going to do that, it taks a while...16:52
pedroalvarezkejiahu: why?16:52
persiardale: Which line in the paste is line 13062?16:52
rdaleit is before the snippet i pasted, but 'Checking lzma' is appearing in the build log and that is after the line with the error16:53
kejiahupedroalvarez, it's within moonshot chassis, and all access can only be done through iLO and virtual serial port, and today the nodes response quite slow for some reason.16:54
rdalepersia: if i remove all the code in the snippet i pasted, i don't get an error - so it is in there somewhere16:56
pedroalvarezkejiahu: wasn't the system nfsbooting? can you change the /etc/fstab file in the nfs server and reboot?16:56
kejiahupedroalvarez, yes the nfs server was in the same chassis, and I have to access it through the some routine16:58
pedroalvarezargh! :)16:58
*** a1exhughe5 has quit IRC16:59
kejiahupedroalvarez, http://paste.baserock.org/nowesikahu it still gets those faults, excpet the one for /dev/sda217:01
*** sherm_ has joined #baserock17:02
rdalepersia: busybox doesn't like  PKG_CHECK_MODULES() in the configure script17:03
*** gfinney has quit IRC17:03
pedroalvarezkejiahu: hm... "Failed to set file attributes: Inappropriate ioctl for device"?17:04
kejiahupedroalvarez, I feel it's more like a problem of netboot server rather than local hardware, as ed got similar problem on all the rootfs which used to be working before today, but corrupted today17:06
pedroalvarezkejiahu: looks like a known bug, but I don't know if that is the root cause17:06
pedroalvarezmaybe you were using before systemd 218 and now you are using systemd 219?17:07
pedroalvarezif that's the case, then this may help: https://www.mail-archive.com/search?l=systemd-devel@lists.freedesktop.org&q=subject:%22%5Bsystemd-devel%5D+%5BPATCH%5D+journal%3A+fix+Inappropriate+ioctl+for+device%09on+ext4%22&o=newest&f=117:07
kejiahupedroalvarez, but I didn't make any upgarding to my devel system..17:09
pedroalvarezkejiahu: oh, I thought it was a new deployment. So same system was working and now it doesn't?17:10
kejiahupedroalvarez, so this issue happens on both new deployed system, and old deployed ones. and they wera all on same nfs server17:13
pedroalvarezI'm out of ideas then :/17:16
kejiahupedroalvarez, thanks for your help anyway. seems tiago got the idea17:17
tlsaroot17:21
edcraggpedroalvarez: a trove deployed in the moonshot was interfering with the existing netboot server17:22
pedroalvarezhm.. true.. troves have a nfs server17:23
persiaHrm?  Doesn't the cluster declaration of kernel command line identify the nfs server from which to boot?17:26
*** mdizzle has quit IRC17:36
*** ssam2 has quit IRC17:43
*** sherm_ has quit IRC17:49
*** bashrc has quit IRC17:55
*** elevenarms has quit IRC18:00
*** elevenarms has joined #baserock18:02
*** wschaller_ has joined #baserock18:02
*** jonathanmaw has quit IRC18:04
*** wschaller has quit IRC18:05
pedroalvarezjjardon: is your +1 only for libarchive? or also for persistence-* lorries?18:05
jjardonpedroalvarez: it was for libarchive, but +1 for the other as well (you already noticed about the indentation)18:09
pedroalvarezjjardon: yay! thanks18:10
*** tiagogomes has quit IRC18:17
*** edcragg has quit IRC18:18
*** edcragg has joined #baserock18:19
*** edcragg has quit IRC18:24
*** wschaller_ has quit IRC18:34
straycatAttributeError: 'dict' object has no attribute 'startswith'18:47
straycatoh ansible, it was all going so well...18:47
SotK:(18:47
straycatearlier i got an error message detailing exactly what i'd done wrong and exactly what i should do to fix it, it was very good18:48
straycatoh i suck18:50
SotKaccidental yaml dict?18:51
straycatincluding a var definition under vars_files18:51
straycat*included18:51
SotK:)18:51
* SotK spams the mailing list and heads home18:51
franredummm, I break morph....http://paste.baserock.org/ovogutizec  -- compiling baserock/openstack-v419:04
persiaOh wow.19:06
persiaIt is admittedly a complex system, but still, we should be able to deal with that.19:07
franredI've tried to increase the recursion limit but it does not fix it, so I wonder if we create a loop in some part :S19:08
persiaThe thing that confuses me is that it looks like only seven layers, which doesn't seem particularly deep19:08
franredpersia, I cut the log, it is much bigger, sorry to not make the "<snip>" more visible19:09
franreds/much//19:09
persiaAh, I see line 59 now.19:09
persiaDo you know how many layers it has?19:10
franred168, if I haven't checked it wrong19:15
persiaThat seems excessive.19:19
persiaMight be useful to have a loop detector, as I suspect that is the problem.19:19
persiaMaybe add a print statement that reports *which* component is being analysed along the way?  That way there is a handy list of the candidates for the loop (and maybe we can see the loop).19:20
franredpersia, Im doing that :)19:21
franredI need to come back home, I will continue debugging this tomorrow19:29
franredpersia, looks like it get annoyed because 2 strata includes the same chunk - for example I had pycrypto in openstack-common and in openstack-services and openstack-services includes openstack-common19:35
franredwhen removed pycrypto from openstack-services it gets annoyed for the next common chunk19:36
persiaAh, yes, of course.19:36
persiaBecause you have pycrypto build-depending on openstack-common, which build-depends on pycrypto, etc.19:36
persiaGenerally speaking, you want to have each chunk in only one place.19:37
persiaIf you need them to be different for some reason, then identify them differently using @19:37
persia(grep @ in definitions for some examples)19:37
franredyep, but the error is pretty awful - we should say something like you have duplicated your chunk in A and B19:37
persiaBut if you do have them be different, try really hard not to include both in a system.19:37
persiaIndeed.  I like the report ybd produces about duplicates.19:38
persia(don't use ybd to build: it doesn't yet, but it does produce nice reports about definitions)19:38
franredpersia, I don't need them duplicated, I moved out of openstack-services because was common with others strata19:38
persiaOh, excellent.  So it's just cleanup to remove the duplicates.19:39
franredyep19:39
persiaCool.19:39
straycatiirc there's a TODO to add cycle detection, we've just been busy with everything else morph has to do19:47
*** rdale_ has joined #baserock19:48
straycatcome to the bug squashing party and add cycle detection to morph!19:48
*** rdale has quit IRC19:49
*** rdale_ has quit IRC19:50
franredok, looks like I managed to clean all the duplicated chunks so the cycle has disappeared19:51
*** rdale has joined #baserock19:54
persiaAnd it builds now?19:56
*** gary_perkins has quit IRC20:06
pedroalvarezGiven that he has sent a patch, I'd say 'yes'20:30
*** rdale_ has joined #baserock20:41
*** rdale has quit IRC20:44
*** chiru has joined #baserock20:48
*** zoli__ has joined #baserock21:57
*** chiru has quit IRC22:01
*** zoli__ has quit IRC22:18
*** zoli__ has joined #baserock23:05
*** zoli__ has quit IRC23:28

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!