IRC logs for #baserock for Wednesday, 2016-10-19

*** CTtpollard has quit IRC00:54
*** jude_ has quit IRC03:21
*** jude_ has joined #baserock03:22
*** gtristan has joined #baserock04:03
*** fay_ has joined #baserock07:16
*** persia has quit IRC07:30
*** persia has joined #baserock07:33
*** ctbruce has joined #baserock07:38
*** rdale has joined #baserock07:50
*** mwilliams_ct has joined #baserock08:45
*** jjardon_matrix has quit IRC08:45
mwilliams_ctMorning baserockers. SotK's advice worked a treat yesterday, so thanks again :). on a different note, is there a magic curl command that can be run to force a trove to do a remote trove scan immediately?08:46
pedroalvarezmwilliams_ct: I think that's done in the readconf service08:56
pedroalvarez(i don't know for sure, but at least that forces lorry controller to read the configuration)08:57
*** jjardon_matrix has joined #baserock08:59
mwilliams_ctpedroalvarez: I think that just reads and adds anything modified recently, but ls-troves.service seems like it should do what I need. I'd forgotten that the curl commands are called by systemd units, I'll look through them. thanks!08:59
pedroalvarezyes, that one sounds more likely09:00
pedroalvarezbut it looks like is also a queue of jobs, and ls-troves won't do anything until the trove is at the top, and with the counter at 0 (or now)09:01
paulsherwoodgtristan, others: http://paste.baserock.org/umukiboqib09:02
paulsherwoodis that a clear enough break point? i'm plannign to write to the list also09:03
gtristanpaulsherwood, That is clear enough, although I might be inclined to hold off on it09:07
paulsherwoodgtristan: why?09:08
*** locallycompact has joined #baserock09:09
* paulsherwood notes that it's about time we formalized that ybd upstream is no longer github (in the repo itself)09:09
gtristanpaulsherwood, right now I think we are to fork ybd and let the work materialize, at which point I would approach the baserock community and propose that they use the new tooling09:09
gtristanwhere baserock is ideally one of the hopefully many use cases for the tooling09:09
paulsherwoodand that work will happen at gitlab/baserock, or elsewhere?09:10
locallycompactpossible gotchas for https://gitlab.com/baserock/ybd/builds/5239746 anyone?09:12
paulsherwoodwhich sandbox is this using?09:14
paulsherwoodalso, what's happened to the cache-key calculations?09:14
locallycompactThey're happening silently09:15
paulsherwoodfhs-dirs is the first non-bootstrap chunk, fwiw, so it's the first chunk that does an actual chroot iiuc09:15
locallycompactright09:15
locallycompactI don't get this problem in the python 3.6 docker image09:16
locallycompactit gets to autoconf and fails because it can't find automake09:16
paulsherwoodand it creates device nodes09:16
locallycompactall other things equal it gets this far in docker09:16
locallycompacthttps://gitlab.com/baserock/ybd/builds/518862809:17
locallycompactcan't chroot into debris beause there's no bash..09:26
locallycompacthmm09:26
* paulsherwood notes something is up with gitlab ci for ybd now09:27
paulsherwoodhttps://gitlab.com/baserock/ybd/builds/527093609:27
locallycompactsorry09:28
locallycompactgo once more09:28
paulsherwoodThis build is stuck, because you don't have any active runners that can run this build.09:29
locallycompactpublic are back one09:30
locallycompact*on09:30
paulsherwoodtvm09:30
*** gtristan has quit IRC10:08
*** gtristan has joined #baserock10:15
*** gtristan has quit IRC10:21
* paulsherwood notes that the ybd cache server is down to 7GB of free space10:36
locallycompactI set min-gigs to zero and ybd deleted all of my artifacts11:28
* tiagogomes thinks it is very scary that ybd is deleting stuff as root11:30
locallycompactI have no idea how to debug this fhs-dirs thing11:31
locallycompacttwo hours building and then ybd deletes my artifacts11:31
tiagogomesI wonder what happened of someone did `ln -s /usr ~/.cache/ybd/artifacts/foooooooooooooooo`11:32
tiagogomesArtifact splitting in ybd seems wrong.11:49
paulsherwoodpatches welcome11:55
leemingi wonder if it is what i was hitting locallycompact , related to the devices11:55
leemingminus the ybd deleting all your stuff:P)11:55
*** leeming has quit IRC12:00
*** leeming has joined #baserock12:01
paulsherwoodlocallycompact: afaict if you set min-gigs to 0 ybd should not delete anything. are you sure about your claim above?12:03
paulsherwoodtiagogomes: i've been asking for help to make ybd run as non-root for over a year12:03
tiagogomesahh, it is weird that in strata, you can chose which chunk artifacts will go to a stratum, through two different ways12:04
* paulsherwood has to believe this is possible, since other build tools dont require root.12:04
* locallycompact doesn't know what's real anymore12:06
tiagogomespaulsherwood, building is certainly possible without root. It may be possible that some write extension requires root though12:06
locallycompactchecking for suffix of object files... configure: error: in `/root/ybd/tmp/tmp48yn3pqx/stage2-libstdc++.build/o':12:06
locallycompactconfigure: error: cannot compute suffix of object files: cannot compile12:06
locallycompactwhat12:06
paulsherwoodlocallycompact: well, maybe best not to heckle, if you're not sure? :)12:07
locallycompactIt definitely deleted all of my artifacts12:07
paulsherwoodlocallycompact: do you have a log for that?12:07
locallycompactno that was hours ago12:08
paulsherwoodtiagogomes: device nodes appear to be impossible without root so far, hence definitions which include device nodes seem to require that any tool attempting to build them (eg morph, ybd) needs to be root at the time12:10
paulsherwoods/appear to be impossible/have defeated us/12:12
tiagogomespaulsherwood hence I suggested that instead of creating those devices nodes, they were bind mounted in the sandbox12:12
tiagogomesbubblewrap even has a command to only bind mount a sanitized list of devices nodes in the sandbox, instead of whole /dev12:12
locallycompactaren't they real "things" that need to "exist" in the resulting tarball though?12:13
paulsherwoodtiagogomes: what locallycompact said12:13
* edcragg thought they're mostly there when needed for a build12:13
tiagogomesIf devtempfs doesn't help and they need to go the final root fs, you could use fakeroot to create them - I tested this and it works.12:14
persiaSome software only installs as root, and most software prefers to be installed as root.  Doing this without granting root to the build system is usually done through some way to pretend to be root while in the chroot (e.g. fakeroot).12:15
SotKI vaguely remember some trouble with fakeroot when people were looking at bit-for-bit reproducibility, but may be mistaken12:16
SotKthat was faketime, ignore me :)12:17
paulsherwoodafaict the nodes we care about are /dev/console /dev/full /dev/null /dev/urandom /dev/zero - can we assume they are always present on any reasonable host?12:18
paulsherwoodand would it always be ok (and non-leaking) to consume them from the host for builds?12:18
tiagogomesOr could also send a patch to bubblewrap that creates the device nodes instead of fakeroot :)12:18
tiagogomesAnyway. I think devtmpfs solves that problem.12:19
tiagogomesI don't know if it is configured for baserock kernels. Anyone has a running baserock os to check?12:20
locallycompactso I'm in the debris folder for this12:34
locallycompacthttps://gitlab.com/baserock/ybd/builds/527768412:34
locallycompactchroot in and I go12:34
locallycompact~ # ls12:35
locallycompact/bin/sh: ls: not found12:35
locallycompact~ #12:35
locallycompactoh wait12:35
locallycompacthmm yes no ls12:35
locallycompactbin is symlinked to tools/bin and there's an ls in there12:36
locallycompactwhat is setting the $PATH when I do a manual chroot in to a debris folder?12:38
locallycompactand what is symlinking tools/bin to bin and why12:41
tiagogomeshmm12:42
tiagogomesbin shouldn't be symlinked to tools/bin after the build-essential12:43
tiagogomeshave you done any change in definitions?12:43
locallycompactV1012:44
locallycompacthttps://gitlab.com/baserock/definitions/tree/lc/01012:44
locallycompactI need to make it behave now so past behaviour may not indicate anything12:45
locallycompactwhy does /bin symlink?12:46
locallycompactand why does /bin not end up in the path for autoconf12:46
tiagogomesbecause autoconf, in your version is depending on boostrap chunks: stage2-busybox, stage2-binutils, …12:46
tiagogomesyou got the dependencies wrong12:47
locallycompactmaybe I'm not being clear12:48
locallycompactwhy does anything need to symlink /bin ever12:48
locallycompactand why is it not in the path12:48
locallycompactand how does manual chrooting pick up its path from12:49
locallycompactall I need is to make /bin in the path for autoconf the dependencies aren't important12:50
tiagogomesThere is a symlink from /bin to tools so that after all build essential chunks are built, we can just remove the temporary stage 1 and stage 2 binaries that were installed on /tools, and they don't end up in the final root fs.12:50
tiagogomesIf /bin is on $PATH, so /tools will be due the symlink12:51
locallycompact /bin isn't on path, that's the problem12:51
locallycompact /tools/bin is on path12:51
tiagogomeson path of what? The manual chroot or the ybd chroot12:51
locallycompactanything12:52
locallycompacthttps://gitlab.com/baserock/ybd/builds/527768412:52
locallycompactPATH=/usr/bin:/tools/bin:/usr/lib/ccache:/usr/bin:/bin:/usr/sbin:/sbin12:52
tiagogomesthat looks +- correct, as it is /tools/bin that should be on $PATH and not /tools.12:53
locallycompactIt's /bin that i need12:53
locallycompactthat has autoreconf in it12:54
locallycompactno..12:54
locallycompactsorry12:54
locallycompact /usr/bin/autoreconf12:54
locallycompactit's there12:54
locallycompactchroot doesn't find it12:54
locallycompact[root@sunwell tmppkbj1xt6]# chroot .12:55
locallycompact/usr/bin # ./autoreconf12:55
locallycompact/bin/sh: ./autoreconf: not found12:55
locallycompactbut it's there12:55
locallycompactit executes outside of chroot12:55
pedroalvarezmaybe missing libraries?12:56
locallycompacthow does that follow?12:56
tiagogomesThat's because what is not being found is /bin/sh, and not the autoreconf tool?12:57
edcraggit could be a kernel error code being reported back12:57
locallycompactah12:57
tiagogomesAnyway, I suggest that you fix your dependencies. Like I said, autoreconf shouldn't depend on stage2 chunks12:57
locallycompactI don't see how that results in autoreconf not found, sorry12:58
edcraggyou sometimes get not found because it can't find the libs, not the file12:59
locallycompactoh /bin is in that path13:00
locallycompactwhich libs?13:01
mwilliams_ctIn another conversation, pedroalvarez has told me I've found a bug in Trove. Where do I collect my medal?13:01
mwilliams_ctOr more importantly, where do we report bugs these days?13:02
SotKmwilliams_ct: https://storyboard.baserock.org/#!/13:02
* mwilliams_ct shall do so13:02
tiagogomesmwilliams_ct may I gift you with an oxo cube?13:03
mwilliams_cttiagogomes: YES13:03
* SotK wonders which bit of trove has the bug, since he just noticed we don't have a project in storyboard for the trove-setup repo13:04
mwilliams_ctSotK: it's either lorry or lorry-controller13:05
locallycompacthow does somehting look for libs exactly?13:07
tiagogomesYou can find the libraries that a binary requires by running `ldd $binary`13:09
locallycompact$       not a dynamic executable13:09
locallycompact / # ldd /usr/bin/autoreconf13:09
locallycompact$       not a dynamic executable13:09
tiagogomesBy that I take that autoreconf is a shell script13:10
locallycompactoh it's perl13:11
locallycompactand perl is not here for some reason13:11
* locallycompact wonders how this ever worked without autoconf listing perl as a dependency13:16
locallycompactalso help2man13:16
locallycompactno it has help2man13:17
locallycompactbut same error help2man: not found13:17
*** edcragg has quit IRC14:05
*** edcragg has joined #baserock14:11
*** anahuelamo_ has joined #baserock14:11
*** anahuelamo has quit IRC14:12
*** CTtpollard has joined #baserock14:49
tiagogomesThere is the following problem with definitions, every system besides the minimal system, is not including the build-essential-minimal artifacts.15:13
locallycompactAnybody know this error in bison configure.ac:26: error: version 'UNKNOWN' doesn't follow Gnits standards15:15
mwilliams_cthas anyone used qt5 in a baserock system?15:23
mwilliams_ctwe're having troubles dealing with its submodules http://git.baserock.org/cgit/delta/qt5.git/tree/.gitmodules?h=5.7.115:23
SotKmwilliams_ct: what form of trouble?15:24
mwilliams_ctfwiu, the upstream maintainers assume you are using their systems, and therefore have a variety of scripts that initialise submodules assuming their upstream (see eg http://git.baserock.org/cgit/delta/qt5.git/tree/init-repository?h=5.7.1 ) rather than trove15:24
mwilliams_ctthe .gitmodules is assuming that access is available over ssh rather than https15:25
locallycompactyou need to override the urls in definitions15:26
SotKindeed15:26
locallycompactgrep definitions for submodules there's examples15:27
mwilliams_ctdo we have any examples of that?15:27
mwilliams_ctahh15:27
mwilliams_ctok, thanks locallycompact SotK15:27
*** CTtpollard has quit IRC16:03
*** fay_ has quit IRC16:04
*** ctbruce has quit IRC16:15
*** gtristan has joined #baserock16:25
locallycompactanybody got any clue about this https://gitlab.com/baserock/ybd/builds/528969916:36
pedroalvarezconfigure.ac:26: error: version 'UNKNOWN' doesn't follow Gnits standards16:38
locallycompactdoes that clue also come with a solution?16:41
pedroalvareznope, but google drops some results16:41
locallycompactdo any of those results have a solution16:41
tiagogomesjjardon, anahuelamo_ I have mad the jjardon/usr-merge branch to build with success.16:41
* pedroalvarez wonders if locallycompact has even tried to google the issue16:42
locallycompactfor hours16:42
anahuelamo_that's great tiagogomes, thanks!16:42
tiagogomesThere is a fix need for ybd, due not doing artifact splitting properly - or at least how morph does.16:42
locallycompactpedroalvarez, are you being entirely serious16:42
*** anahuelamo_ is now known as anahuelamo16:42
pedroalvarezfirst result, baserock irc logs with the same issue16:42
pedroalvarezseems like we had that in the past16:43
rdaleartifact splitting in ybd appears to have regressed - it certainly worked until recently16:43
locallycompactyes, as usual baserock chat log results of problems without solutions16:43
tiagogomesrdale the gist is that the build-essential-minimal artifacts are not being included in any other system but the minimal-system16:44
* pedroalvarez stops digging through the logs16:44
anahuelamotiagogomes, where can I take a look?16:44
tiagogomesanahuelamo is this blocking you? I still have to do more testing on this before I send a pull request. But if you are blocked by this at the moment, I can push a wip branch somewhere.16:47
anahuelamotiagogomes, I started working on this today, after reading some documentation, so it's not blocking me at the moment16:48
pedroalvarezlocallycompact: second google result points at: https://github.com/scfc/bison-php/issues/1#issuecomment-288846416:50
pedroalvarezand irc discussion points at: no git in the environment16:50
pedroalvarezcommit c6b6225e4f5872ea749fde416bca8e6b89cb5424 in definitions seems to be related too16:51
pedroalvarezalso 7c2bfac98e93e42ffcb23e0d43aee8f3d2c01c6916:53
pedroalvarezI take that c6b6225e4f5872ea749fde416bca8e6b89cb5424  was missing the removal of "--no-git"16:55
locallycompactremove git and add the tarball echo?16:56
locallycompactor remove --no-git16:56
pedroalvarezwe went to that direction and went back to add git16:57
pedroalvarezso, remove --no-git16:57
locallycompacttrying16:57
locallycompactsame16:57
pedroalvarezhm.. try echoing a tarball version file?16:58
pedroalvarezof course, the thing to investigate is why this is working in master of definitions16:59
locallycompacttarball version seesm to work17:02
locallycompactnow with pkg-config17:04
locallycompact /usr/share/aclocal/ltversion.m4:12: warning: ill-formed serial number 'ltversion.m4', expecting a version string with only digits and dots17:04
locallycompactconfigure.ac:12: error: Libtool version 2.2 or higher is required17:04
locallycompact/usr/share/aclocal/libtool.m4:40: LT_PREREQ is expanded from...17:04
locallycompactconfigure.ac:12: the top level17:04
locallycompactautom4te: /usr/bin/m4 failed with exit status: 6317:04
locallycompactaclocal: error: echo failed with exit status: 6317:04
locallycompactautoreconf: aclocal failed with exit status: 6317:04
*** locallycompact has quit IRC17:15
jjardontiagogomes: nice! is the branch in gitlab? so you can run the ci through it?18:25
jjardonmwilliams_ct: the weston-qt system includes qt518:27
jjardon locallycompact seems the m4_esyscmd([build-aux/git-version-gen .tarball-version]) macro in configure.ac is not working; probably because bison is missing the git dependency18:34
*** gtristan has quit IRC19:57
*** vgrade12 has joined #baserock20:32

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!