IRC logs for #baserock for Tuesday, 2016-01-26

*** gtristan has joined #baserock04:37
*** gtristan has quit IRC04:56
*** ctbruce has joined #baserock08:34
*** rdale has joined #baserock09:00
*** bashrc_ has joined #baserock09:06
*** tiagogomes_ has quit IRC09:37
*** tiagogomes_ has joined #baserock09:52
*** ssam2 has joined #baserock09:55
*** ChanServ sets mode: +v ssam209:55
*** jonathanmaw has joined #baserock10:04
*** toscalix has joined #baserock10:13
*** edcragg has joined #baserock10:18
*** Lachlan1975 has joined #baserock10:21
*** gtristan has joined #baserock10:31
*** Zara is now known as Zara__10:31
*** Zara has joined #baserock10:36
* paulsherwood wonders if there are any sandboxlib/container/mount/proc level wizards here able to comment on https://github.com/devcurmudgeon/ybd/issues/149 ... somehow it seems sandboxlib trips over on a busy mount10:45
* paulsherwood glances at ssam2 and richard_maw, for example10:45
paulsherwoodanother example http://paste.baserock.org/aqanuvayoz10:46
* paulsherwood notes that this is the first artifact after stage2, maybe that's a factor?10:48
richard_mawbefore then it's not in full sandbox mode10:48
*** toscalix has quit IRC10:48
paulsherwoodbut i've only seen this in docker/concourse10:48
paulsherwoodtrue, richard_maw10:48
ssam2weird10:49
*** SotK is now known as SotK_10:49
richard_maware Docker containers allowed to mount /proc?10:49
paulsherwoodthat's a fair question, i don't know10:49
ssam2they're presumably in a process namespace, so no reason to deny them mounting their version of /proc in theory10:50
pedroalvarezI believe I have built things using docker10:50
* richard_maw vaguely recalls there being a kernel config option for allowing multiple /proc mounts too, and is checking10:50
paulsherwoodpedroalvarez: using ybd?10:50
ssam2richard_maw: the problem seems to be that it previously failed to *unmount* /proc10:50
ssam210:50
ssam2WARNING: ['umount', '/root/ybd/tmp/tmp4uzeNE/dev/shm'] failed: umount: tmpfs: mountpoint not found10:50
ssam2WARNING: ['umount', '/root/ybd/tmp/tmp4uzeNE/proc'] failed: umount: proc: mountpoint not found10:50
pedroalvarezpaulsherwood: it was morph10:51
ssam2so possibly the directories get deleted while sandboxlib is unmounting them... but i've no idea how that could happen10:51
*** SotK has joined #baserock10:51
pedroalvarezbut I was assuming that they need the same permissions10:51
richard_mawssam2: I was interpreting that as just that the unmount logic wasn't accounting for whether they were successfully mounted in the first place10:51
richard_mawssam2: since later in the http://paste.baserock.org/aqanuvayoz log it prints out the exception that the mount failed10:52
ssam2hmm10:53
ssam2the tempdir names are the same though10:53
ssam2/root/ybd/tmp/tmp4uzeNE/proc and /root/ybd/tmp/tmp4uzeNE/proc10:54
ssam2looks like the 1st command succeeds but the 2nd command fails10:54
*** locallycompact has joined #baserock10:55
richard_mawssam2: I wouldn't assume that jut because the unmount result is printed earlier, that it was executed earlier10:55
richard_mawI've seen something similar happen when user namespaces were involved, but I don't think docker does those10:56
locallycompactIs this about the concourse bug?10:58
locallycompact'cause concourse doesn't use docker10:59
paulsherwoodi thought it did?11:01
locallycompactconcourse uses https://github.com/cloudfoundry-incubator/garden-linux11:01
paulsherwoodah11:01
locallycompactyou can pull docker fs images into that11:01
paulsherwoodmaybe sandboxlib needs to assert what happens on the mount?11:02
richard_mawpardon?11:02
radiofreessam2: ah, bit of an issue i have with sandboxlib btw11:02
radiofreeif XDG_RUNTIME_DIR isn't set to something sensible, then the shm tmpfs is mount-bound to a folder... not on a tmpfs (the one set by ybd)11:03
radiofreewhich causes issues for some things11:03
radiofreecould it fall back to just use /tmp if XDG_RUNTIME_DIR isn't set?11:03
radiofreehttp://git.baserock.org/cgit/delta/sandboxlib.git/tree/sandboxlib/linux_user_chroot.py#n5211:04
paulsherwoodrichard_maw: never mind. i'm trying to multitask, failing.11:05
richard_mawwell, you may be interested to learn that concourse uses user namespaces11:07
locallycompactrichard_maw, what does that mean exactly?11:07
richard_mawthat there's many more restrictions of what can work11:08
locallycompactrichard_maw, so is there anything I can do to make this work?11:12
ssam2raidofree: i don't really follow, could you write it up as an issue report?11:12
ssam2or better yet, do a patch? :-)11:13
radiofreessam2: ok!11:13
richard_mawlocallycompact: unsure yet, I think the issue may be that the container's /proc is more locked down than the procfs it's trying to mount into the subcontainer11:16
richard_mawotoh I'm not sure whether linux-user-chroot works when inside a container, even with user namespacing11:17
* richard_maw recalls interesting interactions with suid binaries and user namespaces11:17
richard_mawhm, if they are privileged containers then they don't do user namespacing11:19
richard_mawok, the problem may be that in your container you only have /proc mounted read-only11:26
richard_mawso you're not allowed to mount one as read-write11:26
richard_mawwhich sandboxlib is doing, by not explicitly requesting a read-only mount of /proc11:26
richard_mawnote that bind-mounting /proc into the container would also work11:26
richard_mawprovided your bind-mount keeps all the flags that were set11:27
richard_mawso if your /proc is mounted NOSUID or RDONLY then you need to set those in the mount options11:27
locallycompactit looks mounted rw to me11:31
richard_mawhm, privileged containers?11:33
locallycompactnot sure how to determine that11:34
richard_mawI think it's a config option11:34
locallycompactIf it's in garden linux I'm not sure I have that available11:35
* ssam2 wonders if Garden could one day replace Sandboxlib11:39
richard_mawhm, from what I could infer from the code (difficult given I'm unfamiliar with go), it looks like they must be, but you can detect whether your containerisation is user namespaced by whether /proc/self/ns/user is the same both inside and out11:39
richard_mawssam2: it's a big pile of go that I'd prefer to avoid if I could11:40
* richard_maw has to get back to his work now11:40
ssam2it does have a component named "Gardeners Question Time" though..11:40
locallycompactrichard_maw, which part were you looking at?11:44
richard_mawlocallycompact: mostly https://github.com/cloudfoundry-incubator/garden-linux/blob/master/containerizer/system/namespacing_execer_linux.go#L34 and https://github.com/cloudfoundry-incubator/garden-linux/blob/master/containerizer/initc/main.go#L7311:48
locallycompactthx11:49
benbrown_if by privileged you mean "privileged: true" in the concourse yaml, that results in /proc/self/ns/user being the same both inside and out11:49
richard_mawbut I can see I misread the read-only mounting of /proc now11:49
locallycompactI thought that just meant sudo11:49
locallycompactbenbrown_,11:49
benbrown_locallycompact: not sure, docs just say "the task will run as root with full capabilities"11:51
* richard_maw recommends haivng a play around in a garden container and mounting /proc to see whether you can trigger the mount failure yourself11:51
perrylssam2: bit belated, but glad to see the baserock url changes! :) sorry i've not had more time to look at the cgit work, i've been a bit swamped as of late :(12:05
ssam2i nearly had the hyperlink thing going too but i deleted all my changes with `git reset --hard`12:10
perryloh no :(12:15
pedroalvarezgit reflog?12:20
ssam2it was uncommited changes12:23
ssam2not very smart!12:24
ssam2cgit is going to be a bit broken for a little while13:38
ssam2on git.baserock.org13:38
ssam2ok, good news!13:40
ssam2it's a bit of a hack so far, but strata link to their contents now: http://git.baserock.org/cgit/baserock/baserock/definitions.git/tree/strata/apache-httpd-server.morph13:40
benbrown_ssam2: !13:42
benbrown_that's really cool!13:43
pedroalvarezit is!13:46
radiofreessam2: awesome!14:04
pedroalvarezLeaving here a broken file with that new feature: http://git.baserock.org/cgit/baserock/tests/bad-morphs.git/tree/bad-json.morph14:04
radiofreethat change is one of the most useful things ever14:05
richard_mawooh14:05
radiofreessam2: any chance it could work with the morph: filed as well? :)14:05
ssam2radiofree: yes, done now :-)14:22
ssam2thank perryl for the initial work14:22
tiagogomes_this was very cool, but I wonder why those changes didn't go to gerrit14:22
tiagogomes_is infra work not being reviewed in public anymore?14:23
perrylssam2: looking brilliant! :)14:23
ssam2tiagogomes_: it will go into Gerrit14:23
ssam2tiagogomes_: but i wanted to get it working first14:24
ssam2tiagogomes_: There's a chicken-and-egg problem if I submit a change without actually deploying it first, because who knows if the patch actually works?14:24
tiagogomes_yes, but git.baserock.org shouldn't be the test playground14:24
ssam2tiagogomes_: are you volunteering to join the infra team?14:24
* tiagogomes_ shuts up14:25
ssam2you would be welcome to join :-) me and pedro and gary don't seem to ever have much time to spend on it14:25
franredwow!! ssam2, great job! pretty useful!14:46
pedroalvarezssam2: found a couple of issues. I have no idea how difficult is to solve them but just in case there are quick fixes:14:58
pedroalvarez- Subsystems in clusters don't work: http://git.baserock.org/cgit/baserock/baserock/definitions.git/tree/clusters/initramfs-test.morph14:58
pedroalvarez- Links to other files (morph field) will always go to the master branch even if you were looking into a different branch or specific commit14:58
pedroalvarezsecond one might be difficult to solve14:58
locallycompactthis mako chunk's repo field ends in .git and so links to mako.git.git http://git.baserock.org/cgit/baserock/baserock/definitions.git/tree/strata/python2-core.morph14:59
pedroalvarezgood point15:00
pedroalvarezshould the plugin handle that case, or should we fix definitions?15:01
ssam2locallycompact: good catch. thanks15:06
ssam2plugin can easily handle that case15:06
*** locallycompact has quit IRC15:20
ssam2subsystems, repos ending in .git, and files from old commits should now all work15:39
ssam2i don't like that links to files that don't exist are still linkified, but it'd be a bit of effort to check if they do before creating the link..15:40
ssam2would need to actually run a git command15:40
pedroalvarezno need for such a thing15:40
pedroalvarezwell, Sam is now the contributor of the month15:41
pedroalvarezand from 2016 so far :)15:41
*** ssam2 has quit IRC15:56
*** ctbruce has quit IRC16:17
*** locallycompact has joined #baserock17:05
*** rdale_ct has joined #baserock17:06
*** rdale has quit IRC17:07
*** faybrocklebank has quit IRC17:22
*** toscalix has joined #baserock17:23
*** trn has quit IRC17:28
*** trn has joined #baserock17:48
*** bashrc_ has quit IRC18:01
*** jonathanmaw has quit IRC18:04
*** locallycompact has quit IRC18:24
*** Lachlan1975 has quit IRC18:38
*** mwilliams_ct has quit IRC18:42
*** edcragg has quit IRC18:44
*** toscalix has quit IRC20:04

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!