IRC logs for #baserock for Tuesday, 2014-11-18

*** cosm [~Unknown@cspc154.cs.man.ac.uk] has quit [Ping timeout: 255 seconds]00:17
*** cosm [~Unknown@host-78-150-56-250.as13285.net] has joined #baserock00:35
*** petefoth [~petefoth@host-78-151-29-232.as13285.net] has joined #baserock07:07
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock07:25
SotKpaulsher1ood: I made the commits on the system I was testing Mason with and forgot to fix my git config, I can fix it and force push if its an issue08:15
paulsher1oodSotK: it would be an issue at the point of seeking reviews... so if these patches are expected to survive, probably yes, worth fixing sooner rather than later?08:16
*** tiagogomes [~tiagogome@213.15.255.100] has joined #baserock08:17
SotKI wasn't expecting these patches to survive until review :)08:19
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:22
pedroalvarezjjardon: no worries, I just wanted to know how worried should I be :) 08:29
pedroalvarezjjardon: btw, well done with systemd! 08:30
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:37
*** rdale [~quassel@92.Red-88-20-2.staticIP.rima-tde.net] has joined #baserock08:39
paulsher1oodSotK: np08:44
jmacsI want to make a change to ceph.configure which is in the root of git://git.baserock.org/baserock/baserock/definitions; do I need to use morph edit to do this, or is it something else now?08:47
paulsher1oodjmacs: you can just edit that file in your baserock/baserock/definitions branch08:48
paulsher1oodthen run build/deploy08:49
paulsher1oodthese days i tend to just work in a constant /src/workspace/mymachine/baserock/baserock/definitions checkout. 08:49
pedroalvarezyes, easy like that, and not morph commands involved08:50
jmacsSure, where would I push that change? I want to work in public08:50
paulsher1oodif you have push access to gbo... branch baserock/jmac/foo08:50
paulsher1oodotherwise, github?08:50
paulsher1oodor your favourite git host08:50
jmacsOK.08:53
* paulsher1ood should write up the rest of his current 'workflow'08:56
jjardonjmacs: I used to use Gitorious before I get gbo git access09:04
radiofreegitorious' uptime has been a bit dodgy lately09:06
*** mariaderidder [~maria@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:07
jmacsIt's OK, I've finally remembered my github password09:07
jmacsCan you push a repository that doesn't already exist on github?09:16
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:17
straycatSotK, *facepalm*09:34
straycatthanks for pointing that out, I'll sort the deploy commands out when I get a moment09:35
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:35
Mode #baserock +v ssam2 by ChanServ09:35
*** rdale [~quassel@92.Red-88-20-2.staticIP.rima-tde.net] has quit [Ping timeout: 255 seconds]09:45
*** Krin [~mikesmith@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:47
*** rdale [~quassel@92.Red-88-20-2.staticIP.rima-tde.net] has joined #baserock09:52
jmacsHmm, no, I can't seem to clone a baserock repo into github or push a new branch. Someone must have done baserock work on github before, right?09:57
Kinnisonjmacs: There's no reason you couldn't push a brand new (to github) piece of content09:57
Kinnisonjmacs: Have you created a repo to push it into?09:57
jmacsNo; do you think it would help if I made a blank repo to overwrite?09:58
KinnisonYes you have to make the repo09:58
Kinnisongithub hangs all its access control off the repo existing09:58
Kinnisonjust tell it you want a blank repo (i.e. without licence or readme)09:59
Kinnisonand it should work09:59
jmacsIt worked, thanks Kinnison 10:00
KinnisonNo probs10:01
robtaylorgitlab public rather than github though - keep it libre!10:01
jmacsBit late for that10:02
robtaylorah well10:08
jmacsI see your point, though.10:08
* robtaylor is making a concerted effort himself on such things10:10
* Kinnison write his own F/LOSS git hosting solution first10:13
Kinnison:-)10:13
radiofreepaulsher1ood: how do i actually use a docker?10:13
radiofreea docker? docker? a docker container?10:13
radiofreei built a genivi image on a jetson, have installed docker stuff, want to see if i can run weston in the docker container10:14
radiofreeis that something i'd want to do?10:14
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Quit]10:15
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:15
robtaylorrichard_maw: i think a more capable pivot_root would maybe look like pivot_namespace and pivot_cgroup10:16
ssam2radiofree: I'm not sure how much use Docker is for graphical stuff10:17
ssam2containers are intended to communicate through the host with sockets10:17
ssam2although you can bind in device nodes and whatever if you want to get funky10:18
ssam2but then I'm not sure it can /share/ the device with the host, it'd need to take over graphics completely10:19
radiofreessam2: yeah i saw that, i saw some discussion on it being usable10:19
radiofreeerm.. usable for CUDA at least10:19
ssam2oh, cool10:19
radiofreei'm thinking i could share the drm device, then.. magic?10:19
ssam2it's worth a try, anyway :)10:20
ssam2I'd start with `docker run --device=/dev/drm:/drv/drm-host --interactive --tty --rm myimage /bin/bash` or something 10:21
radiofreei see, how do i get to "myimage", is that just a folder?10:21
ssam2that needs to be a docker image10:21
richard_mawrobtaylor: the former pivots all processes in one namespace into another and the latter pivots all processes in a cgroup into another?10:22
radiofreessam2: how would one go about turning a rootfs tarball into a docker image?10:22
ssam2radiofree: you can import a tarball by doing: `cat system.tar | docker import - mynewimage`10:22
radiofreeooh10:22
ssam2it should then show up in `docker images`10:22
robtaylorrichard_maw: maybe more like moves a non root namespace to the root namespace, ditto chgroup10:22
robtaylor*cgroup10:22
richard_mawmy brain hasn't sufficiently warmed up enough yet to be able to join that up with how to atomically migrate processes into a new version of their mount tree10:26
franredI want to add rebar and erlang-sd_notify to the Trove would be better if they are under erlang.lorry or do you want me to create a erlang-packages.lorry ?10:38
franrednote: rebar is a erlang compiler10:39
pedroalvarezah..  you mean you want to lorry new things10:42
franredpedroalvarez, yes10:43
paulsher1oodradiofree: you could try 'docker pull baserock/14.29'10:43
paulsher1oodhttps://registry.hub.docker.com/u/baserock/14.29/10:43
* paulsher1ood will push a 14.46 at some point10:43
* franred will lorry the erlang packages creating erlang-package.lorry file10:46
franreds/package/packages/10:46
paulsher1oodcan/did we standardise naming for strata to contain 'foo and common things that use foo or are required by foo' ?10:49
paulsher1oodeg ruby-stuff, erlang-stuff, go-stuff etc?10:49
franredpaulsher1ood, it is better to standarize for lenguage packages, because requirements could be a little bit mess10:54
franredfor example this erlang packages that I want to lorry are for rabbitmq which is a python package which is required by openstack... but why could not be required for other package/s in the future? 10:55
paulsher1oodfranred: i'm not fully understanding you. but in any case, the word 'package' is not currently used in baserock i think, and we should continue to avoid it?10:56
rdale'ruby environment'?10:58
franredpaulsher1ood, it is not used in baserock but for refering to them we should continue use "python-packages" "erlang-packages" because their systems/users call them in that way and will be easy for us to find them10:58
paulsher1ooddo others agree? i'm in disgreement with franred here i think10:58
SotKwe currently have a python-packages.lorry file, so erlang-packages.lorry makes sense to me11:00
richard_mawpackages have implications that I didn't fully understand until I needed to think about how to do online updates without them11:01
* paulsher1ood surrenders to SotK's logic11:01
* richard_maw doesn't11:01
richard_mawthey should be python-modules IMO11:01
paulsher1oodworks for me :)11:02
rdaleSotK is talking about names of lorries though not names of strata, although maybe they are always supposed to be the same11:02
franredthat is also my point rdale11:02
* SotK agrees that we should *not* have a foo-packages stratum11:04
* franred renames erlang-packages.lorry to erlang-modules.lorry11:05
pedroalvarezrdale: re qtwebengine-chromium. How big is it?11:05
straycatwhat's wrong with erlang-packages ?11:05
straycatwe have python-packages11:05
*** fay_ [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 255 seconds]11:06
rdaledoes it have the python interpreter in it?11:06
straycatno that's in core11:06
franredFrom Erlang documentation: Erlang code is divided into modules  - so I don't have any reason to continue called erlang-packages11:07
straycatokay11:08
*** fay [~fay@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock11:09
ssam2paulsher1ood: have you pushed ARM images to the Docker registry? if not, 'docker pull' will not be much help to radiofree11:09
fay is now known as fay_11:09
rdalepedroalvarez: i've just checked out the qtwebengine-chromium git repo and it is 1.2G11:20
paulsher1oodjmacs: nice work on https://vimeo.com/112157682 :-)11:22
paulsher1oodssam2: ah! no11:23
jmacsI'm not so happy with it11:23
paulsher1oodjmacs: why?11:23
* paulsher1ood has produced a reasonable number of films, and feels qualified to +2 it11:23
jmacsI described a build result as an 'image' and there's background noise in the audio11:24
paulsher1oodjmacs: it's readable, and the audio is clear11:24
jmacsMostly, yes, but there's a big thunk from a door slamming near the end11:25
paulsher1oodjmacs: i think your artistic temperament is showing :-)11:26
paulsher1oodseriously, it's easy to follow what you're doing, does the job. 11:26
jmacsOK, well I can't really argue with your judgement on films, so thank you.11:26
paulsher1oodheh :)11:27
rdaleit looks fine to me to - audio ok, and i could read the text on the terminals11:28
paulsher1oodjmacs: one question - why is the deploy so slow?11:29
jmacsGood question. I haven't had a chance to look into it yet.11:30
paulsher1oodnp11:30
paulsher1oodfranred: i've +1 your lorries... you need 1 more...11:31
franredpaulsher1ood, cheers11:32
straycatfranred, where's the patch?11:33
paulsher1oodon the ml11:33
franredstraycat, on the mailing list11:33
straycatfranred, so we don't strictly speaking need both rabbitmq repos, but if we have the fork we may as well carry the original?11:37
franredstraycat, we may or may not so I would like to lorry both just in case anyone wants to use the original for other reasons in the future11:38
straycatmkay11:38
franredwe don't know if/how the fork get synchronized11:38
straycatlooks fine to me, +111:39
franredcheers11:39
paulsher1oodfranred: notice ssam2's comment before merge please11:40
* paulsher1ood was too quick with his +111:40
straycatyeah i agree with ssam211:41
straycati actually misread and thought the fork was fork_rabbitmq >.>11:43
* DavePage wants rabbit on a fork now :(11:43
petefothA welsh one for me please11:46
pedroalvarezhm... I've just deployed a system with new systemd and it doesn't have network connection 11:48
Kinnisonis it missing the unit we wrote to do the ifups ?11:49
pedroalvarezi believe that Javier said that it wasn't needed11:50
KinnisonOh11:50
ssam2I don't think that unit is needed, jjardon gave an example of the new configuration required11:50
ssam2I forget where though11:50
pedroalvarezssam2: thanks, I didn't know about that11:51
jjardonpedroalvarez: check /etc/systemd/network11:51
* ssam2 finds http://www.freedesktop.org/software/systemd/man/systemd.network.html11:51
pedroalvarezjjardon: it's present in the system11:51
jjardonpedroalvarez: the default configuration will only configure the ethernet interfaces with dhcp11:52
pedroalvarezjjardon: so "Name=en*" should only match in ethernet interfaces?11:53
paulsher1oodssam2: you mentioned on the list 'we may as well use the btrfs storage backend'... but https://blog.docker.com/2013/05/btrfs-support-for-docker/ ?11:53
paulsher1oodis btrfs officially supported?11:54
robtayloryep11:54
jjardonpedroalvarez: yeah11:54
paulsher1oodoh, cool. is there some magic, robtaylor ?11:54
* paulsher1ood googles harder11:54
richard_mawyou also benefit from btrfs even if you haven't got docker configured to use the btrfs backend, since the vfs backend uses btrfs' clone ioctl to do fast file copies11:55
ssam2paulsher1ood: in the 18 months since that blog post there has been some progress :)11:55
robtaylorpaulsher1ood: google 'docker graph backends'11:55
paulsher1oodssam2: i expect so. but google 'docker btrfs' doesn't really show it :-)11:55
pedroalvarezjjardon: hm.. that didn't work with eth0 in openstack.. 11:55
paulsher1oodrobtaylor: thanks11:55
richard_mawpedroalvarez: hm, it sounds like you're not getting the deterministing device naming then11:56
* richard_maw can't remember how that is configured11:56
richard_mawah, pedroalvarez: look in /etc/udev/rules.d for persistent-net.rules11:57
jjardoneth0 doesnt exist in systemd11:57
jjardonpedroalvarez: ^11:57
* paulsher1ood was meaning to say, updating to latest systemd seems to drop his network config 11:58
paulsher1ood(still working network, but not the expected eth0 and fixed ip)11:58
richard_mawpaulsher1ood: ah, your custom networking setup would need to be translated to networkd config, rather than /etc/network/interfaces11:59
richard_mawI don't think anyone here has much experience with it though11:59
paulsher1oodrichard_maw: aha, thanks12:00
richard_mawjjardon: we need to keep things working between updates, can we have networking configuration which will work for eth0?12:01
jjardonpaulsher1ood: check /etc/systemd/network, this is the only thing you need: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/strata/foundation/systemd.morph#n2112:03
jjardonto configure all the ethernet interfaces with dhcp12:04
richard_mawjjardon: that only works if you've got the deterministic interface nameing12:04
richard_mawif you've still got the config file in udev that makes it stay as eth0, then that config won't work12:05
jjardonrichard_maw: remove that udev file then?12:05
richard_mawthat's a manual step for updating though, as-is if anyone upgrades rather than re-deploying, they have a broken system12:06
pedroalvarezrichard_maw: what file is this? I only have one under rules.d which contains 'KERNEL=="fuse", MODE="0666"'12:06
richard_mawif we'd put all these config files in /usr, rather than /etc we wouldn't be having this problem12:07
jjardonpedroalvarez: whatd the output of ifconfig -a?12:07
pedroalvarezjjardon: http://paste.baserock.org/ozemejajas.sm12:08
* Kinnison notes that with the new systemd this means the simple network configure extension will no longer work as expected12:08
Kinnisonbreaking virtualbox deployment etc12:08
pedroalvarezjjardon: note, I've modified the networkd config and restarted12:08
jjardonKinnison: that extension has been removed from all the systems with systemd12:09
richard_mawpedroalvarez: ah, what we've actually got for our systemd config to prevent it renaming devices is we emptied 80-net-name-slot.rules12:09
Kinnisonjjardon: without a replacement offered?12:09
richard_mawpedroalvarez: http://git.baserock.org/cgi-bin/cgit.cgi/delta/systemd.git/commit/systemd.morph?h=baserock/morph&id=d7498df6813b97f60aa04efc16c561804f30a9dc12:09
jjardonKinnison: networkd was the replacement12:10
pedroalvarezjjardon: I believe that Kinnison is talking about this: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/tree/morphlib/exts/simple-network.configure12:12
pedroalvarezrichard_maw: thanks! so by removing that file we avoid the renaming of (e.g) eth* to en*?12:14
jjardonpedroalvarez: yeah, I think we are talking about the same? http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/commit/?id=54f36e8b4a5a58d872980c13aba5bcc4726283f912:15
pedroalvarezjjardon: oh, yeah sorry12:16
pedroalvarezbut yes, I think we should test the virtualbox deployment and check if a replacement is needed12:17
pedroalvareznote that I've no idea what networkd is doing now, and if simple-network is not needed anymore12:18
jjardonpedroalvarez: "systemd-networkd is a system service that manages networks. It detects and configures network devices as they appear, as well as creating virtual network devices." http://www.freedesktop.org/software/systemd/man/systemd-networkd.service.html12:20
robtaylorshould do a much better job thanb simple-network ;)12:26
robtaylorrichard_maw: would you like to dicsuss those ideas from yesterday further at some point?12:27
radiofreei upgrade from a factory image to a new systemd image and the network worked for me12:27
radiofreeit was en*12:27
jjardonuh, ok to commit this? we are compiling master of libndp: http://paste.baserock.org/ahujovigud.sql12:30
radiofreelooks ok to me12:30
franredjjardon, looks ok to me too12:31
rdalecan unpetrify-ref: refer to a tag as well as a branch?12:32
franredrdale, yes12:32
rdaleok thanks12:32
ssam2jjardon: +112:33
jjardonradiofree, franred thanks, pushed12:33
jjardonsorry ssam2 , too late to add you!12:34
* ssam2 sulks12:36
paulsher1oodradiofree: it worked for me, but not with the config i was expecting ;)13:04
richard_mawrobtaylor: sure, you just happened to ping me while I was at lunch13:05
richard_mawjjardon: simple-network is still needed for things like VirtualBox deployments, where you sometimes need static IPs.13:06
richard_mawTo do this properly we need a migration strategy to convert /etc/network/interfaces into /etc/systemd/network and to fix up simple-network to generate /etc/systemd/network config if networkd is available13:07
richard_mawwe can drop the former if we declare we don't care about upgrading, but we need the latter, rather than your patch to drop simple-network.configure13:08
richard_mawIMO simple-network.configure should be generating the /etc/systemd/network config, rather than putting it in the chunk13:09
richard_mawwe should avoid putting files in /etc during chunk builds when we could put them in /usr/13:09
ssam2i didn't realise that, I apologise for saying in review that only the minimal-system would be affected13:10
richard_mawit's intricate, I don't blame anyone for making this mistake, but we need to fix it13:12
Kinnisonjjardon: networkd is not a drop-in replacement for simple-network.configure13:16
Kinnisonjjardon: in fact, networkd clearly needs just as much configuration13:16
jjardonrichard_maw: yeah, I agree with you simple-network.configure should be generating the /etc/systemd/network/*.network files13:22
jjardonKinnison: what use cases baserock used to support apart from configuring the ethernet network interface with dhcp?13:23
richard_mawstatic ip config13:24
Kinnisonpaulsher1ood will likely be very sad if he can't configure interfaces statically like we used to for multi-virtualbox deployments13:24
paulsher1oodKinnison: actually, i'll live...13:25
jmacsI'll also be sad if I can't set up ceph clusters that way13:25
pedroalvarezi'm sad because openstack instances read the cloud-config script doing a GET request to an http server13:27
pedroalvarez(impossible to do without network)13:27
jjardonrichard_maw: any system with a example of that config in definitions?13:29
franredcould someone have a look what happens with the erlang-modules lorry file? how could I check why is failing to clone it? (they are not in g.b.o)13:30
franreds/\? /\? or/13:31
richard_mawfranred: I'll have a poke around13:31
pedroalvarezjjardon: there is an example of static ip here: http://wiki.baserock.org/devel-with/#index5h2 13:32
pedroalvareznot sure if was that what you were asking for13:32
franredrichard_maw, cheers13:32
jjardonmmm, I think a easy way to have <eth0> back is using a .link : http://www.freedesktop.org/software/systemd/man/systemd.link.html13:35
jjardonbut let me investigate more13:35
richard_mawfranred: your lorry file is invalid json, because you missed out the " on the sd_notify name13:40
franredrichard_maw, :/ ouch, sorry, Im going to fix this13:41
franredand cheers13:41
jjardonpedroalvarez: ok, thanks. So none of the systems in definitions is broken, rigth?13:45
*** jjardon [sid723@gateway/web/irccloud.com/x-kcnlxhpledlirbpg] has quit []13:55
*** jjardon [sid723@gateway/web/irccloud.com/x-hioklikdbudulumq] has joined #baserock13:56
richard_mawjjardon: maybe, but definitions doesn't contain the sum total of deployments. paulsher1ood used to do custom NETWORK_CONFIG13:59
paulsher1oodrichard_maw: correct14:02
paulsher1oodbut actually, my customization is manual, so not sure that baserock needs to support it by default?14:02
robtaylorrichard_maw: when would be good for you?14:19
richard_maw20-25 minutes from now, I've got a bunch of stuff I need to do now, then a short meeting14:20
robtaylorrichard_maw: cool, that should be good for me. ping when you're ready14:21
*** mwilliams_ct [~mikewilli@access.ducie-dc1.codethink.co.uk] has joined #baserock14:27
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Read error: Connection reset by peer]14:29
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock14:29
Mode #baserock +v ssam2 by ChanServ14:29
richard_mawrobtaylor: ping14:44
robtaylorkpong14:45
richard_mawI didn't know there was an in-kernel implementation of that14:45
robtaylorof?14:45
richard_mawpong14:45
richard_mawlame joke14:45
richard_mawanyway14:45
robtaylorah hah :)14:45
robtaylorso, does my crazy idea still seem cool after a nights cognition?14:46
richard_mawwe could have systemd do a pivot and re-exec, but then the entire userland needs to be able to handle graceful re-exec14:46
robtayloryeah14:47
richard_mawfreezing, pivoting and using gdb to make processes re-open directory fds might work, but that's just scary14:47
robtaylori'd have it so pid 1 is a syustsem d, but with only one unit, that nspawns the rest of the sysystem14:47
robtaylorwhen you want to run a new system, you ask pid 1 to nspawn the new system, and we have some way to handover the sockets between 1st container and the 2nd14:48
robtaylor(probably by the systemd in 1 communicating with the systemd in 2 - you can bindmount the systemd socket from root into 1, root into 2, and 2 into 114:49
robtaylorthen bring down 1 when it's quiesecent (maybe with a snapshot before killing for added live rollback magic)14:50
richard_mawyou can socket activate through nspawn, so for a subset of the available services you don't need to do any clever socket migration14:50
robtaylorooh yes14:51
robtayloryou just have the root sytemd own the sockets, and activate in the container14:51
robtaylordon't think that'll work out of the box - i think it just supports starting a new container for a given socket activated service at the moment14:52
richard_mawhm, yeah, you'd need to have one service nspawn, and the rest depend on that, and nsenter14:54
robtayloryeah, that sounds like a good way to prototype it14:55
robtaylorits mostly doable in shell, i think14:55
richard_mawI just fear there's be little interest in actually doing the work required for atomic online update if the majority of userland needs patching to support graceful re-exec14:55
robtaylorhmm14:57
robtaylorit may be possible to make it work nicely14:57
robtayloryou know how systemd delays the accept until the service is up and running?14:58
robtaylorit could delay accept until the old service is down and the new service is up14:58
robtaylor(if a service can't be graceful with itself)14:59
robtaylorI epcet apache will just work anyhow14:59
robtaylorif you graceful-stop in conatiner 1 and start it in conatiner 215:00
robtaylorand that's the usecase everyone cares about =)15:00
robtayloractually, any service that support graceful restart should be able to cope with itself being launched while its still gracefully shutting down15:00
robtaylorotherwise there'd be a bunch of cases in the normal packaged world that would be problematic15:01
richard_mawI'd like if we could handle desktop and shell server cases too15:02
robtaylorhmm15:03
ssam2rdale: are you blocked on having fribidi and crman and qtwebengine-chromium merged? I guess I can give you a +2 if so and merge the thing now15:04
*** apespace [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock15:04
robtaylorrichard_maw: how would you see desktops working?15:04
richard_mawrobtaylor: terminals could re-exec with the scrollback, pwd and env set, but there's a lot of state that would need the shell to support it natively15:04
rdalefribidi and bullet3 are needed for enlightenment, which is part of the qt5 system. the qtwebengine-chromium isn't needed until qt5.4 is released at the beginning of december15:05
robtaylorI think if you have a service still runnig in the old system, and you want to leave it running, that's up to you15:05
robtaylorrichard_maw: thats just the same case as a normal packaged system15:05
robtaylorrichard_maw: if im running irssi in a screen and i upgrade package upgrade irssi, it doesnt reload itself15:06
robtaylorand its got stuff loaded in memory that's still on disk15:06
robtaylorthere's no memory or cache cost to that being the last process running in a container against it being the last process running with the old binaries mmaped15:08
*** genii [~quassel@ubuntu/member/genii] has joined #baserock15:08
richard_mawthe weird bit is that user-directed file-system browsing is different to the packaged online update method, as the working directory and any directory fds are still referring to the old version15:08
robtayloryep15:08
richard_mawso after initiating an atomic update, the shell in your sshd doesn't see the update unless you log back in15:09
robtaylorhmm, can we just have a way to setns things you want to migrate?15:09
robtaylorso old stuff is still in the old cgroup, so you know they're still 'tainted', but can be pulled into the new namespace15:11
richard_mawjust changing the mount namespace they're in is not sufficient on its own, since cwd, root and open fds still refer to the old version, independent of the namespace15:11
robtaylordesktops, it think you just have a 'apply upgrade' popup (which in the new system bring the desktop up to gdm in the background, then pressing apply upgrade just logs you out and switches to teh new system)15:12
richard_mawif you wanted to move processes around you'd probably have to inject code to get it to reopen directory fds, but you can change cwd and root with pivot_root in most cases15:12
robtaylorrichard_maw: you can't pivot_root arunning process15:13
robtaylorrichard_maw: at least not when i last saw it15:13
* richard_maw looked at the code and couldn't see anything preventing it working for running processes15:14
robtaylorthe syscall is for a process to pivot_root itself, no?15:14
richard_mawhistorically, and currently if I understand it correctly, it moves multiple processes. The manpage sugggests that you shouldn't rely on it working for other processes as it's functionality that may be deprecated15:15
robtaylorrichard_maw: man 2 pivot_root15:16
richard_mawyep, been there, that's where it warns that it may not change other processes15:17
richard_mawI'll have to check what do_each_thread iterates over though.15:18
richard_mawi.e. whether it's _every_ thread in the system, or just those associated with the process15:18
robtaylorits just the process15:18
robtaylorthat bit you're reading is 'make sure that when you kill you're old rootfs you don't have any processes left using it'15:19
robtaylorbut that doesnt really matter so much on a modern system15:19
richard_mawI think I've got older manpages on my Debian system, since it explicitly says it may or may not change it for all processes in the namespace15:20
richard_mawok, so they've deprecated that finally15:20
robtayloranyhow, you *can* do all that with setns15:21
robtaylorbut yeah, moving things between filesystems without their cooperation is going to get messy15:22
richard_maware you suggesting we can setns another process, or are we relying on cooperation?15:23
robtayloryou *can* setns another process15:23
robtaylorbut it's cwd will be interesting, i expect15:23
richard_mawusing setns on another process is news to me15:24
robtaylorthis may rewuire some kernel reading to figure the nitty gritty15:24
robtayloroh, seems i've imagined that15:24
richard_mawso we're on either injecting code into the frozen process to make it move itself, writing a syscall to move every process, or having systemd pivot and restart the processes, which isn't much better than running the apps in containers15:28
robtayloryep15:28
richard_mawsyscall is unlikely to be accepted, as it's probably a massive security risk15:28
robtayloryep15:29
richard_mawinjecting code is scary, but potentially workable15:29
robtayloruse criu?15:29
rjekptrace15:29
rjekI think you can use ptrace to execute system calls on behalf of other processes15:29
richard_mawI've done it before with gdb15:30
rjekYou can certainly use gdb to... yes15:30
robtayloryep, ptrace15:30
richard_mawwhich is how criu does it, but actually using criu for this may be overkill, since AIUI you'd have to serialise and deserialise, and you'd probably lose the sockets you were using in the process15:31
robtayloryou can probably modify it to do less15:32
robtaylorbut i'd see that as definitly secondary15:32
rjekAlthough, won't the process still have file handles open to files in the old root?15:32
rjek(Please tell me if I've got the wrong end of the stick)15:32
robtaylora 'nice to have' tool that you can use to move a process from the old system to the new15:32
robtaylorrjek: indeed15:33
richard_mawrjek: you are correct15:33
robtaylorits all sorts of scary15:33
rjekYou can probably do something evil for files it has open read-only by freopening them15:33
richard_mawrjek: but that's not really any more of a problem than a package based online update15:33
robtaylorand potential security risks..15:33
rjekAnd just hope the process isn't keeping state :)15:33
rjekrobtaylor: I suppose in packaged-based systems the packager knows if they need to restart the service, or just send it a HUP for it to gracefully reload.15:33
rjekAnd that information is carried in the upgrade itself.15:34
richard_mawunless it's firefox, which notices if it's referring to files that no longer exist15:34
rjekYeah, some things behave quite nicely in that respect.15:34
rjekLots of web frameworks do it when in development mode, but don't in production.  Which kinda makes me sad, as it means a brief glimer of downtime.15:35
robtaylorI kinda see the case richard_maw wants to support though. It'd be good if you could do the sort of 'upgrade this thing, test if my stuff works now' that package systems allow you to do15:36
rjekNod.15:37
richard_mawthere's also the question of whether you _can_ run injected code in a frozen process15:38
rjekrichard_maw: ptrace, then unfreeze, then inject?15:39
rjekAssuming a) you can ptrace a frozen process, and b) the process promptly does something that you can trap and trigger on.15:40
straycatOur setuptools bootstrap stuff differs from what's suggested in their READMEs, I guess this is because we don't allow access to the net when building. Problem is the same process doesn't work for the 'new' setuptools.15:42
robtaylorrichard_maw: http://lwn.net/Articles/454304/15:45
richard_mawrjek: I think PTRACE_{SEIZE,INTERRUPT,LISTEN} mean you don't need to stop them like that any more15:45
rjekOh cool15:45
richard_mawrobtaylor: already there :-)15:45
robtaylorheh15:45
robtaylorand http://lwn.net/Articles/525675/ as well, i presume? ;15:45
robtaylor;)15:45
richard_mawnot that one15:46
robtaylorfirst cut prototype could just use criu as is, mind you15:46
robtaylorthen optimise15:46
richard_mawassuming criu gracefully handles being restored in a different tree to that it left15:47
robtayloryeahhh15:47
* robtaylor no know15:47
* richard_maw goes to find if they've got an IRC channel15:48
robtayloryeah15:49
robtaylorit seems it will work, but ymmv depending on how the process behaves wrt the filesystem changing underneath it15:50
robtaylorso should be fine for all the normal 'package upgrade' cases15:50
robtaylorThe things that it expects to be there are the things you have open() in the process, i *think*15:51
* richard_maw is asking in #criu15:51
richard_mawthere may be interest in cheap migration of processes between containers on the same system, so if it acceptably handles files being different on the target to the source, then it may even be upstreamable15:56
robtayloryep, i think thre's enough with criu's infrastructure to do it15:56
robtaylorand i think the criu guys will take patches for lots of odd usecases, judging from their wiki15:57
richard_mawmmm, rebootless kernel upgrade15:58
robtaylorrichard_maw: with criu?15:59
richard_mawit's one of the listed weird usage scenarios15:59
robtaylorok, well i shall leave you with that for a while. bbiab15:59
apespaceany idea what "Nonexistent host networking interface" means when starting a VM with http://wiki.baserock.org/guides/no-frills/16:01
*** tiagogomes [~tiagogome@213.15.255.100] has quit [Ping timeout: 264 seconds]16:03
paulsher1oodapespace: i think it's a vbox issue. click on network settings on the vbox ui... then try booting it?16:03
jmacsapespace: You probably don't have a host-only network present - it's usually a one-time thing when you install VirtualBox16:03
apespace"nonexistent host networking interface, name ''"16:04
paulsher1oodapespace: were you able to try what i suggested?16:05
paulsher1oodi believe i had the same experience in the video i made of the no-frills steps16:06
ssam2straycat: how does it not work ? what's the error?16:07
straycatthere's no error, setuptools just doesn't seem to be installed correctly16:07
ssam2oh, so it's the installation of setuptools itself that doesn't work16:07
straycat#pypa have pointed me at https://pip.pypa.io/en/latest/installing.html#install-pip for bootstrapping16:07
perrylapespace: in modifyvm, have you added '--hostonlyadapter2 vboxnet0'?16:08
straycatif we have to do it using that, then it's going to be a little more complex than what we're doing now16:08
straycatso i might try and see if there's a simpler way16:08
ssam2ouch, yeah16:08
apespaceperryl: no I havn't added that16:09
richard_mawooh, criu has a mode to run syscalls in another process16:09
perrylapespace: i had a similar issue with networking when creating a vm myself, if that solves the issue it may be worth updating no-frills16:10
ssam2straycat: the fedora packaging seems to still use setuptools' setup.py16:11
apespaceI'll try it16:11
ssam2straycat: http://pkgs.fedoraproject.org/cgit/python-setuptools.git/tree/python-setuptools.spec (although that's a pretty unreadable spec file)16:11
straycatssam2, what version of setuptools is that though?16:11
straycatssam2, I admit I'm finding it hard to believe this can't be done more simply.16:12
apespace"Couldn't start the virtual machine because the following physical interfaces were not found: vboxnet0 (adapter 2)"16:13
perrylargh :( must be a different issue to what i had...16:14
apespaceone thing I notice in virtualbox is that under network settings there is no host-only adaptor16:15
jmacsapespace: Add one in the GUI; the default settings should be OK16:15
jmacsI suspect those instructions aren't tested often because it's a pain for developers to uninstall and reinstall virtualbox16:16
ssam2straycat: seems to be v7.016:16
apespacejmacs: is there any way to do that from the commandline?16:17
straycatssam2, yeah i noticed16:17
jmacsapespace: There is, but I'd have to look it up. Hang on...16:17
jmacsapespace: "vboxmanage hostonlyif create" should do it16:18
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]16:21
*** apespace [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 244 seconds]16:26
radiofreebaserock systems that use an initramfs, are they upgradable?16:27
paulsher1oodrichard_maw: ^^ ?16:27
richard_mawthey certainly used to be, and should still be, why do you ask?16:28
paulsher1oodradiofree: is this to do with jetsons? i'm currently wondering whether my bricking was me, or something else16:31
*** apespace [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock16:31
radiofreepaulsher1ood: the btrfs patches are pretty flakey, so i doubt it was you16:41
radiofreerichard_maw: so we wouldn't need a u-boot with btrfs support then?16:41
*** petefoth [~petefoth@host-78-151-29-232.as13285.net] has quit [Quit: petefoth]16:43
*** br_logger2 [~ubuntu@85.199.252.189] has joined #baserock16:44
richard_mawradiofree: u-boot still needs to load the initramfs and kernel from btrfs16:45
radiofreeyeah that's why i'm thinking it won't be useful :\16:46
apespacethe hostonlyif worked, but virtualbox still stops with "error: Failed to open/create the internal network 'HostInterfaceNetworking-' (you might need to modprobe vboxnetflt to make it accessible)"16:47
radiofreehmm.. or maybe we could specific "BOOTLOADER_INSTALL" as well?16:47
radiofreeso on the jetson mmcblk0p1 is the bootloader partition (ext), which the kernels, device trees ext.. + extlinux.conf16:47
radiofreemmcblk0p2 is the actual btrfs partition16:47
radiofreehow is BOOTLOADER_INSTALL currently use i wonder16:49
* radiofree should know since he wrote it16:49
br_logger2 is now known as br_logger16:49
persiaWould anyone else find checksums for the images at download.baserock.org useful?  I'm currently having fun with proxy servers for the images at http://download.baserock.org/cloud/ , and would like to be able to verify I have the right data.16:50
*** br_logge1 [~ubuntu@85.199.252.110] has quit [Quit: Lost terminal]16:50
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 245 seconds]16:50
radiofreerichard_maw: would you have any massive objections to adding a "jetson-tk1" type for "BOOTLOADER_INSTALL"16:50
apespacepersia: sha25616:50
radiofreecurrently there is only "extlinux", which goes off and runs extlinux --install real_root16:50
radiofreei was thinking the jetson-tk1 one could mount /dev/mmcblk0p1, copy over the new kernel/dtb, update the extlinux.conf16:51
persiaapespace: Works for me, if I can get real data.  Personally, I prefer to have multiple checksums, to reduce the chance of collision16:51
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock16:51
richard_mawradiofree: I thought `extlinux --install` was why we decided to split BOOTLOADER_INSTALL up into installing the config and installing the bootloader itself16:53
apespacepersia: maybe gpg sign + sha25616:54
radiofreerichard_maw: indeed, we want to use an extlinux.conf on a jetson (BOOTLOADER_CONFIG_FORMAT) but not to actually install it16:54
radiofreeso for the jetson, rather than "none" for INSTALL (it defaults to "extlinux), we set it to "jetson-tk1", which installs the extlinux.conf/kernel on a ext2 partition for u-boot to use16:55
persiaapespace: That gives me two algorithms, so I like it, but requires me to accept the authenticity of a key: how do I know which is the role key (i.e. who certifies, and why do I trust the certifier)16:55
persiaBut seriously, only md5sums would meet my immediate need16:55
rjekmd5+sha256 in a manifest file, the manifest file having been signed16:56
jmacsWill systemd start an ExecStart job as root by default?16:56
robtaylorrichard_maw: so, do you think we have a plan? :)16:56
persiarjek: Sure, although that raises the certification issue again.16:56
ssam2jmacs: I'm not sure, actually, I struggled with this the other day16:56
richard_mawradiofree: I forsee a lot of patching of the write extension for every dev board if we were to go down that route16:56
jjardonmmm, seems since baserock-14.29 we are not using annotated tags. Is this intended?16:57
radiofreerichard_maw: i wrote it like that :\16:57
jmacsssam2: I'm seeing some odd problems whereby the same bash script works fine if executed by root on the console, but not if run by systemd16:57
persiaBut does anyone have permissions to write checksums?  Could they do so?  I'd be happy with any scheme, but it ought be done an environment where the authenticity of the source is known.16:57
radiofreethere's a "install_function_dict" that has the functions for each type16:57
radiofreeso it involves adding the new type + the implementation function16:57
franredjmacs, if you don't specify user, it would use root, tough16:58
radiofreei don't see it as a *terrible* thing to write specific bootloader implementation things16:58
jmacsfranred: That's what I expected.16:58
richard_mawradiofree: I'd prefer something generic, like a pre configuration extension step which makes the disk image, partitions and formats, then when morph is writing the rootfs the kernel automatically goes in the right place16:59
radiofreethis would be a massive benefit for us, it would mean we don't have to patch u-boot16:59
franredjmacs, maybe shell problems? bash - sh?16:59
radiofreerichard_maw: what about for upgrades?16:59
richard_mawradiofree: but that idea's not yet mature, so I suspect we may have to put the per-device badgering in16:59
jmacsI'm executing it with bash -c "/root/setup-things | tee /root/setuplog"16:59
richard_mawradiofree: how does putting the kernel and config on /boot work with upgrades in your model?16:59
jmacs...and setuplog looks fine; it  exactly matches what comes out when I run manually17:00
radiofreethat's essentially what i want to change it over to, mmcblk0p1 will become /boot17:00
richard_mawrobtaylor: I'm going to try criu for migrating processes to the new tree17:00
richard_mawrobtaylor: criu even has a mode for just running syscalls in the target process, so I may not need the whole checkpoint logic17:01
radiofreestock u-boot reads the extlinux.conf from that partition, boots into btrfs17:01
robtaylorrichard_maw: cool!17:01
radiofreethe jetson-tk1 "BOOTLOADER_INSTALL" updates the extlinux.conf file in /boot, copies the new kernel/dtb there etc..17:01
richard_mawradiofree: you'd also need to change the system-version-manager script again to do that then17:01
radiofreerichard_maw: is that currently hardcoded to look for extlinux.conf in the very root of the btrfs device?17:02
richard_mawand to copy the kernels and initramfs into paths relative to the root of the btrfs device, since that's how we specified upgrades to work17:03
radiofreewould prioritising the presence of /boot/extlinux.conf be considered acceptable?17:04
jmacsThanks franred - looks like I might need an "After=network.target" in my systemd script17:04
franredjmacs, cool :)17:05
jmacsJust mentioning it so we have a log :)17:05
jjardonpedroalvarez: Any reason to not use annotated tags in the latest baserock releases?17:06
pedroalvarezjjardon: nope, i thought it was only needed in morph when we were versioning it17:07
richard_mawradiofree: I'd rather we fixed up the btrfs patches, but if we can't do that then I think the system-version-manager script should check for BOOTLOADER_INSTALL = jetson-tk1 in /baserock/deployment.meta, and if that's set copy the kernel, initramfs and update the config in /boot17:07
radiofreeok!17:08
*** sambishop [~sambishop@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock17:09
jjardonpedroalvarez: AFAIK whitout annotated tags is not possible to know who made the tag, or when17:11
richard_maweven with you can't know, since that data can be faked17:11
jjardonrichard_maw: I can know who faked17:12
richard_mawhow?17:13
jjardonor at least when17:13
richard_mawnope, that's also fakeable17:13
jjardonanyway, I think you are putting an extreme case as an example17:13
jjardonrichard_maw: what about if you sign the tag?17:14
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock17:15
richard_mawI don't know the details of how signing works, but if it's done properly you can know who was faking the commit author and date17:15
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Changing host]17:15
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock17:15
persiaRather, you can know which key.  Translating a key into a person requires a chain of trust of the certifiers17:15
straycatssam2, I think this is a bug in setuptools17:20
straycatssam2, I just checked out 7.0 and it installs fine17:20
jjardonso, please: I think its a good idea to annotate (and sign) the releases ;)17:20
richard_mawyou need a key infrastructure that we currently don't have to make that worthwhile17:21
straycatI guess this means for now we can base our patch off 7.017:21
*** rdale [~quassel@92.Red-88-20-2.staticIP.rima-tde.net] has quit [Read error: Connection reset by peer]17:21
*** rdale [~quassel@92.Red-88-20-2.staticIP.rima-tde.net] has joined #baserock17:22
jjardonrichard_maw: ? simple sign with your pgp key17:22
ssam2straycat: cool!17:23
richard_mawjjardon: we don't all have them, nor a web of trust to make our keys useful for identifying that we were the ones to sign it17:23
richard_mawI'd be happy to see a proposal for how we should sign our artifacts17:24
richard_mawbut we've got a few proposals that seem to have become stuck in the mailing list as it is17:25
*** simonh_ [~simonh@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]17:26
jjardonrichard_maw: create a gpg key is trivial; Im sure several members of the team already have keys and would be happy to sign other peoples keys if needed17:26
persiarichard_maw: For signing, the public key infrastructure mostly works: the key is having the signing key be certified by individuals known to be reasonably trusted.17:27
persiaBut I agree, the right place for this discussion is the mailing list.17:29
richard_mawI'm happy for this discussion to continue, but I don't know enough about the implications to make a reasonable contribution.17:29
persiaAs a side effect, we can usefully identify the membership of the release team, where heretofore has been somewhat blurrily defined.17:29
jjardonrichard_maw: when you became a GNOME maintainer, you have to sign your releases: only condition is to create a key and use it, thats all17:29
apespacebinary releases are in general signed17:31
*** mauricemoss_ [~simonh@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock17:45
*** tiagogomes [~tiagogome@host-92-16-23-104.as13285.net] has joined #baserock17:58
*** tiagogomes [~tiagogome@host-92-16-23-104.as13285.net] has quit [Client Quit]17:59
*** apespace [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Lost terminal]18:00
*** Krin [~mikesmith@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]18:03
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]18:03
*** mariaderidder [~maria@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Ex-Chat]18:04
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]18:05
pedroalvarezgreat, I migrated paste.b.o, testirclogs.b.o, and testgerrit.b.o18:19
pedroalvarezI found many problems when doing a new mason deployment, I hope to send patches tomorrow with the fixes18:19
* straycat think it might be nice to have some tweaks so that morph can do more when you're offline18:24
*** rdale [~quassel@92.Red-88-20-2.staticIP.rima-tde.net] has quit [Read error: Connection reset by peer]18:24
straycatso have a stage at the start where you update the caches you need to update, fetch the things you need and then start building, perhaps18:25
*** rdale [~quassel@92.Red-88-20-2.staticIP.rima-tde.net] has joined #baserock18:26
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]18:31
persiastraycat: Could that be a --download-only argument to morph, perhaps with an --offline argument to tell it not to even try getting more cache?18:32
straycatpersia, I was thinking more that it'd be useful to do the cache stuff upfront, so that once your build starts you can be offline18:35
persiaFor how I use it, I wouldn't want to start a build, and then go offline midway through (although, thinking about it, I'd like more parallelism, so that downloads were happening even when a build was happening).18:35
persiaMy thought was that the --download-only option would update all the caches (both gits and objects) for the artifact to be built, so that when one made changes, one would only do the local building, keeping the cache.18:36
persiaNote that this usage model may differ from that of other users18:36
straycatOh I see, yeah a --download-only would be just as useful for my case I think. So I could run --download-only then continue offline later18:39
persiaOr immediately, if you liked.18:40
persiaIt also lets you test if everything is cached.18:40
persiaSo if I was working on an ARM laptop and deploying to a PPC board, I might want to use --download-only to make sure I had all the artifacts for a system in the cache, so I could then fiddle with different deployments offline.18:41
persia(because distributed build systems don't work as well offline :) )18:41
* straycat nods18:54
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock19:01
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock19:39
*** zoli_ [~zoli_@linaro/zoli] has quit [Read error: Connection reset by peer]19:39
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Remote host closed the connection]19:54
*** wdutch [~william@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Quit]20:04
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock20:42
*** zoli_ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Changing host]20:42
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock20:42
*** zoli_ [~zoli_@linaro/zoli] has quit [Read error: Connection reset by peer]21:10
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock21:10
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Remote host closed the connection]21:44
*** genii [~quassel@ubuntu/member/genii] has quit [Remote host closed the connection]22:30
*** ChanServ [ChanServ@services.] has quit [shutting down]23:11
*** ChanServ [ChanServ@services.] has joined #baserock23:14
Mode #baserock +o ChanServ by card.freenode.net23:14

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!