IRC logs for #baserock for Thursday, 2015-01-22

*** simonh_ [] has joined #baserock01:59
*** mauricemoss_ [] has quit [Ping timeout: 264 seconds]02:06
*** zoli_ [] has joined #baserock05:43
*** zoli_ [] has quit [Changing host]05:43
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock05:43
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]07:14
*** zoli_ [] has joined #baserock07:41
*** zoli_ [] has quit [Changing host]07:41
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock07:41
*** grahamfinney_ [] has joined #baserock07:51
*** grahamfinney [] has joined #baserock07:54
*** wdutch [] has joined #baserock08:16
*** wdutch [] has quit [Read error: Connection reset by peer]08:17
*** wdutch [] has joined #baserock08:17
pedroalvarezTrue. I guess that I assume nobody is using minimal systems. 08:18
SotKcould I get a quick review of this lorry by any chance? (Kinnison +1'd it a few days ago but it got swept away before anyone gave a second +1)
paulsherwoodSotK: +108:57
*** mdunford [] has joined #baserock08:57
pedroalvareztiagogomes_: 2015-01-21 20:41:51 Build of baserock:baserock/definitions f2532705c6e110bf0012cac0933abcbf1c5ed848 systems/devel-system-ppc64-generic.morph ended successfully09:03
*** franred [] has joined #baserock09:06
*** mariaderidder [] has joined #baserock09:09
*** bashrc [] has joined #baserock09:11
*** jonathanmaw [] has joined #baserock09:20
Kinnisonpedroalvarez: coo09:21
*** sambishop [] has joined #baserock09:22
SotKKinnison: is your +1 of my lorry above still valid?09:24
KinnisonI remain confused as to why we pick https rather than git:// but that isn't worth complaining about, so yes +109:24
SotKthats what github provides in its link box, I'll change it to git if you like (I normally do, but forgot this time)09:25
bashrcto avoid port blocks?09:25
Kinnisonif is suffering from port blocks, we have worse things to contend with09:26
KinnisonSotK: I don't mind -- in theory https + certificate-checking is more secure09:26
SotKOK, merged, thanks!09:28
*** mdunford [] has quit [Ping timeout: 245 seconds]09:31
tiagogomes_pedroalvarez, cool, can I use the armv7lhf machine that I was using yesterday09:38
pedroalvareztiagogomes_: yup09:38
*** mdunford [] has joined #baserock09:47
*** mdunford [] has quit [Ping timeout: 276 seconds]10:00
franredIm creating some lorries for some apache repositories, do we care if we mirror from the git repos which apache give to the comunity ( ? or we want to lorry from the svn repositories (
*** Krin [] has joined #baserock10:03
franredthe repositories are httpd, apr and apr-utils10:03
pedroalvarezhm.. I'd prefer git whenever possible10:05
franredsaid that the git repositories are a mirror from the svn repositories (which are the "in development" ones)10:06
pedroalvarezhow frequent are they updated? can we know that?10:08
KinnisonShould they choose to switch to git, they're more likely to go with their conversions than any we produce, we should lorry their git repos unless they're abandonware10:09
franredpedroalvarez, "...These mirrors contain the full version histories (including all branches and tags) of the mirrored codebases and are updated in near real time based on the latest svn commits..."10:12
franredKinnison, ok.10:12
pedroalvareznear real time sounds enough to me :)10:12
pedroalvarezand makes the lorry lighter (I believe)10:13
franredjust a warning for the future, they have mirrors in github too, but these are not updated as frequent as the one in
franredpedroalvarez, yeah, I just need to check if apr and apr-util git repos are fine, last time I've tried I cloned empty repositories10:14
pedroalvarezfranred: maybe they didn't have a master branch?10:14
franredI will check10:15
pedroalvarezis the branch that git checks out by default10:15
franredthat is a good point actually10:15
*** lachlanmackenzie [] has joined #baserock10:15
*** mdunford [] has joined #baserock10:16
franredummm, pedroalvarez, no, apr looks empty10:22
franredand apr-util looks outdated10:22
franredbut their github mirrors looks ok10:23
pedroalvarezfranred: can you /msg me the git urls from and gihub?10:23
franredpedroalvarez, done10:26
pedroalvarez"warning: You appear to have cloned an empty repository."10:29
pedroalvarezHm... let me find someone from apache10:31
franredpedroalvarez, ta10:43
franredI have another question.... we have lorried psycop2 but for windows.... o.O! - can we remove that repo and its lorry? I want to lorry the one we need instead10:45
pedroalvarezso I guess that it cannot be used at all10:47
pedroalvarezand it hasn't been used yet10:47
franredit is not in use if you are asking that10:47
franredwe are using github:psycopg/psycopg2 instead10:47
pedroalvarezI think that updating its git url and change the lorry to force push all the things could be ok10:48
pedroalvarez(but, I'm never sure about lorries)10:48
pedroalvarezwill downstream troves get the change?10:49
pedroalvarezI don't know10:49
franredpedroalvarez, so, what is your suggestion?10:50
pedroalvarezwait for opinions :)10:51
franredok, waiting... :)10:57
richard_mawdownstream troves will get the change10:57
*** grahamfinney_ [] has quit [Quit: Ex-Chat]11:14
jmalkI seem to be using a not-up-to-date version of morph. given this guide - - I'd have thought going to that directory and `git pull` would update it, but it's telling me everything is up to date. any advice please?11:46
pedroalvarezjmalk: if you do `git branch` it shows you master?11:47
pedroalvarezalso, what shows this command? `cat /usr/bin/morph` 11:47
jmalkpedroalvarez: git branch in /src/morph is master, yes11:47
pedroalvarezjmalk: looks ok, I think you are using latest one11:49
pedroalvarezyou can also run `morph --version` to doublecheck11:49
pedroalvarezit should show you: f2f3e4e5f3d5162890d03494f405cff5177aaad811:50
franredjmalk, is that morph script in /usr/bin?11:50
jmalkalas, 1cafde326cd1858980a5a13aa685b624792a851411:50
pedroalvarezfranred: yes11:50
jmalkpedroalvarez: yes, /usr/bin/morph11:50
franredyes, sorry11:51
pedroalvarezjmalk: hm... odd11:51
jmalkpedroalvarez: very. maybe I should just remove morph and clone it afresh?11:51
pedroalvarezjmalk: does `/src/morph/morph --version` shows the same sha1?11:53
jmalkpedroalvarez: it does11:54
pedroalvarezthen is a matter of upgrading the git clone11:54
paulsherwoodjmalk: which morph? and is that a script?11:54
paulsherwoodsorry `which morph`, and try editing the result11:54
pedroalvarez`which morph` is going to show always /usr/bin/morph11:54
paulsherwoodpedroalvarez: 'always' is a very strong word11:55
* paulsherwood doesn't trust software in general11:55
pedroalvarezpaulsherwood: haha, ok. I mean, following our "using latest morph" guide, doesn't modify the result of `which morph`11:55
jmalkwhich morph returns /usr/bin/morph11:56
pedroalvarezjmalk: rename the /src/morph folder and clone it again11:56
paulsherwoodjmalk: and /usr/bin/morph is editable and is a script calling /src/morph ?11:57
jmalkpaulsherwood: it is, yes11:57
paulsherwoodand  in /src/morph git status shows clean?11:57
paulsherwoodand there git log shows the desired commit at the top?11:57
jmalksrc/morph git status is not clean! it has an outgoing/ folder which I used to prepare a covering letter.11:58
paulsherwoodaha. so maybe try causing /src/morph to be at the desired state via git checkout, or git reset, git pull etc?11:59
jmalkhmmm rm outgoing/, now directory is clean, but still can't pull to get to completely up-to-date12:00
pedroalvarez`git checkout master && git pull`?12:00
SotKwhere did you clone it from?12:00
paulsherwoodgit remote -v12:00
jmalkah, git knows about "github" (my account) and repo - do I need to specify origin ( when pulling I presume?12:01
pedroalvarez`git fetch` will fetch everything from that origin12:02
pedroalvarezand I'd use `git reset --hard` to go to master of that origin12:03
jmalksorted! that was a fun little mystery, thanks for the help everyone12:03
*** sambishop [] has quit [Ping timeout: 244 seconds]12:18
*** mdunford [] has quit [Ping timeout: 245 seconds]12:20
*** sambishop [] has joined #baserock12:33
* pedroalvarez deploys a baserock system to an usb stick using usb passthrough on his vm12:35
pedroalvareznext step plug it into a machine and boot the system :)12:35
nowsterAsking for background info: is Baserock now wedded to the facilities that btrfs provides?12:43
nowster(ie. would developing on a system without btrfs be an exercise in pain?)12:45
*** wdutch [] has quit [Quit: Quit]12:45
richard_mawour update method currently relies on it, if you have an alternative way of doing updates, such as replacing the rootfs of a VM, then you could get away without it12:46
persianowster: There has been talk about support also for ZFS, which has the same sort of features, but I don't believe that is yet either complete or merged.  You should also be able to use a different write extension to cause the resulting filesystem to be anything (but updates won't work).12:53
pedroalvarezSo, after deploying a system to the usb. My machine loads the initramfs system ok, then the kernel, and then it panics. Not sure about how to debug this, nor what can be the issue12:56
pedroalvarezrichard_maw: any hint?12:56
*** mdunford [] has joined #baserock12:56
richard_mawpedroalvarez: I didn't even get that far with USB pass-through12:57
persiaIF the iniitramfs stuff is working, then the kernel can *boot* on the hardware.  Perhaps the kernel doesn't have the right drivers to access the / filesystem from USB?12:57
pedroalvarezgood point12:58
*** sambishop [] has quit [Ping timeout: 245 seconds]12:59
richard_mawit's difficult to guess without any further info, if the kernel panic text isn't useful, having scrolled away from the useful output, you could keep back-scroll if you were using a serial port13:00
persiaOr a console with many rows: with the right arguments, some screens can provide 200 or so.13:00
*** sambishop [] has joined #baserock13:01
persiaSomething else to try is to use an initramfs that *doesn't* try to mount / off storage.13:02
persiaThen try to mount the storage directly, and investigate the error messages.  Once you can mount storage, mount some kernel modules, and start modprobing until you can mount your target / properly: this gives you a good guide to what needs to be turned on in the kernel configuration.13:03
richard_mawI think there may be a kernel command-line argument to say "run this in your initramfs, rather than /init"13:07
richard_mawif it exists, you could say to use /bin/sh as your init13:08
persiaIt does exist.  I believe it is "init=", so "init=/bin/sh" for example.13:09
richard_mawah, put `rdinit=/bin/sh` and you can see if it's the initramfs script that's at fault13:09
richard_mawpersia: if you've got an initramfs, then the kernel expects init= to be handled by the initramfs13:09
persiaAh.  Thanks for the correction.  I thought the argument worked the same whether there was an initramfs or not.13:11
pedroalvarezthanks for your help. I'll try with that and see how it goes.13:14
pedroalvarezI managed to see the kernel panic test (by recording it: but is not useful at all. I will try rdinit now.13:22
richard_maw"ata3.01: failed to resume link" doesn't sound good13:43
richard_mawit does get as far as executing mount though13:44
pedroalvarezso, I assume that the kernel is not finding the usb stick 14:14
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]14:17
* pedroalvarez adds usb3.0 support to the bsp14:22
*** pacon [~pacon@] has joined #baserock14:26
*** bashrc [] has quit [Ping timeout: 240 seconds]14:34
*** bashrc [] has joined #baserock14:36
*** zoli_ [] has joined #baserock14:38
*** zoli_ [] has quit [Changing host]14:38
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock14:38
*** pacon [~pacon@] has quit [Read error: Connection reset by peer]14:40
*** grahamfinney [] has quit [Quit: Ex-Chat]14:47
*** grahamfinney [] has joined #baserock15:00
*** bashrc [] has quit [Ping timeout: 245 seconds]15:07
*** bashrc [] has joined #baserock15:09
richard_maw looks like an interesting solution for recording videos of terminals, which is what we generally have15:15
richard_mawI wonder if it was inspired by Lars' live coding thing15:15
paulsherwoodmuch o/win 3315:16
*** mdunford [] has quit [Ping timeout: 255 seconds]15:16
jmacsHmm, looks good15:17
pedroalvarezI gess I can't use it to record the kernel eror trace :/15:19
pedroalvarezenabling usb3  has worked. Now the kernel says that it has found the usb device, but it's still panicking 15:20
richard_mawpedroalvarez: if you use screen to attach to a serial port it would work, but at that point you don't need to turn it into an ascii video15:22
*** mdunford [] has joined #baserock15:31
jmalkI'm running on a VM, but keep getting asked for root@'s password: what have I forgotten to do?15:34
persiaGrant passwordless ssh from your development VM to your development VM.15:34
persiaEasiest way is to generate a local id_rsa pair, and append to authorized_keys in .ssh/15:35
jmacsssh-keygen; ssh-copy-id root@localhost15:36
jmacsDoes the same thing15:37
jmalktvm folks15:37
*** mdunford [] has quit [Ping timeout: 240 seconds]15:42
*** mdunford [] has joined #baserock16:01
persiaDoes morph clear the ccache cache when the ABI for a build is changed (e.g. when patching the C library in interesting ways)?16:11
* paulsherwood doubts it, but may be pleasantly surprised16:13
KinnisonSeems unlikely, but the compiler ABI is part of the key for ccache16:18
richard_mawpersia: is this the ABI of dependencies, or the ABI of the object you're compiling?16:21
richard_mawI don't think the ABI of dependencies can affect builds at this point, since ccache only caches .o files16:22
persiarichard_maw: I'm thinking of the object: my thought is that if we have the same ABI string and the same arguments, ccache will try to reuse the blobs, even if we do this with a radically differently patched C library.16:22
Kinnisonccache only caches intermediate objects16:23
Kinnisonso different libc shouldn't make any odds16:23
Kinnisonif the patches alter the headers then the preprocessed code will differ and ccache will build it afresh16:23
KinnisonIn general, you have to work hard to lie to ccache16:24
Kinnison(or be a compiler developer)16:24
persiaKinnison: As a thought example, do you want to assert that remains true if I build a bunch of armv7l objects, then modify the configuration so my new "armv7l" is identical to "armv7lhf" and try to rebuild things?16:24
persiaBeing changes to kernel API headers, libc, gcc, etc.16:25
Kinnisonif it's identical then the compiler will have changed16:25
Kinnisonand thus ccache will behave right16:25
Kinnisonsince the ABI will no longer be softfloat16:25
persiaSo ccache will automatically notice the ABI features, rather than the actual ABI?16:25
persiaEven if the arguments passed to gcc are the same?16:26
KinnisonI believe the gcc specs form part of the key16:26
KinnisonI believe in Baserock, the cache key of the compiler forms part of ccache's key16:26
persiaFor morph, yes.  For ccache?16:26
Kinnisonrichard_maw: ^^ ?16:26
persiaAha!  If that is the case, then my worry is unfounded, and I am delighted.16:26
Kinnisonlets hope richard_maw bears out my belief then16:27
richard_mawKinnison: that's broadly correct, the current approach is to include the cache keys of the toolchain chunks in ccache's hashing algorithm16:27
richard_mawthis is currently implemented by hashing the /baserock metadata files16:27
Kinnisonduring the ccache chunk prep?16:28
richard_mawit's a little fragile, so I'd prefer to alter morph to export the cache key of a build to its build commands, so we could include the cache id from morph in the hashing algorithm16:28
richard_mawKinnison: no, at every build unfortunately, morph sets environment variables telling ccache that it needs to hash those files into ccache's key16:29
KinnisonI think that's overly worrisome16:29
Kinnisoni.e. it will reduce cache hits dramatically16:29
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]16:29
richard_mawwhen I looked, ccache didn't have a way to inspect the specs for hashing16:30
richard_mawits options were to hash the `gcc` binary (which doesn't always work, and they rely on this in their test suite)16:30
richard_mawinclude the mtime of the gcc binary16:30
richard_mawor include the version string of the gcc binary16:30
richard_mawor include a set of files as defined in an environment variable (what we do)16:30
* paulsherwood notes that json.dumps(hash_factors, sort_keys=True).encode('utf-8') seems to provide a consistent result for hashing, even on nested dicts, which is rather easier to undertand than the current things morph does.16:31
Kinnisonrichard_maw: If we passed in the chunk cache key during the build, for gcc to include as the build ID (which goes into the binaries) then hashing the gcc binary should be plenty16:32
richard_mawa lot of its correct operation assumes either you've got a distro handling this to ensure toolchains hash distinctly, or you're fully aware of the risks when you build your own toolchain16:32
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock16:32
straycatpaulsherwood, i don't follow that16:34
paulsherwoodstraycat: it may be a non-sequitur. i just thought if the cache algorithm might be getting some love, there appears to be a simpler way to code a soln. i may be wrong. i often am16:36
simonh_ is now known as mauricemoss_16:37
richard_mawpaulsherwood: it's constructing the hash_factors that is the difficult computation anyway, and to me it doesn't make a great amount of difference whether we do something to construct the hash_factors data structure, or pass hash factors directly to the hashing function. It does however take more memory to construct the hash factors and the dumped string before feeding into the hashing algorithm, instead of feeding the hashing data into the alg16:37
paulsherwoodwhat's difficult about constructing the hash_factors? it's just a set of things? 16:39
richard_mawKinnison: true, but if we bake the hash of the ccache chunk into ccache, it doesn't need to hash the gcc binary at every build16:39
persiaI think hasing every build is a feature.16:39
straycatwhat are hash factors?16:39
richard_mawpaulsherwood: difficulty isn't the point. To me it's the same amount of work to construct the hash_factors structure as to feed the hash factors directly into the hashing algorithm.16:39
persiaIt was my worry that we might not do it for every build that caused me to raise the point in the first place16:39
straycatI know of load factors16:39
richard_mawpaulsherwood: the difference is that you use more memory building the hash_factors structure16:40
paulsherwoodin this context, hash factors are (say) *-commands, ref, repo, build-depends etc16:40
paulsherwoodrichard_maw: isn't that a very small amount of memory compared to other things ?16:41
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]16:42
richard_mawpaulsherwood: it's roughly the same amount of memory again, twice, since it needs to be kept in data structure form and string form at the same time.16:43
richard_mawbut I agree that it's not enough memory that it should be the deciding factor16:44
paulsherwoodit's maybe 1k of memory?16:44
paulsherwoodand this is one line of code vs several inscrutible functions?16:44
paulsherwoodbut as i've said elsewhere 'anyone who believes in egoless programming doesn't know many programmers', so i accept i may be failing to see straight16:46
KinnisonI think paulsherwood is trying to suggest a one-liner to replace the _hash_id method of cachekeycomputer -- wouldn't produce identical results but ought to be as safe16:49
richard_mawI agree it could do better. I don't know of any built-in dict-walking code that we could use to hash it easily.16:49
KinnisonBut it'd rely on json being able to hash tuples as well as lists16:49
Kinnisonand I don't know if it can16:49
paulsherwoodi don't think there need to be tuples?16:49
KinnisonThere are in morph right now16:49
Kinnisonso we'd have to refactor more than the place your oneliner helps to be able to use it16:50
Kinnisonif it can't16:50
KinnisonAnd yes, I'm aware that in changing the cache key computation algorithm we'd be rewriting not refactoring16:50
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock16:50
richard_mawKinnison: json will convert tuples to lists, so it's not round-trippable, and it doesn't take into account the types, however we currently hash both tuples and lists tot he same thing anyway, so that wouldn't be a regression16:50
paulsherwoodunderstood :)16:50
paulsherwoodif someone is going to actually think about this, please take a look at - i'm not saying it's hashing everything that morph does, but the mechanism could, and i think it's easier to grok16:52
paulsherwoodi'll stop pitching, now16:53
richard_mawmy only remaining concerns are that dumping to json is a bit of an abuse of the interface, when all we need is a stable serialisation16:53
richard_mawand if we were to change the computation to hashing dumped json, it would invalidate all our chunks, so we'd have to rebuild everything16:53
bashrcMade a separate "scripted for virtualbox"
paulsherwoodbashrc: why is that better than just cutting and pasting from ?17:01
paulsherwoodbashrc: i'm not trolling, am interested to know why you think this is better17:02
bashrcyes, that is quite similar17:02
bashrcprobably a circular reference17:03
persiabashrc: Nice.  Have you looked at vagrant?  I'd like to be able to host something that lets folk run `git clone foo; cd foo; vagrant up` to get a basic development environment.17:03
franred"Hi, Im facing the following problem: iptables-restore v1.4.21: iptables-restore: unable to initialize table 'nat'\n\nError occurred at line: 2\nTry `iptables-restore -h'"17:04
franredwith the 3.18 kernel17:05
franredAFAIK, nat module is available in the kernel --> strata/bsp-x86_64-generic/linux-x86-64-generic.morph:- scripts/config -e NF_NAT17:05
franredhow can I debug this? (also note that lsmod does not give me any information)17:06
nowstertrying to cross-bootstrap here... using a recent gcc... hitting a linker error in building gcc:
nowsterthe compiler's missing crt?.o17:13
richard_mawthat usually means the -B paths are wrong, but error handling when building gcc is... interesting17:14
nowsterIt hasn't built any crt?.o files, but it has built some of their precursors.17:16
richard_mawat this point, it's supposed to be using those17:16
richard_mawI usually start by running gcc in verbose mode so it says which internal command is failing17:17
richard_mawsorry, this isn't something I can help someone else debug, I only know where to poke as I'm doing it17:17
nowsterMy gut feeling is that it hasn't built libgcc correctly17:18
Kinnisoncrt.o is from libc isn't it?17:19
franredummm looks like NF_NAT_IPV4 was renamed as IP_NF_NAT in 3.17 kernel:
richard_mawKinnison: aye17:20
*** jonathanmaw [] has quit [Quit: Leaving]17:34
* richard_maw failed to catch up with his e-mail backlog, but is down to two pages, rather than 2017:41
*** Krin [] has quit [Remote host closed the connection]17:58
*** bashrc [] has quit [Quit: Lost terminal]18:03
*** mariaderidder [] has quit [Quit: Ex-Chat]18:17
*** mdunford [] has quit [Ping timeout: 264 seconds]18:23
*** rdale [] has quit [Ping timeout: 246 seconds]18:52
*** rdale [] has joined #baserock18:56
*** rdale [] has quit [Ping timeout: 244 seconds]19:00
*** grahamfinney [] has quit [Quit: Ex-Chat]19:01
*** rdale [] has joined #baserock19:03
*** rdale_ [] has joined #baserock19:05
straycatheh, pip's tests make morph's look fast19:06
*** rdale [] has quit [Ping timeout: 246 seconds]19:08
*** rdale [] has joined #baserock19:12
*** rdale_ [] has quit [Read error: Connection reset by peer]19:12
*** rdale [] has quit [Ping timeout: 256 seconds]19:16
*** rdale [] has joined #baserock19:20
*** rdale_ [] has joined #baserock19:24
*** rdale [] has quit [Ping timeout: 264 seconds]19:25
*** rdale [] has joined #baserock19:25
*** rdale_ [] has quit [Ping timeout: 264 seconds]19:29
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]19:44
*** lachlanmackenzie [] has quit [Ping timeout: 264 seconds]19:59
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock20:28
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]20:31
*** franred [] has quit [Remote host closed the connection]20:38
*** zoli_ [~zoli_@linaro/zoli] has joined #baserock21:29
*** petefoth [] has quit [Ping timeout: 245 seconds]22:11
*** zoli_ [~zoli_@linaro/zoli] has quit [Remote host closed the connection]22:57

Generated by 2.15.3 by Marius Gedminas - find it at!