IRC logs for #baserock for Monday, 2015-05-18

*** zoli__ has joined #baserock04:57
*** petefoth has quit IRC05:13
*** zoli__ has quit IRC05:55
*** petefoth has joined #baserock06:23
*** zoli__ has joined #baserock06:37
*** zoli___ has joined #baserock06:40
*** zoli__ has quit IRC06:40
*** mike has joined #baserock06:44
*** sherm_ has joined #baserock06:44
*** mike is now known as Guest4221506:44
*** Guest42215 has quit IRC06:47
*** sherm_ has quit IRC06:47
*** sherm_ has joined #baserock06:47
*** fay_ has quit IRC07:02
*** fay_ has joined #baserock07:05
*** paulw has joined #baserock07:22
*** fay_ has quit IRC07:23
*** a1exhughe5 has joined #baserock07:24
*** fay_ has joined #baserock07:24
*** rdale has joined #baserock07:39
petefothInterestign article on OpenStack at
*** mariaderidder has joined #baserock07:59
*** bashrc has joined #baserock08:17
pedroalvarezgreat, mason has that Parted is building using a submodule outside of g.b.o08:18
pedroalvarezs/has/has found/08:18
Kinnisonpedroalvarez: huzzah08:18
pedroalvarezthis is really good08:19
*** sambishop has joined #baserock08:24
rjekpedroalvarez: Sadmaking and delighting at athe same time.08:29
*** jonathanmaw has joined #baserock08:30
pedroalvarezI hope once fixed this Mason is happy08:30
*** sherm_ has quit IRC08:31
*** gary_perkins has joined #baserock08:43
*** CTtpollard has joined #baserock08:45
*** franred has joined #baserock08:46
*** sherm_ has joined #baserock08:47
pedroalvarezfranred: thanks for reviewing on baserock-dev :)08:50
tlsaanyone able to +2 this: ?08:56
*** pacon has joined #baserock08:59
*** ssam2 has joined #baserock09:00
*** ChanServ sets mode: +v ssam209:00
richard_mawtlsa: taking a look.09:02
richard_mawoh, franred beat me to it09:04
tlsaah, thanks franred09:04
franredrichard_maw, I can wait to merge it after you finish your review09:04
richard_mawfranred: +209:05
franredtlsa, merged09:05
tlsathanks both :)09:05
*** tiagogomes_ has joined #baserock09:06
*** sherm_ has quit IRC09:11
radiofreein foundation we build dbus-pre, and dbus09:12
radiofreeit's used only for systemd09:13
radiofreeoh right, --disable-systemd09:13
radiofreesystemd needs dbus, then dbus needs systemd09:13
richard_mawI'm not sure systemd *does* need libdbus any more09:14
radiofreewe seem to be using a highly specific branch as well baserock/dbus-1.8.16-1-g3c05557/genivi/J-1.009:14
richard_maw*that* is because genivi needed weird patches09:14
radiofreeoh wonderful, so potentially we can't just bump it to dbus-1.9.1609:15
KinnisonI guess we need to analyse what the patches were that genivi needed and determine whether or not they still need them09:15
Kinnisonit may have been AF_BUS related09:15
radiofree"add support for custom marshalling"09:15
radiofreethey're still adding that to the top of their genivi branches (at least as of 1.9.4)09:16
*** edcragg has joined #baserock09:16
*** Krin has joined #baserock09:16
radiofreei'll see how cleanly this applies to 1.9.1609:16
rdalei would have thought dbus marshalling is pretty flexible - what is it that they are marshalling that is so tricky?09:16
radiofreepatch applied without problems09:20
jonathanmawI just had a chunk fail to build because the .git directory belonged to user 1000, instead of root09:21
rdalei'm totally unconvinced it is needed given that everyone else seems to have no problems with xml based idls etc09:21
*** tiagogomes_ has quit IRC09:22
*** tiagogomes_ has joined #baserock09:22
jonathanmawfor some reason a lot of my repos in /src/cache/gits belong to user 100009:23
KinnisonOn a pure Baserock system or a chroot?09:24
*** sherm_ has joined #baserock09:24
jonathanmawKinnison: pure baserock.09:24
KinnisonHow odd09:24
Kinnisondoes your system have a uid 1000 in it?09:24
jonathanmawbaserock has no user 1000, my best guess is that someone did something silly with rsync09:24
KinnisonOr did you rsync stuff from elsewhere ?09:24
* richard_maw was just able to point someone at the work we did for openstack because they also wanted to run openvswitch with networkd09:35
Kinnisonrichard_maw: cool09:35
*** zoli___ has quit IRC09:37
*** zoli__ has joined #baserock09:38
franredpedroalvarez, you have reviewed with -1 but I can't see any comments about why :)09:39
SotKpedroalvarez: do you know what happens if you restart the stuck build you saw on Friday?09:43
pedroalvarezSotK: so, stop the build and kick a distbuild again?09:45
pedroalvarezSotK: I can try that, but I don't know yet. I upgraded Morph to the version that we are using in 15.19.2 to see if the problem still exists (and stop bothering you)09:46
SotKpedroalvarez: yes please09:47
pedroalvarezto stop bothering you? :)09:47
pedroalvarezSotK: thanks for helping09:48
SotKhaha, to restart the build :)09:48
SotKno problem09:48
* richard_maw tested with a live atomic update by having systemd pivot itself into the new root09:52
richard_mawI think it ought to make a new network namespace on pivot and un-propagate mounts in the old root, so it can unmount them in the new root09:54
richard_maws/network namespace/mount namespace/09:54
straycathrm, older versions of git-review seem to have --draft, but my version does not09:55
* richard_maw notes his shutdownramfs patches haven't been reviewed09:56
* pedroalvarez is going to be less busy this week09:58
pedroalvarezI'll be doing some reviews, starting with shutdwonramfs09:59
jjardonHi, is there a way to tell morph to generate the images in a specific filesystem format (ie, ext4 instead btrfs)10:13
pedroalvarezjjardon: I believe that there isn't a way to do that10:14
pedroalvarezyou will have to do your own write extension10:14
richard_mawjjardon: not currently10:14
* jjardon is in trouble :)10:15
richard_mawwhat is the nature of your Baserock emergency?10:15
jjardon2 problems: 1. generate an image smaller that 32MiB 2. Generate an image that is not btrfs10:18
jjardonthis is meant to be a OpenBMC image to flash a aspeed BMC chip10:19
rjekIs ext4 suitable for use on the Flash?10:20
richard_mawhm, I understand your attempt at stripping the binaries as a system integration command isn't sufficient. You may be able to solve that by manually adding strip commands to the chunks that go into your system.10:20
ssam2jjardon: a .write extension can do anything you want, there's no need for it to be btrfs10:20
rjekfor example, is it exposed as a block device or a MTD, in which case something like jffs or ubi might be more appropriate?10:20
ssam2jjardon: some .write extensions use code from the morphlib.writeexts module that creates a btrfs filesystem layout, but it's fine not to use that10:20
jjardonrjek: I think MTD10:20
jjardonrjek: thanks for the hint though10:21
rjekjjardon: I'd probably avoid extN, then.  Do you know what the reference image uses?10:21
jjardonI will try to write a new extension then10:22
rjekIf you're not careful, extN cooks flash, and I imagine your BMC's flash is not easily replacable!10:23
jjardonrjek: no idea, kejiahu do you know?10:24
kejiahujjardon: not particular familar with what fs should be used in MTD, but there is a list here:
kejiahujjardon: JFFS2 is something I heard about a lot, may be go for that at first?10:27
rjekJFFS2 is a bit old-school.10:28
straycathuzzah submitting drafts is actually just git-review -D >.>10:28
rjekI think UBI is the fashion-statement of the moment.10:28
* kejiahu changes his mind to UBI :P10:28
* straycat has submitted draft topic for comments10:28
jjardongrepping the I see references to jff210:29
rjekProbably easier to go with what they know already, then10:29
straycati was working on this a bit last week, but it's still not quite finished, i may get some time to finish it off tomorrow if i'm lucky10:29
jjardonrjek: this project uses a 6 year old kernel, so I would not expect anything "fashion-statement" :)10:30
richard_mawstraycat: I see no changes in the topic, is that meant to happen?10:30
rjekjjardon: hahahaha10:30
rjekjjardon: OK, JFFS2 it is :)10:30
franredstraycat, I can't see nothing in that link...I don't know id there are public drafts10:30
rjekWe used JFFS2 at Simtec; it's very nice but not scalable: large file systems (>128MB or so) take too long to mount on puny ARMs.10:30
rjek(As it's a log-style file system: it needs to scan the whole of flash to find the latest version of the log.)10:31
straycatrichard_maw, franred, I guess I must have to explicitly add you as revieweres10:31
richard_mawrjek: how does squashfs compare?10:32
straycatwhich… isn't exactly what i wanted tbh10:32
rjekrichard_maw: squashfs is read-only10:32
rjekSo has no journal to worry about10:32
straycatrichard_maw, franred, try again10:34
*** zoli__ has quit IRC10:34
franredstraycat, yeah, that works10:34
richard_mawstraycat: I see it now10:34
richard_mawwell, sort of10:35
richard_mawI can see that it exists, but not its contents so far10:35
straycatyou can see only one change?10:35
richard_mawstraycat: yes10:35
franredstraycat, but I can only access the last patch10:35
straycatthat makes sense, I only added you to the last patch10:36
richard_maw appears to be an empty page10:36
jonathanmawsuperstitious gerrit10:37
richard_mawpatch request of the beast10:37
straycatrichard_maw, I'm not sure what to make of that10:37
richard_mawtbh I'd be happier with avoiding drafts, slapping RFC to the beginning of all the patch names and preemptively -2ing it10:38
radiofreecan i have some gerrit help please10:39
radiofreei need to update
radiofreehowever one patch has already been merged10:39
straycatrichard_maw, *nod*10:39
radiofreeand also now is complaining about a dependency issue10:39
radiofreeshould i abandon the webkit patch + qtmultimedia and resubmit?10:40
pedroalvarezradiofree: never abandon if you plan to resubmit10:40
richard_mawI usually got away with rebasing the whole series, dropping merged patches and pushing the whole series again10:40
jjardonkejiahu: mmm, seems they are using ext2 for the wedge board:
radiofreewell one of the patches i need to resubmit has already been merged?10:40
radiofreeso rebase whole, drop merged one, resubmit with same topic branch?10:41
* straycat publishes ripsum/requests-in-db10:41
pedroalvarezradiofree: and keeping the change-id's10:41
radiofreeok, thanks10:41
straycathrm? i've not needed to do that10:42
straycatassuming you kept the change-id of the merged patch, i would haven't expected there to be a problem there10:43
kejiahujjardon: I don't think ext2 was supported in MTD, unless they have FTL device in between the flash and BMC chip. we don't have FTL on the board as far as I can tell, so JFFS2 may be the best choice for us now.10:55
franredstraycat, Im just curious, have you check what is the performance using the database? (comparison between with and without the database in a build and in distbuild)10:56
straycatfranred, no10:56
franredstraycat, Im not clear about what do you try to fix/achieve with the introduction of the database (maybe expanding the commit message would give us the idea about the RFC?)10:59
straycatsorry, yes probably should have11:00
jjardonkejiahu: right, thanks for checking11:03
*** zoli__ has joined #baserock11:03
kejiahujjardon: you are welcome :)11:05
straycatfranred, i updated the commit msg on
franredstraycat, cheers11:08
*** mariaderidder has quit IRC11:09
radiofreejust pushed a branch to, it worked, but what's trove got to do with it?
radiofree(got to do with it, got to do with it)11:10
ssam2you mean the 'remote: [] Notifying Mason of changes...' ?11:11 is a Trove11:11
ssam2and it has that hook which hasn't done anything for about 3 years11:11
ssam2I removed that in 'master' but I guess it must live in /home somewhere so it hasn't been fixed when upgrading git.baserock.org11:11
ssam2i'll try remove it manually if I can remember how11:11
radiofreeyes that remote thing11:12
ssam2i've removed the hook from, hopefully that will remove that useless message11:22
ssam2thanks for the reminder11:22
ssam2interesting boot failure on Jetson:
ssam2this is a report from our friends in Bulgaria11:44
ssam2it is built and deployed from the baserock-15.19.2 tag apparently11:44
ssam2pedroalvarez tested the 15.19 Jetson image on a Jetson, and didn't see this problem11:45
pedroalvarezindeed I tested that11:45
pedroalvarezthere are some variables: how have they flashed the image11:45
pedroalvarezmaybe more11:45
radiofreeroot=/dev/mmcblk1p2 ?11:45
radiofreelooks like some device tree issue as well, are they using a custom kernel or something?11:46
radiofreessam2: can you ask them what they're deployment cluster looked like?11:47
ssam2good idea11:47
radiofreessam2: they forget to set DTB_PATH11:48
radiofreeDTB_PATH: "boot/tegra124-jetson-tk1.dtb"11:49
radiofreealso i'm going to assume ROOT_DEVICE is incorrectly set as well11:49
radiofreeunless they want to boot off an sd card11:49
ssam2oh, they do want that11:50
*** mariaderidder has joined #baserock12:09
radiofreelorry request
ssam2radiofree: looks ok, +1 from me12:27
ssam2is anyone thinking about submitting a talk to linuxcon europe / or embedded linuxcon europe this year?12:28
ssam2it's in Dublin so not far for us Manchester folk12:28
rjekI also have a lorry request:
* rjek wants to experiment with running SpamAssassin and ClamAV inside a container in the short term12:41
ssam2is there a reason you need it to be mirrored by the Baserock project?12:47
rjekStrikes me as useful12:47
ssam2i didn't ask radiofree that because i guessed he wants to use it in our genivi-baseline reference system12:47
rjek(And it's only small)12:47
ssam2i've no problem with it, if you need it there, so +1 from me. but it's not a requirement to have stuff hosted in just to use it in a system built with Baserock12:48
ssam2it only needs to be mirrored in if you want to use it in one of the supported 'reference systems'12:48
rjekAnother use case I can think of is when working on a dev system in a dev system, and you suddenly need to use a tool you've not got, having a chroot to hand might be a big time-saver12:48
franredssam2, I was thinking to go12:50
ssam2rjek: do you plan to add it to one of the reference systems, then?12:53
rjekssam2: It might be handy in the dev reference system, yeah: but I plan on using it (at least in the short term) for building sysadmin systems.12:54
*** pacon has quit IRC12:56
ssam2so you want a 'sysadmin' system that's capable of downloading a Debian chroot into itself? seems reasonable if you want access to stuff from Debian inside a Baserock system12:58
radiofreei don't want my lorry anymore12:58
radiofreesorry if it's already been merged12:58
ssam2radiofree: i didn't merge it yet. also, you were right about the missing DTB_PATH12:59
* richard_maw grumbles about the lack of CONFIG_DEVPTS_MULTIPLE_INSTANCES in our kernels13:02
richard_mawit makes nspawn work less well13:03
ssam2i think that's needed for Docker also13:03
* richard_maw checks when DEVPTS_MULTIPLE_INSTANCES was introduced, to decide whether we can enable it for all our kernels, or just some of them13:04
rjekssam2: Yes, precisely.13:05
rjekDavePage: But I think it might be useful as a general tool in dev images too: the installed size is pretty tiny, as it's just a bunch of shell scripts13:05
rjekssam2, not DavePage.13:05
* rjek should probably also submit an appropriate morph definition and patch for dev system13:06
ssam2rjek: when would it be better than just using a Debian system, though?13:06
rjekssam2: Yes.13:06
rjekssam2: But that's not always an option :)13:06
richard_mawpff, CONFIG_DEVPTS_MULTIPLE_INSTANCES came out in 2.6.29, I'll enable it globally13:10
paulsherwoodmason looks unhappy13:11
paulsherwooddoes audiomanager build for other folks? not for me13:11
jjardonpaulsherwood: what is audiomanager? what error are you getting?13:19
pedroalvarezpaulsherwood: mason were really unhappy. I've dealing with them, and they seem to be happier now13:25
Kinnisonpaulsherwood: there were a large number of issues with audiomanager -- I believe Graham had some patches on baserock-dev which helped matters13:28
ssam2SotK: should be up and running for you now13:44
SotKssam2: thanks!13:44
ssam2web ui is on port 808013:44
richard_mawpaulsherwood: that only appears to contain warnings to me, not the errors, it's probably parallel make obscuring the issue13:48
* paulsherwood tries max-jobs 113:48
radiofreepaulsherwood: audiomanager built for me, but i have swap13:50
paulsherwoodwhat difference would swap make?13:51
rjekmoar RAMs13:51
richard_mawless likely to have compiles fail because the compiler ran out of memory13:52
* paulsherwood notices his VM only has 2 GB13:53
*** sherm_ has quit IRC14:03
jjardonssam2: hi, maybe you can reconsider now that definitions version has been bumped?14:05
ssam2once there's a new release of Baserock we can merge that14:15
ssam2I think if we merged it now, Baserock 15.19 would crash when trying to deploy 'master' of definitions14:15
ssam2and if we bump VERSION to 4 now, Baserock 15.19 will refuse to build 'master' of definitions14:16
jjardonah, ok, will -2 it then14:16
*** sherm_ has joined #baserock14:18
radiofreepaulsherwood: adding swap means you can compile qtwebkit on a jetson in about 1hour 20minutes instead of 4hours14:20
Kinnisonradiofree: nice14:20
* rjek is surprised it makes it faster (rather than simply possible.)14:20
rjekBut I suppose it frees up more actual RAM by swapping stuff out that is not in the working set14:20
richard_mawrjek: it means you don't have to work around the crashes by setting max-jobs: 114:21
Kinnisonrjek: swap lets you have datasets swapped, without it only text segments and RODATA segments can be paged.  As such it helps a lot when you have n parallel C++ compilations14:21
Kinnisonrjek: swapping out the datasets is way more efficient than constantly paging the code you're trying to run14:22
rjekKinnison: Aye.14:22
radiofreerjek: it's because of max-jobs14:24
* rjek nods14:25
*** rdale has quit IRC14:25
rjekI suppose the issues of in CPU power / memory latency are somewhat different on ARM, hand't really thought about that.14:25
*** rdale has joined #baserock14:26
* richard_maw does a little dance as CONFIG_DEVPTS_MULTIPLE_INSTANCES makes machinectl login work14:26
radiofreei do actually want my audiomanagerdemo lorry :(14:31
radiofreethis [13:10:30] <radiofree> lorry request
ssam2radiofree: it's easier for me if you can send it to gerrit14:34
ssam2but if someone else can give this a +1 i shall merge it from your patch. this time!14:35
straycat looks like it might be useful for a cpan import tool extension14:43
* paulsherwood confirms audiomanager builds with a 4GB vm14:44
paulsherwoodhow do we enable swap? can it be done by defauly?14:45
ssam2straycat: I think tlsa was looking at that14:45
rjekpaulsherwood: mkswap on a block device, then swapon that block device14:45
rjek(ie, say add another block device, if it comes up as /dev/vdc, mkswap /dev/vcd && swapon /dev/vcd)14:45
paulsherwoodrjek: you're assuming i understand what you're writing. could someone write up a guide on the wiki?14:46
rjekpaulsherwood: I am assuming that :)14:46
* paulsherwood doesn't, sadly14:47
rjekpaulsherwood: Swap requires another block device (partition, virtual drive, or whatever)14:47
* rjek tries to remember if you can swapon to a file on a file system; if so you might be able to create a big file with truncate(1) and swapon to that...14:47
paulsherwoodrjek: i get that, i'm just interested to know what the actual commands would be14:48
* rjek experiments.14:48
rjekpaulsherwood: If you had a new virtual drive as /dev/vdc, literally: mkswap /dev/vdc && swapon /dev/vdc14:48
paulsherwood(and ideally get them on the wiki, rather than have them lost in the mists of irc)14:48
rjek(This will obviously destroy any existing content on /dev/vdc)14:48
paulsherwoodand the command to make a new virtual drive?14:49
rjekswapon: /root/swap: skipping - it appears to have holes.14:49
straycatyou could make a loopback device?14:49
rjekpaulsherwood: That depends entirely on what virtualising tool you're using14:49
paulsherwoodbah :)14:49
rjek$ dd if=/dev/zero of=swap bs=$((1024*1024)) count=204814:51
rjek$ mkswap swap14:51
rjek$ swapon swap14:51
rjek(add 2GB of swap via a 2GB file in your file system)14:51
rjek(On future reboots, only `swapon swap` will be needed.)14:51
paulsherwoodcould this be done as system-integration commands for build and dev systems?14:52
radiofreepaulsherwood: i sent a guide to baserock-dev ages ago14:52
paulsherwooddid you?14:52
* paulsherwood searches14:52
rjekDate: Tue, 10 Mar 2015 11:36:01 +000014:52
rjekMessage-ID: <>14:53
rjekOr that :)14:53
straycattlsa, ping?14:54
* straycat gets the feeling importing from cpan will be even less yielding than pypi14:57
straycattlsa, did you look at gitpan?14:57
tlsahad not heard of it14:57
*** Krin has quit IRC15:28
*** a1exhughe5 has quit IRC15:59
*** gary_perkins has quit IRC16:00
*** jonathanmaw has quit IRC16:23
ssam2paulsherwood: there are simple instructions here:
ssam2oh, radiofree beat me too it16:35
ssam2to it16:35
paulsherwoodradiofree: /src/workspace/reference/baserock/baserock/definitions # swapon /src/swapfile16:43
paulsherwoodswapon: /src/swapfile: insecure permissions 0644, 0600 suggested.16:43
paulsherwoodswapon: /src/swapfile: swapon failed: Invalid argument16:43
jjardonpaulsherwood: chmod 600 /src/swapfile ?16:45
radiofreeit should work without the permissions16:45
radiofreeand does work for me Swap:     10485756     399324   1008643216:45
radiofreepaulsherwood: did you remember to do mkswap /src/swapfile16:46
paulsherwoodradiofree: yup16:46
paulsherwood524288 -rw-r--r--    1 root     root     536870912 May 18 16:42 /src/swapfile16:46
paulsherwood/src/workspace/reference/baserock/baserock/definitions # chmod 600 /src/swapfile16:47
paulsherwood/src/workspace/reference/baserock/baserock/definitions # swapon /src/swapfile16:47
paulsherwoodswapon: /src/swapfile: swapon failed: Invalid argument16:47
radiofreebtw i'm not sure how useful a 512M swap file will be...16:47
* radiofree isn't sure why he used 512M as the example16:47
radiofreepaulsherwood: sounds like something is broken, this works for me on a jetson16:48
*** sherm_ has quit IRC16:48
paulsherwoodwhich swapon => /sbin/swapon16:48
*** mariaderidder has quit IRC16:49
radiofreepaulsherwood: worked for me in a vm16:50
radiofreepaulsherwood: swapon --version16:50
paulsherwoodswapon from util-linux 2.26.116:51
radiofreethat's what i'm using on the jetson, it works there16:52
* radiofree tries to find a vm image with 2.26.116:52
* radiofree fails16:53
radiofreei'll download the latest br image16:54
radiofreeonly thing i can suggest is doing rm /src/swapon and going through the steps again16:55
radiofreesorry /src/swapfile16:56
*** bashrc has quit IRC17:00
radiofreeagain, worked for me in latest baserock17:04
*** ssam2 has quit IRC17:05
*** sherm_ has joined #baserock17:14
*** tiagogomes_ has quit IRC17:16
rjekOh, btrfs. :)17:28
rjekDon't do that :)17:28
rjekYou *might* be able to trick it by creating an ext3 file system in a loopback file, and then putting your swap file in that!  But I wouldn't recommend it.17:29
* paulsherwood decides not to try17:29
rjekAdd a new virtual drive to your VM :)17:29
paulsherwoodyes, that's probably the answer17:29
pedroalvarezpaulsherwood: ooi why adding swap?17:33
* radiofree never has a btrfs /src17:33
* rjek never has a btrfs :)17:34
*** sherm_ has quit IRC17:35
paulsherwoodpedroalvarez: to speed up build? :)17:36
radiofreepaulsherwood: it's only going to speed up the chunks that fail because they run out of ram, and we've set max-jobs: 1 to17:37
jjardonseems there are wip patches to add swap files to btrfs:
rjekProbably better to put it in a partition/LV anyway17:40
rjekBut on a fat dev PC, just throw loads more actual RAM at your VM17:40
* SotK found a script which creates swap on btrfs, I'm not sure how sane it was though :)18:00
jjardonHi, gnome-shell and perl stop lorrying (last commit ), can someone take a look, please?18:34
jjardon(last commit in g.b.o is from 2015-02-21 in the gnome-shell case, from 2012 in the perl one)18:36
pedroalvarezjjardon: thanks for reporting. I'll have a look18:51
jjardonpedroalvarez: thanks!18:52
pedroalvarez it's my duty :)18:52
*** zoli__ has quit IRC19:49
*** zoli__ has joined #baserock21:01
*** zoli__ has quit IRC23:07

Generated by 2.14.0 by Marius Gedminas - find it at!