| *** gtristan has quit IRC | 05:48 | |
| *** gtristan has joined #baserock | 06:16 | |
| *** paulsherwood has quit IRC | 07:15 | |
| *** SotK has quit IRC | 07:15 | |
| *** persia has quit IRC | 07:15 | |
| *** pedroalvarez has quit IRC | 07:15 | |
| *** dabukalam has quit IRC | 07:15 | |
| *** paulsherwood has joined #baserock | 07:16 | |
| *** SotK has joined #baserock | 07:16 | |
| *** pedroalvarez has joined #baserock | 07:16 | |
| *** dabukalam has joined #baserock | 07:16 | |
| *** barjavel.freenode.net sets mode: +v pedroalvarez | 07:16 | |
| *** persia has joined #baserock | 07:18 | |
| *** franred has joined #baserock | 08:21 | |
| *** toscalix has joined #baserock | 08:55 | |
| *** N-a-N has joined #baserock | 08:56 | |
| *** bashrc has joined #baserock | 08:58 | |
| *** edcragg has joined #baserock | 09:15 | |
| gtristan | Wow | 09:24 |
|---|---|---|
| gtristan | I am hot on the trail of figuring out the build issue of building sed/diff/tar against freshly built coreutils | 09:24 |
| *** rdale has joined #baserock | 09:25 | |
| gtristan | and it's... quite interesting... I think pedroalvarez will understand when I call it "fun" | 09:25 |
| * pedroalvarez is here waiting for some fun | 09:25 | |
| gtristan | :) | 09:26 |
| gtristan | I have reduced the problem to this: http://paste.baserock.org/ixiwuweduv | 09:26 |
| gtristan | basically, after compiling strace in the sandbox... I found that cat essentially fails to stat stdin | 09:26 |
| gtristan | interestingly... busybox bash/sh feed stdin to programs in different ways | 09:27 |
| gtristan | so, I think that cat is "ok" actually, something is obviously wrong and it's hard to pinpoint | 09:28 |
| pedroalvarez | wow, this might be too much fun for me! | 09:28 |
| gtristan | I have built a static binary of that program above, both with glibc and with musl | 09:28 |
| gtristan | hehe | 09:28 |
| gtristan | so in every case... echo "pony" | ./test... succeeds with stat | 09:29 |
| gtristan | the problem occurs with bash, never busybox sh... when the redirection technique is used: | 09:29 |
| gtristan | ./test <<_EOF\npony\n_EOF | 09:30 |
| gtristan | now, I also have several shells to test with... I have native musl busybox and native musl bash 2.15b | 09:30 |
| gtristan | those are static compiled | 09:31 |
| gtristan | then I have baserock staged busybox sh, and baserock staged bash 4.3+patches | 09:31 |
| gtristan | both of those dynamically linked against glibc | 09:31 |
| *** tiagogomes has joined #baserock | 09:32 | |
| gtristan | both static musl bash and baserock staged 4.3 bash fail this test when run inside the chroot | 09:32 |
| gtristan | outside of the chroot, I only have must static bash... and it passes the test ! | 09:33 |
| gtristan | when performed both on the virtfs share, or outside of the virtfs share | 09:33 |
| gtristan | s/must/musl | 09:33 |
| gtristan | so with all of these combinations handy, I think the interesting comparison is... static linked musl bash A.) Fails the test in the chroot... B.) Passes the test outside of the chroot | 09:34 |
| pedroalvarez | same problem if you redirect from a file? | 09:35 |
| pedroalvarez | hm...same static linked musl bash? or recompiled inside the chroot? | 09:35 |
| gtristan | I tried with the same original static linked musl bash which is default in the aboriginal image, from both inside and outside of the chroot | 09:37 |
| gtristan | aha | 09:38 |
| gtristan | Ok, I can reproduce completely without any baserock packages built | 09:38 |
| gtristan | I had been suspecting this but wasnt sure to believe it or not | 09:39 |
| gtristan | :-/ | 09:39 |
| gtristan | virtfs related... | 09:39 |
| pedroalvarez | maybe | 09:40 |
| gtristan | So, that test... with static musl bash... run in the chroot sandboxing script I have... without *any* baserock packages staged | 09:40 |
| gtristan | one sec... | 09:40 |
| gtristan | this one: https://github.com/gtristan/aboriginal-controller/blob/master/control/bin/sandbox | 09:41 |
| gtristan | that sandbox script basically assumes that things might be staged... and stages toybox/busybox/make/bash in the absence of already staged tools | 09:41 |
| gtristan | so that sandbox... on a fresh directory... PASSES the test outside of the virtfs share | 09:42 |
| gtristan | FAILS the test inside the virtfs share | 09:42 |
| gtristan | whew | 09:42 |
| gtristan | this has to do with mmap implementation of the filesystem at / | 09:43 |
| gtristan | yikes | 09:43 |
| * pedroalvarez feels like the best rubber duck in #baserock | 09:43 | |
| gtristan | haha | 09:44 |
| pedroalvarez | :) | 09:44 |
| gtristan | you know... in montreal there was (is?) a company called discrete logic | 09:44 |
| gtristan | they do stuff like image manipulation and such | 09:44 |
| gtristan | anyway, I knew some people worked there, ... they had this teddy bear in the common lounge | 09:45 |
| gtristan | and people were supposed to go and tell that teddy bear about their problems | 09:45 |
| pedroalvarez | hahaha | 09:45 |
| gtristan | not sure how well it worked :) | 09:45 |
| *** jonathanmaw has joined #baserock | 09:49 | |
| *** ssam2 has joined #baserock | 10:03 | |
| *** ChanServ sets mode: +v ssam2 | 10:03 | |
| gtristan | pedroalvarez, with help from teddy bears and rubber ducks in #bash, I was able to work around the issue | 10:07 |
| pedroalvarez | nice! | 10:08 |
| gtristan | seems that for that to work, bash does not require features from /... but it does require... probably the mmap implementation... from /tmp | 10:08 |
| gtristan | so bind mounting /tmp into the sandbox fixes this | 10:08 |
| gtristan | this was quite a ride :) | 10:09 |
| pedroalvarez | wow! it was | 10:09 |
| pedroalvarez | to end up mounting /tmp | 10:09 |
| pedroalvarez | haha | 10:09 |
| * gtristan had started with a line-by-line in tar's configure | 10:09 | |
| gtristan | narrowing it down to where it invokes cat with <<__EOF | 10:10 |
| gtristan | then finding that all the stuff it set in the environment didnt matter, but #!/bin/bash vs #!/bin/sh made the difference | 10:10 |
| gtristan | then to looking through cat.c | 10:10 |
| gtristan | downloading and building strace in the sandbox (which... we should totally have in some low level of baserock, btw) | 10:11 |
| gtristan | isolating the broken fstat64 | 10:11 |
| gtristan | and... down to this yes... mounting /tmp :) | 10:11 |
| robtaylor | gtristan: you may want to bindmount a mktemp -d to reduce potential attack surface on build server | 10:16 |
| *** ctbruce has joined #baserock | 10:21 | |
| gtristan | robtaylor, could be... could be just interesting to prevent accumulation of crap in /tmp in the aboriginal guest from build to build too | 10:26 |
| gtristan | I wonder if there is potential to fix the glibc build bug too | 10:27 |
| gtristan | i.e. we suffer from https://sourceware.org/bugzilla/show_bug.cgi?id=5033 | 10:27 |
| gtristan | perhaps glibc folks could be convinced to build the archive in /tmp | 10:28 |
| * gtristan deserves supper | 10:29 | |
| * gtristan back soon | 10:29 | |
| robtaylor | gtristan: you do ;) | 10:31 |
| robtaylor | gtristan: seems the last entry was a request for patch, so maybe we could just do that | 10:31 |
| *** gtristan has quit IRC | 10:33 | |
| *** gtristan has joined #baserock | 10:59 | |
| *** locallycompact has joined #baserock | 11:06 | |
| *** vgrade_w has joined #baserock | 11:08 | |
| pedroalvarez | oops | 11:13 |
| pedroalvarez | I think I missed this patch when upgrading fuse: http://git.baserock.org/cgit/delta/fuse.git/commit/?h=baserock/morph&id=0479b766923a01553366d8dbf197ac5dabf4b9d6 | 11:13 |
| pedroalvarez | edcragg: I guess fuse will fail to build in aarch64 | 11:14 |
| pedroalvarez | (for whenever you hit the error..) | 11:16 |
| tiagogomes | is that still needed ¬_¬ | 11:17 |
| pedroalvarez | tiagogomes: well, I would assume so | 11:17 |
| *** locallycompact has quit IRC | 11:18 | |
| *** franred has quit IRC | 11:21 | |
| *** locallycompact has joined #baserock | 11:25 | |
| * gtristan has never been so satisfied to see sed configuring | 11:34 | |
| *** franred has joined #baserock | 11:34 | |
| edcragg | pedroalvarez: haven't seen fuse fail if it has indeed been built in any of the systems i've build | 11:37 |
| edcragg | built* | 11:37 |
| pedroalvarez | edcragg: that change was merged this weekend | 11:43 |
| edcragg | fuse built successfully as late as the 9th of march | 11:47 |
| *** franred has quit IRC | 11:47 | |
| edcragg | so, depending on when it was merged, i'm not sure if it is required...? | 11:50 |
| pedroalvarez | what we merged was an upgrade to fuse 2.9.4, and when upgrading I missed that patch for aarch64 | 11:52 |
| pedroalvarez | that merge was this weekend, so my guess is that whenever you update your definitions, your build of fuse will fail | 11:52 |
| edcragg | right, ok, i'll change the ref and try to rebuild that at some point | 11:54 |
| edcragg | it's the sort of thing you'd expect upstream to have fixed already though | 11:54 |
| edcragg | i'm guessing | 11:55 |
| pedroalvarez | I can see a few patches upstream in master | 12:00 |
| pedroalvarez | but not in 2_9 branch | 12:01 |
| *** franred has joined #baserock | 12:01 | |
| pedroalvarez | so yes, fixed upstream, but not present in this release | 12:01 |
| edcragg | cool | 12:06 |
| *** franred has quit IRC | 12:49 | |
| gtristan | interesting... building btrfs-tools complains with: No package 'ext2fs' found | 13:08 |
| pedroalvarez | but fails? | 13:09 |
| gtristan | yes | 13:10 |
| gtristan | and fails | 13:10 |
| gtristan | oddly, I'm looking, and it really should be failing | 13:10 |
| gtristan | it's not configured with --disable-convert | 13:11 |
| pedroalvarez | is it build-logs time? | 13:11 |
| gtristan | and it's configure.ac clearly asserts ext2fs in the case that --disable-convert is not passed | 13:11 |
| * gtristan serves up build log | 13:13 | |
| gtristan | http://paste.baserock.org/esoqitepip | 13:13 |
| pedroalvarez | gtristan: dobule checking, btrfs-progs? | 13:16 |
| pedroalvarez | http://paste.baserock.org/okerarikop | 13:16 |
| gtristan | http://git.baserock.org/cgit/delta/btrfs-progs.git/tree/configure.ac?h=parser&id=19a806f575cd010734fa5793c4b3bfb49926dc62#n104 | 13:16 |
| gtristan | thats the id which is in definitions for btrfs-progs | 13:17 |
| gtristan | right, you say: checking for EXT2FS... yes | 13:17 |
| gtristan | but where could that possibly come from ? | 13:17 |
| pedroalvarez | maybe.. EXT2FS was provided by something in build-essential? | 13:17 |
| pedroalvarez | busybox? | 13:17 |
| gtristan | nope | 13:17 |
| gtristan | and I build busybox in build-essential in exactly the same way | 13:18 |
| gtristan | fwiw | 13:18 |
| pedroalvarez | e2fsprogs in core | 13:18 |
| pedroalvarez | /baserock/e2fsprogs-devel.meta: "usr/lib/pkgconfig/ext2fs.pc" | 13:19 |
| gtristan | aha | 13:19 |
| gtristan | I see | 13:19 |
| gtristan | ok, that's my bad :) | 13:19 |
| gtristan | pedroalvarez, got it... it's on the way :) | 13:20 |
| * gtristan had a bad e2fsprogs build | 13:21 | |
| gtristan | from a day or two ago | 13:21 |
| pedroalvarez | gtristan: btw, I've sent this patch yesterday: https://gerrit.baserock.org/#/c/1986/1 | 13:22 |
| pedroalvarez | added you and jjardon as reviewers, for whenever you have time. Should be quick | 13:22 |
| paulsherwood | folks, jjardon has raised this https://github.com/devcurmudgeon/ybd/issues/193 but my immediate reaction is to wonder whether this could be related to the extension itself? | 13:27 |
| pedroalvarez | paulsherwood: does ybd handle subsystems? | 13:31 |
| gtristan | I have _always_ had some issues with the writeext.py extension... usually because my host tooling was not *exactly* the versions that the extensions wanted | 13:32 |
| gtristan | not sure if this can be related | 13:32 |
| gtristan | but the extensions are finicky wrt to host tools | 13:33 |
| pedroalvarez | paulsherwood: if it does, maybe subsystems are being deployed after the main system is deployed.. and that will trigger this error | 13:33 |
| pedroalvarez | this check of the configuration extension is making sure that the path specified for the initramfs system actually exists | 13:33 |
| paulsherwood | pedroalvarez: i believe it does handle subsystems | 13:33 |
| paulsherwood | SotK implemented it, iirc :) | 13:34 |
| paulsherwood | (just in case he's listening :-) | 13:34 |
| pedroalvarez | heh, he won't be around today I think | 13:35 |
| *** vgrade_w has quit IRC | 13:37 | |
| *** vgrade_w has joined #baserock | 13:37 | |
| gtristan | looks promising | 13:40 |
| * gtristan is building systemd, btrfs-tools and fuse | 13:40 | |
| gtristan | time to pack it in, I think I'll restart a clean build and see if I can get as far building overnight | 13:40 |
| paulsherwood | jjardon: ah, i understand now, wrt https://github.com/devcurmudgeon/ybd/issues/195#issuecomment-196314829 | 13:44 |
| paulsherwood | all: wouldn't it make sense to have build-steps in DEFAULTS, rather than the schema? | 13:44 |
| paulsherwood | s/wouldn't it/surely it would/ | 13:45 |
| richard_maw | how do you define which keys are valid build steps? | 13:45 |
| paulsherwood | by having them in defaults, rather than in the build tool? | 13:46 |
| richard_maw | no I mean when you say foo-commands, how do you know that it's a valid build step? | 13:46 |
| paulsherwood | currently ybd falls back to https://github.com/devcurmudgeon/ybd/blob/master/ybd/config/defaults.conf | 13:46 |
| *** gtristan has quit IRC | 13:47 | |
| paulsherwood | a valid build step is what we say it is, i think? previously morph had it hard-coded iirc | 13:47 |
| * paulsherwood thinks schemas should define the syntax, not the content... but may be wrong | 13:48 | |
| paulsherwood | richard_maw: have i answered your question? | 13:49 |
| richard_maw | well, in the absence of a top-level "commands" key in the chunk definition file, the commands are syntactic elements, meaning you need to enumerate them in the schema as well as defaults | 13:49 |
| richard_maw | http://paste.baserock.org/ateworovuh | 13:50 |
| richard_maw | if you had the latter then you could say that every entry under "commands" is a command that your schema can validate the same way | 13:51 |
| paulsherwood | i don't think that's necessary, but i'm probably missing your point | 13:51 |
| richard_maw | ok, let's try it this way around | 13:51 |
| paulsherwood | are you -1 on my suggestion to put the build-steps into DEFAULTS? | 13:51 |
| richard_maw | how does the schema know which keys in a chunk definition are commands? | 13:51 |
| paulsherwood | why should the schema care? | 13:52 |
| richard_maw | because the schema needs to be able to tell you "you've put a string there, rather than a list of strings" | 13:52 |
| richard_maw | which it can do currently, because we define *-commands at the syntactic level, so those keys are special and it knows that those need to be validated as commands | 13:53 |
| *** gtristan has joined #baserock | 13:53 | |
| paulsherwood | ok... so you *are* -1 on my suggested approach? | 13:53 |
| richard_maw | how do you propose to validate commands after moving the definition of which keys are commands to the DEFAULTS file? | 13:55 |
| paulsherwood | i'm struggling to get my head round this again, so please bear with me... | 13:57 |
| paulsherwood | but note that... if the build-steps change in the schema, which is now *outside definitions*, then it means that definitions won't tightly define build-steps any longer... | 13:58 |
| paulsherwood | (i think) | 13:58 |
| paulsherwood | ... so we're introducing uncertainty (in the engineering sense) where none existed before, i fear | 13:58 |
| richard_maw | I'm all for defining the build steps in DEFAULTS (though I think they should be per-build-system rather than a global list). | 14:01 |
| richard_maw | I'm just failing to articulate | 14:01 |
| paulsherwood | this is a difficult topic | 14:02 |
| paulsherwood | can i get a pro-tem +1 on adding them to DEFAULTS while we reason about schemas separately? | 14:02 |
| * richard_maw no longer votes, so merge it if you want | 14:03 | |
| paulsherwood | ah, ok | 14:03 |
| paulsherwood | ssam2: ? | 14:03 |
| paulsherwood | others? | 14:03 |
| richard_maw | paulsherwood: short term I think your easiest approach is to require that commands match ".*-commands$", and change the schema something like: http://paste.baserock.org/dazikuwele | 14:08 |
| ssam2 | paulsherwood: are you envisaging something like this http://paste.baserock.org/ilequqanop in DEFAULTs, then ? | 14:09 |
| richard_maw | since if you can't syntactically tell which keys are meant to be commands, then you can't syntactically validate them | 14:09 |
| * paulsherwood has brainfade... can not compute this :/ | 14:11 | |
| richard_maw | schema knows that build-commands is a command-sequence because it's in the schema that the build-commands key has a value that's a command-sequence | 14:12 |
| richard_maw | because that's in the schema, you can validate build-commands is a command-sequence | 14:12 |
| ssam2 | basically the idea is allowing definitions.git to customise what command-sequences can be used, right? | 14:14 |
| ssam2 | so you could invent blabble-commands or whatever | 14:14 |
| ssam2 | the JSON-Schema schemas obviously can't validate such user-defined things, but that's not necessarily bad | 14:15 |
| ssam2 | the question for me is whether it's a good idea to allow user-defined command sequences, or whether it's better to have a specific set that everyone has to use | 14:15 |
| paulsherwood | ssam2: yup. in which case, inventing them in DEFAULTS seems sensible to me, rather than having to futz with spec.git | 14:17 |
| * richard_maw can't answer that, just that if you wanted to instead ignore user-defined commands then you lose the ability to catch that an optional field has been mis-spelled | 14:17 | |
| ssam2 | certainly, if they are to be user-defined then they should be defined in definitions.git, not spec.git | 14:18 |
| ssam2 | richard_maw: yes, it does mean you could accidentally write 'buld-commands' and the json-schema wouldn't catch that. but the build tool could check that manually | 14:18 |
| ssam2 | i guess allowing them to be user-defined is good, my gut feeling is to stick with the set that we have and mandate everyone use those, but i'm probably not thinking big enough here :-) | 14:19 |
| * richard_maw previously considered the idea for the kernel build-system | 14:19 | |
| richard_maw | since you want to have the selection of the base config and the enabling of customisations to be different steps | 14:20 |
| richard_maw | IMO | 14:20 |
| richard_maw | since you can then override them differently | 14:20 |
| richard_maw | so a per-device-config step and a baserock-userland-config step | 14:21 |
| richard_maw | since then you can put the usual configs in DEFAULTS and only have to deal with replacing the per-device ones to fix quirks | 14:21 |
| radiofree | so you'd just define the defconfig + any additional CONFIG options in your build instructions? | 14:22 |
| radiofree | rest is from a set of baserock defaults? | 14:22 |
| ssam2 | that's an interesting idea | 14:23 |
| richard_maw | radiofree: that kind of thing, yeah | 14:23 |
| radiofree | that would be rather nice | 14:23 |
| richard_maw | it'd certainly be handy to not have a whole bunch of duplicated config in every kernel | 14:23 |
| ssam2 | yes | 14:23 |
| ssam2 | on an unrelated note: https://bugzilla.gnome.org/show_bug.cgi?id=763619 :-) | 14:24 |
| paulsherwood | cool! | 14:25 |
| paulsherwood | jjardon: when you say "I've changed DEFAULTS to add a new build step" which commit did that, please? | 14:28 |
| pedroalvarez | ooi, is anybody looking at gerritbot? | 14:42 |
| jjardon | paulsherwood: https://gitlab.com/jjardon/baserock-definitions/commit/af6151eae706d12c881be7ab28eeb5625ec72814 | 14:45 |
| paulsherwood | jjardon: tvm | 14:46 |
| paulsherwood | jjardon: please try ybd master? | 14:54 |
| jjardon | paulsherwood: seems to be working, thanks! | 15:06 |
| paulsherwood | cool! | 15:06 |
| *** cyndis has quit IRC | 15:24 | |
| jjardon | hi, my system doesn't want to boot, I tihnk because baserock initramfs-scripts doesnt support a system without /sbin/init; richard_maw am I rigth? | 15:25 |
| pedroalvarez | without /sbin/init? :/ | 15:27 |
| pedroalvarez | which system is this ooi? | 15:28 |
| jjardon | one without sysv compatibilities (pure systemd) | 15:28 |
| jjardon | also same problem if init is in /usr/bin/init, for example | 15:29 |
| pedroalvarez | http://git.baserock.org/cgit/baserock/baserock/initramfs-scripts.git/tree/init#n125 | 15:29 |
| pedroalvarez | jjardon: you could set a different init= in the kernel command line | 15:30 |
| pedroalvarez | initramfs will handle that | 15:30 |
| pedroalvarez | so.. in a "pure systemd" system.. what do you have to run to boot the system if it doesn't have /sbin/init? | 15:31 |
| jjardon | pedroalvarez: /usr/lib/systemd | 15:32 |
| pedroalvarez | then... init=/usr/lib/systemd ? | 15:32 |
| pedroalvarez | hehe, I've no idea about these things tbh | 15:33 |
| jjardon | in out systems, /sbin/init is simply symlink to /usr/lib/systemd | 15:33 |
| jjardon | our* | 15:33 |
| pedroalvarez | true (to '/lib/systemd/systemd' in mine, but maybe that has changed) | 15:34 |
| *** cyndis has joined #baserock | 15:34 | |
| pedroalvarez | then, what I've just said, should work | 15:35 |
| pedroalvarez | even with initramfs | 15:35 |
| jjardon | Its still a bug in our initramfs scripts, I think | 15:37 |
| pedroalvarez | why? | 15:37 |
| pedroalvarez | our initramfs scripts default to /sbin/init, but it has support to read the 'init' arg from kernel command line | 15:39 |
| jjardon | because changing the kernel command line every time I boot or locally patch initramfs is not the correct solution, I think; but I think I can keep going with this, thanks! | 15:41 |
| pedroalvarez | i don't get "changing the kernel command line every time I boot" | 15:42 |
| pedroalvarez | if you set "KERNEL_ARGS: init=/path/to/whatever" in your cluster morphology, should work | 15:43 |
| jjardon | oh, didn't think about that, cheers | 15:44 |
| *** cyndis has quit IRC | 15:44 | |
| pedroalvarez | if you want to try it quickly, just modify extlinux.conf | 15:46 |
| *** paulw has joined #baserock | 16:12 | |
| *** cyndis has joined #baserock | 17:05 | |
| *** toscalix has quit IRC | 17:17 | |
| *** rjek_ has joined #baserock | 17:29 | |
| *** rjek has quit IRC | 17:37 | |
| *** rjek_ is now known as rjek | 17:38 | |
| benbrown_ | ssam2: I have no idea how to reply in gerrit in a way that will notify you, will you be CC'd as you have taken part? | 17:44 |
| pedroalvarez | benbrown_: I think the email will go to anyone in the reviewers list | 17:46 |
| benbrown_ | pedroalvarez: cool, ty | 17:46 |
| pedroalvarez | np :) | 17:46 |
| *** vgrade_w has quit IRC | 17:53 | |
| *** ctbruce has quit IRC | 17:53 | |
| *** ssam2 has quit IRC | 17:55 | |
| *** bashrc has quit IRC | 18:01 | |
| *** jonathanmaw has quit IRC | 18:03 | |
| *** tiagogomes has quit IRC | 18:21 | |
| *** edcragg has quit IRC | 18:26 | |
| *** locallycompact has quit IRC | 18:29 | |
| *** rdale_ct has joined #baserock | 19:50 | |
| *** rdale_ct_ has joined #baserock | 19:52 | |
| *** rdale has quit IRC | 19:53 | |
| *** rdale_ct has quit IRC | 19:56 | |
| *** edcragg has joined #baserock | 20:48 | |
| *** edcragg has quit IRC | 21:27 | |
| *** edcragg has joined #baserock | 21:31 | |
| *** rjek has quit IRC | 23:39 | |
| *** rjek has joined #baserock | 23:39 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!