paulsherwood | ssam2: ///win 22 | 00:02 |
---|---|---|
paulsherwood | bah! :) | 00:02 |
*** zoli__ has joined #baserock | 00:02 | |
*** zoli__ has quit IRC | 00:18 | |
*** zoli__ has joined #baserock | 00:25 | |
*** zoli__ has quit IRC | 02:21 | |
*** zoli__ has joined #baserock | 02:30 | |
*** zoli__ has quit IRC | 02:32 | |
*** zoli__ has joined #baserock | 03:45 | |
*** zoli__ has quit IRC | 04:02 | |
*** petefoth has quit IRC | 04:02 | |
*** CTtpollard has quit IRC | 04:02 | |
*** fay_ has quit IRC | 04:02 | |
*** mauricemoss_ has quit IRC | 04:02 | |
*** sambishop has quit IRC | 04:02 | |
*** nowster has quit IRC | 04:02 | |
*** paulw has quit IRC | 04:02 | |
*** flatmush has quit IRC | 04:02 | |
*** paulw has joined #baserock | 04:02 | |
*** CTtpollard has joined #baserock | 04:02 | |
*** sambishop has joined #baserock | 04:02 | |
*** fay_ has joined #baserock | 04:03 | |
*** mauricemoss_ has joined #baserock | 04:03 | |
*** nowster has joined #baserock | 04:03 | |
*** petefoth has joined #baserock | 04:03 | |
*** flatmush has joined #baserock | 04:03 | |
*** juergbi has quit IRC | 05:27 | |
*** juergbi has joined #baserock | 05:36 | |
*** mike has joined #baserock | 07:18 | |
*** mike is now known as Guest3380 | 07:19 | |
*** franred has joined #baserock | 07:32 | |
*** a1exhughe5 has joined #baserock | 07:49 | |
*** gfinney__ has joined #baserock | 08:16 | |
*** gfinney__ has quit IRC | 08:19 | |
*** gfinney__ has joined #baserock | 08:19 | |
*** gfinney__ has quit IRC | 08:20 | |
*** gfinney has joined #baserock | 08:20 | |
*** gfinney_ has joined #baserock | 08:20 | |
gfinney_ | Good morning folks | 08:20 |
*** gfinney has quit IRC | 08:23 | |
*** gfinney_ has quit IRC | 08:23 | |
*** gfinney has joined #baserock | 08:23 | |
*** gfinney_ has joined #baserock | 08:23 | |
*** mdizzle has joined #baserock | 08:35 | |
*** bashrc has joined #baserock | 08:58 | |
*** gfinney_ has quit IRC | 09:03 | |
*** rdale has joined #baserock | 09:06 | |
*** jonathanmaw has joined #baserock | 09:12 | |
*** zoli__ has joined #baserock | 09:27 | |
*** tiagogomes_ has joined #baserock | 09:29 | |
*** gary_perkins has joined #baserock | 09:34 | |
*** ssam2 has joined #baserock | 10:05 | |
*** ChanServ sets mode: +v ssam2 | 10:05 | |
SotK | Does anyone have opinions on how my OSTree artifact cache should deal with stratum artifacts, which are just a file containing a list of chunks? | 10:06 |
SotK | it seems pointless to store them using OSTree, so I'm planning to just store them in the cachedir at the moment | 10:07 |
ssam2 | yeah, they are basically just metadata | 10:07 |
ssam2 | so store them however you store the other meta files | 10:07 |
ssam2 | I guess you could create actual trees for them in the OSTree store, since they wouldn't take up any extra disk space | 10:08 |
ssam2 | and that would maybe make adding stratum-integration-commands easier at a later date... | 10:08 |
ssam2 | but do whatever's easiest for now I think | 10:08 |
persia | IF we move to a strataless model, where we have arbitrary named entities with contents, how would we deal with this? | 10:08 |
SotK | persia: consideration of this is part of why I asked for opinions, in case someone had already thought about that | 10:09 |
persia | I think I agree with ssam2, that doing whatever is easiest/fastest is the right answer for now. I mostly wanted to make sure that we didn't paint ourselves in a corner to not be able to handle differently organised constructs. | 10:10 |
persia | I haven't thought about it in depth, but I suspect that the easiest way is to have metadata associated with each cache object that can identify other cache objects that should be overlaid (and the order) before this cache object is overlaid for the cache object under consideration to be complete. | 10:11 |
SotK | that is my thinking too | 10:11 |
ssam2 | there are two approaches: either use metadata to describe components-that-contain-other-components, or create the union of the components at build time and store that | 10:12 |
ssam2 | we currently use the first approach for strata and the second approach for systems | 10:12 |
persia | My worry with the union-of-components model is the duplication of data. | 10:12 |
ssam2 | sure, but OSTree deduplicates all objects | 10:12 |
persia | So If I have two cache keys, one of which objects contains the other, OSTree will never download things twice? | 10:13 |
ssam2 | it deduplicates at the file level | 10:13 |
ssam2 | so if one contains the other, both contain the same files | 10:13 |
ssam2 | and it'll only download those files ones | 10:13 |
ssam2 | *once | 10:13 |
ssam2 | at least, in theory. I think that the `ostree pull` command still has quite a lot of inefficiencies in practice | 10:14 |
*** Krin has joined #baserock | 10:15 | |
persia | In that case, I'd be happy storing the entire object for strata, as that makes it more consistent. | 10:15 |
persia | And, in practice, doesn't mean any differences in download. | 10:15 |
persia | That said, downloading many small files is *much* slower than downloading an archive of those files (even ignoring compression), because of how TCP works. | 10:15 |
ssam2 | yes. I think there's quite a bit of thought going on right now as to how to make `ostree pull` fast enough | 10:16 |
*** zoli__ has quit IRC | 10:16 | |
ssam2 | walters is keen to not require anything special server-side, which makes it more complex | 10:16 |
ssam2 | i.e. a dumb http server should be able to serve an OStree repo | 10:16 |
ssam2 | i'm not sure if that restriction makes sense for us or not, though | 10:17 |
persia | I think it does. | 10:19 |
persia | I'd rather not impose strong software requirements server side for artifact caches. Anything that can provide a filestore should be sufficient. | 10:19 |
SotK | If we're worried about download speed, surely we could keep the server-side cache as it is and only use OSTree locally? | 10:22 |
*** lachlanmackenzie has joined #baserock | 10:24 | |
ssam2 | SotK: I think OSTree should be faster in theory than downloading each artifact as a separate tarball | 10:26 |
ssam2 | I just don't want to claim that it is faster, when I don't actually know | 10:26 |
ssam2 | pedroalvarez: how did you disable artifact uploading on mason-x86-64.baserock.org ? | 10:29 |
ssam2 | I want to do the same for mason-x86-32.baserock.org but i can't work out how | 10:29 |
persia | SotK: My revised opinion: test and benchmark a couple different things, and choose the fast one, unless doing so is onerous, in which case do it the way that seems possible :) | 10:31 |
pedroalvarez | ssam2: I've already done it, sorry I didn't say anything | 10:37 |
pedroalvarez | ssam2: I just removed a few lines of the /usr/lib/mason/mason.sh script | 10:37 |
ssam2 | oh, thanks | 10:37 |
ssam2 | better to comment them in future so it's clear that you did it :) | 10:38 |
pedroalvarez | true :/ | 10:40 |
pedroalvarez | I guess I like pressing 'd' in vim | 10:40 |
ssam2 | ^VjjjjjjjI#<esc><esc> | 10:45 |
ssam2 | it's only a few more keypresses... | 10:45 |
straycat | what is that magic | 10:53 |
ssam2 | block select, move down a bit, insert # at start of each line you selected | 10:53 |
straycat | i usually x,ys/^/#/ for that sort of thing | 10:54 |
ssam2 | i never thought of that. might be easier | 10:54 |
straycat | mmm but the visual selection seems more intuitive somehow | 10:54 |
rjek | Oh, I thought ssam2 had a face/keyboard interface | 10:59 |
rjek | But it's a vi command :) | 10:59 |
straycat | oh i finally get how this works | 11:00 |
straycat | okay i find the sub easier in this case | 11:00 |
ssam2 | yeah I think it's better too. thanks for the tip! | 11:02 |
straycat | heh np :) | 11:06 |
*** rdale_ has joined #baserock | 11:44 | |
*** rdale has quit IRC | 11:47 | |
franred | could I have a quick review of a tarball lorry? http://paste.baserock.org/ukaxotehiz | 12:08 |
* pedroalvarez can't open the url and hopes that nothing is going wrong with paste.baserock.org | 12:10 | |
straycat | i'm still waiting | 12:11 |
* jjardon neither | 12:11 | |
Kinnison | it opens for me | 12:11 |
franred | umm, I can not open paste.baserock.org ... | 12:11 |
Kinnison | you lot are all on one site, yes? Maybe a routing hiccough | 12:11 |
radiofree | pbo working for me (at home) | 12:12 |
straycat | yeah i can wget it from tardis just fine | 12:12 |
pedroalvarez | I couldn't access any of the machines of DC, but everything looks fine again | 12:12 |
ssam2 | franred: I'm pretty sure that's lorried already | 12:13 |
* pedroalvarez stops panicking | 12:13 | |
ssam2 | from bazaar | 12:13 |
ssam2 | do you need the tarball instead of the bzr repo for some reason? | 12:13 |
franred | ssam2, yes, I know, but we are having troubles to make horizon running from git and none from the tarball | 12:13 |
franred | Krin, can give you more information about this ^^ | 12:15 |
ssam2 | ok... if there's some special magic in the tarball that's missing from the bzr repo (i.e. you can't just generate the tarball from the bzr repo) then +1 | 12:15 |
straycat | ssam2, i think it's because the maintainer of pytz isn't expecting anyone to use his source repo, so he just releases tars and puts them on pypi | 12:16 |
straycat | so the source repo isn't packaged as a python distribution | 12:16 |
ssam2 | right | 12:17 |
straycat | i've seen a couple of other packages like this | 12:17 |
ssam2 | yeah, some maintainers of rubygems like to work this way too | 12:18 |
ssam2 | I guess the only solution is build a cool OS that's all built from git except for their components, so they can see what they're missing out on :) | 12:19 |
*** petefoth has left #baserock | 12:19 | |
franred | :) | 12:19 |
*** petefoth has joined #baserock | 12:20 | |
franred | any other quick review of the previous patch? or ssam2, can I take your review as a +2? | 12:20 |
ssam2 | +2 | 12:20 |
Krin | the bzr repo expects to be built and installed in an environment that we do not currently have in baserock, ripsum explained it but i did not understand his explanation. the long and short of it was that it was not possible to leave the pytz btz as it was because although it builds well enough to work as a dependanciy, actual use of the pytz module fails. | 12:20 |
franred | thanks | 12:20 |
straycat | 150224 12:32:36 [Warning] Unknown character set: 'uft8' | 12:35 |
straycat | 150224 12:32:36 [Warning] Aborted connection 66 to db: 'foom' user: 'foom' host: 'localhost' (init_connect command failed) | 12:35 |
straycat | >.> | 12:35 |
Kinnison | eep | 12:35 |
straycat | well i suck | 12:35 |
Zara | aw | 12:36 |
*** gfinney has quit IRC | 12:43 | |
tiagogomes_ | I've a device tree being installed to "boot/dtbs/3.19.0-rc5+/apm/apm-mustang.dtb". Does anyone knows how to change the install location to not include the kernel version dir | 12:49 |
*** gfinney has joined #baserock | 12:50 | |
*** gfinney_ has joined #baserock | 12:50 | |
radiofree | tiagogomes_: can't you just copy it out? | 12:50 |
tiagogomes_ | radiofree not really, we want the armv8 kernel to be generic, so we are installing all armv8 device trees | 12:51 |
ssam2 | anyone seen a btrfs loopback mount giving lots of 'loop: Write error at byte offset xxx, length 4096' errors? | 12:56 |
ssam2 | this is on an SSD attached to a Jetson, with Linux 3.18 | 12:56 |
ssam2 | hmm. Also our 'build' systems seem to lack fsck.ext4 | 12:57 |
ssam2 | eventually I get a kernel backtrace involving btrfs_commit_transation | 12:58 |
richard_maw | I've seen it happen with a flaky USB pass through for a USB attached device, but that was because the device had completely gone away | 12:59 |
nowster | ssam2: good luck in recovering that filesystem... | 13:01 |
ssam2 | luckily I don't need to. | 13:01 |
jjardon | Hey! ok to lorry the python3 cairo bindings? http://paste.baserock.org/deziloniza | 13:13 |
richard_maw | I think we put python modules in the python-packages/ subdirectory these days | 13:15 |
richard_maw | If we can confirm that this is the case, and that it's not already lorried somewhere I'll +2 the lorry | 13:16 |
*** zoli__ has joined #baserock | 13:17 | |
jjardon | richard_maw: I think it would be a bit strange to have the bindings in different places ( py2cairo and python-packages/pycairo in g.b.o ) , but no problems to me to rework the patch | 13:19 |
*** zoli__ has quit IRC | 13:21 | |
ssam2 | it's a bit strange to have 2 repos for one set of bindings | 13:22 |
ssam2 | most projects do python 2 and 3 out of one git repo | 13:23 |
richard_maw | some do python 2 and 3 out of the same commit, either writing the safe subset, or writing 2to3 compatible code | 13:25 |
jjardon | agree they seem to have created the repo just for python3 since version 1.8: http://cgit.freedesktop.org/pycairo/log/ | 13:26 |
ssam2 | right | 13:27 |
ssam2 | well, maybe put it in the same place as py2cairo then. I agree it seems weird to have 2 related repos in different places | 13:27 |
ssam2 | and we're not going to get to a point where we can move repos around on git.baserock.org any time soon | 13:28 |
jjardon | ok, so should I apply my patch as it is? | 13:28 |
ssam2 | +1 from me | 13:30 |
richard_maw | +1 from me too then | 13:35 |
*** zoli__ has joined #baserock | 13:37 | |
jjardon | merged! thanks! | 13:37 |
*** zoli__ has quit IRC | 13:45 | |
* SotK successfully builds a system using his OSTree artifact cache | 13:48 | |
straycat | cool | 13:54 |
SotK | building the actual system itself took 3:48, I don't know if that is any quicker than normal | 13:55 |
persia | The quoted numbers yesterday were ~5 minutes to build and ~10 minutes to download, so that's better than either. | 13:56 |
* straycat said < 5 minutes | 13:57 | |
straycat | i've never timed it though, so don't rely on that figure | 13:57 |
straycat | probably best to do your own experiements | 13:57 |
* SotK populates an old style cache to try to get a comparison | 13:57 | |
pedroalvarez | I'd like to have a comparison of sizes of the system-artifact | 14:03 |
jjardon | pedroalvarez: what is the best way to run a command in integration time?, should I add a .configure in morphlib/exts/ in the morph tree? | 14:04 |
jjardon | SotK: nice | 14:06 |
pedroalvarez | jjardon: is not well documented I'm afraid, but here there is an example: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/strata/core/ca-certificates.morph | 14:06 |
jjardon | pedroalvarez: thanks! | 14:07 |
pedroalvarez | this example will generate a sh file called 00-update-ca-certs-0000.sh, which will have inside the `update-ca-certificates` call. This script will be included in the ca-certificates-misc artifact. | 14:08 |
* SotK decides something has gone wrong with his test and tries again | 14:08 | |
pedroalvarez | this script will be called at system-artifact constuction | 14:08 |
SotK | pedroalvarez: the system artifact built with OSTree is 1.6M, I am only storing the files changed by stuff done when building the system | 14:09 |
persia | Note that this doesn't work if you need to build-depend on the results of the script. | 14:10 |
pedroalvarez | jjardon: indeed, note that ^^ | 14:10 |
pedroalvarez | SotK: awesome | 14:11 |
jjardon | pedroalvarez: ok, so simply calling "gdk-pixbuf-query-loaders" will put the stuff correctly in /usr/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache? or should I redirect to a specific PATH? | 14:14 |
pedroalvarez | jjardon: if you do what you said, it will have the same consecuences as if you run it on first boot | 14:18 |
jjardon | I mean, my understanding is that "gdk-pixbuf-query-loaders > /usr/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache" will work | 14:19 |
pedroalvarez | (these commands are run once the system is built, in a chroot) | 14:19 |
jjardon | rigth, thanks | 14:19 |
*** zoli__ has joined #baserock | 14:21 | |
*** zoli__ has joined #baserock | 14:21 | |
ssam2 | SotK: i was hoping for better than 3-4 minutes... would be interesting to see what's taking the time | 14:26 |
*** zoli__ has quit IRC | 14:26 | |
ssam2 | although the reduced system size is already a great improvement | 14:26 |
*** zoli__ has joined #baserock | 14:27 | |
SotK | ssam2: it looks like the system integration commands themselves are taking the time to me | 14:29 |
SotK | 2015-02-24 13:51:31 [Build 218/218] [test-system] Running the system integration commands | 14:29 |
SotK | 2015-02-24 13:54:16 [Build 218/218] [test-system] Removing staging area | 14:29 |
*** zoli__ has quit IRC | 14:30 | |
*** zoli__ has joined #baserock | 14:30 | |
pedroalvarez | I'd like to believe that the system-instegration commands are to not taking all that time | 14:32 |
Kinnison | extra debugging will probably point the finger | 14:33 |
SotK | I may be doing something silly, but its already a decent improvement (my comparison build has taken 10 minutes and is still going) | 14:35 |
SotK | 11:55 for the same system before the OSTree work | 14:36 |
ssam2 | sweet! much progress! | 14:36 |
SotK | system artifact for that is 1.8G | 14:37 |
franred | SotK, great!! | 14:39 |
* SotK adds some print statements to debug what on earth is taking so long, and the build takes 01:48 | 14:56 | |
SotK | I love it when print statements make things work better | 14:57 |
ssam2 | echo 2 > /proc/sys/vm/drop_caches and try again :) | 14:58 |
SotK | 05:04 :( | 15:10 |
SotK | 4 minutes of which was the runcmd_unchecked call which runs the system integration command | 15:12 |
SotK | I wonder if the overlayfs stuff is making its containerisation be slow maybe? | 15:12 |
richard_maw | possibly, but my guess is that the CoW overlayfs operation is making writes slow | 15:13 |
straycat | have folks done any kind of ntp configuration for openstack nodes? | 15:29 |
straycat | franred, pedroalvarez ^ | 15:29 |
franred | straycat, not really, I think ntp is configured by systemd now, but I can be wrong | 15:30 |
* straycat nods | 15:30 | |
* pedroalvarez hasn't | 15:30 | |
straycat | okay we could do that then, it'll be needed for swift | 15:30 |
straycat | when i say we i mean pdar/me | 15:31 |
franred | ssam2, Im testing your morph's branch "sam/sourceresolver-fixes", for now looks promising, and someone has to review it more, because fixes important bugs | 15:34 |
franred | someone, apart from me, I meant | 15:35 |
ssam2 | thanks for testing it | 15:35 |
ssam2 | I suspect not many people are using 'master' of Morph right now. If they were, they'd be hitting the bugs that branch fixes | 15:35 |
nowster | ah... I'm using "master". | 15:42 |
ssam2 | commit 88e8767791d85aa0d01715f04b7e0f311f54759e ? | 15:44 |
*** DavePage_ is now known as DavePage | 15:45 | |
SotK | richard_maw: I guess its not that, it takes ~1:30 without the OSTree stuff but with overlayfs | 15:52 |
SotK | as opposed to almost 4 minutes with the OSTree stuff | 15:53 |
ssam2 | that's weird. surely there is no ostree operation happening when running the system-integration commands? | 15:53 |
SotK | there shouldn't be | 15:53 |
ssam2 | perhaps the fact that the system is now a hardlink tree is making it slower? | 15:53 |
SotK | maybe | 15:54 |
jjardon | hi, about versioning of definitions: Do you think it should be a parameter of each morphology file or something that lies in the root directory of definitions? | 15:54 |
ssam2 | jjardon: this was already documented at the meetup I think | 15:54 |
ssam2 | http://wiki.baserock.org/meetup/declarative-definitions/ | 15:54 |
jjardon | great, thanks | 15:55 |
*** zoli__ has quit IRC | 16:03 | |
robtaylor | rdale_: how's it going with musl? | 16:20 |
rdale_ | ok - i've built build-essential with musl and it's all working, and builds chunks in core.morph | 16:30 |
rdale_ | i'm just working out how to apply the linux headers patched for musl in a 'kernel-headers' repo, to a branch based on the v3.12 in the lorried linux git repo | 16:32 |
Zara | hi, I'm getting an error when trying to build 'thrift' as part of building openstack-server.morph. I can paste the end of morph.log if this isn't something we've hit before. | 16:43 |
* SotK wonders how long deploy normally takes | 16:47 | |
SotK | s/deploy/rawdisk deploy/ | 16:47 |
franred | Zara, could you post the error? | 16:48 |
Zara | here's a paste, I included some of the earlier stuff in case it was useful, but there was too much to fit: http://paste.baserock.org/lagabizuzo | 16:50 |
franred | gen-c_glib/shared_service.h:86:69: error: unknown type name 'SharedStruct' --> that's the error. which version of thrift are you using? | 16:52 |
franred | Zara, ^^ | 16:52 |
* SotK boots a system built and deployed with an OSTree artifact cache :D | 16:53 | |
franred | :D :D | 16:53 |
SotK | deployment took ~7 minutes, 5 of which was doing the copy into orig | 16:55 |
Zara | franred: I'm trying to build from this branch, so it's the version of thrift here: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/?h=baserock/franred/openstack-v3 . Not sure where it's defined. | 16:55 |
ssam2 | SotK: sweet! how does it compare when you use tar.write ? | 16:56 |
SotK | ssam2: dunno yet, I'll do that and see | 16:57 |
franred | Zara, that branch should compile without errors, AFAICT - we haven't modify thrift since long time ago :/ | 16:57 |
Zara | yeah, this is why I'm really confused by it. :S I think it's probablyl something weird on my end, but I don't know what would affect it | 16:58 |
Zara | the build failed at step 362 of 407 | 16:58 |
Zara | *probably | 16:58 |
*** Krin has quit IRC | 17:03 | |
*** a1exhughe5 has quit IRC | 17:04 | |
SotK | ssam2: 7 minutes 20 seconds (rawdisk was 7 minutes 32 seconds) | 17:09 |
SotK | now to see how they compare with old deployments | 17:09 |
ssam2 | hmm, that's a bit unfortunate | 17:10 |
SotK | did tar used to be faster? | 17:10 |
ssam2 | I doubt it | 17:10 |
SotK | what is unfortunate then? | 17:10 |
ssam2 | that it takes 7 minutes instead of like, 3 seconds :) | 17:11 |
SotK | heh | 17:11 |
SotK | the "Writing to device" step took 6:33 | 17:11 |
*** zoli__ has joined #baserock | 17:20 | |
pedroalvarez | Maybe what about sysroot.write? | 17:21 |
pedroalvarez | s/Maybe// | 17:22 |
pedroalvarez | that one only puts the rootfs in a folder | 17:22 |
SotK | I'll try that once this old-cache deploy is done :) | 17:22 |
* SotK wonders how we managed for so long with such slow deployments | 17:25 | |
straycat | with despair | 17:27 |
straycat | and various hacks to reduce the number of times you needed to deploy | 17:28 |
straycat | it was even worse when we didn't have liw's xfer-hole thing | 17:28 |
pedroalvarez | deploying an installer system (3 systems in one) was really really slow | 17:29 |
ssam2 | i mostly deployed minimal systems with the gcc binaries removed, then ran them as containers | 17:30 |
ssam2 | but that doesn't work for all cases | 17:30 |
*** mdizzle has quit IRC | 17:38 | |
*** lachlanmackenzie has quit IRC | 17:43 | |
*** bashrc has quit IRC | 17:58 | |
* richard_maw grumbles that the minimal system currently contains gcc, and adds that to his list of things to look at tomorrow | 18:02 | |
pedroalvarez | :) | 18:03 |
pedroalvarez | richard_maw: not sure if this branch is relevant: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/log/?h=baserock/pedroalvarez/finer-splitting-gcc | 18:03 |
*** ssam2 has quit IRC | 18:05 | |
richard_maw | I've made a note of it | 18:06 |
*** jonathanmaw has quit IRC | 18:07 | |
*** Guest3380 has quit IRC | 18:17 | |
*** rdale_ has quit IRC | 18:36 | |
*** gfinney_ has quit IRC | 19:17 | |
*** gfinney has quit IRC | 19:17 | |
*** gary_perkins has quit IRC | 19:45 | |
*** zoli__ has quit IRC | 20:10 | |
*** gfinney_ has joined #baserock | 20:32 | |
*** gfinney has joined #baserock | 20:32 | |
*** zoli__ has joined #baserock | 20:45 | |
*** zoli__ has quit IRC | 20:49 | |
*** zoli__ has joined #baserock | 20:49 | |
*** gfinney has quit IRC | 21:36 | |
*** gfinney_ has quit IRC | 21:36 | |
*** gfinney_ has joined #baserock | 22:13 | |
*** gfinney has joined #baserock | 22:13 | |
*** gfinney_ has quit IRC | 22:14 | |
*** gfinney has quit IRC | 22:14 | |
*** gfinney has joined #baserock | 22:14 | |
*** gfinney_ has joined #baserock | 22:14 | |
*** gfinney has quit IRC | 23:18 | |
*** gfinney_ has quit IRC | 23:18 | |
*** zoli__ has quit IRC | 23:46 | |
*** zoli__ has joined #baserock | 23:47 | |
*** zoli__ has quit IRC | 23:52 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!