| *** Stanto has quit IRC | 00:59 | |
| *** gtristan has joined #baserock | 01:04 | |
| * paulsherwood believes ybd now culls artifacts by default as well as cleaning up tmpdir | 01:29 | |
| *** gtristan has quit IRC | 01:30 | |
| *** gtristan has joined #baserock | 01:51 | |
| gtristan | guys... I cloned git://git.baserock.org/baserock/baserock/lorry.git... I have some python scripts now | 02:27 |
|---|---|---|
| gtristan | how do I run the lorry on a single .lorry file to test the import with the command line ? | 02:27 |
| paulsherwood | gtristan: do you have lorryon your system? | 02:29 |
| paulsherwood | (iirc it's just `lorry lorryfile.lorry`) | 02:30 |
| gtristan | I have a checkout, does that qualify as "on my system" ? | 02:31 |
| * gtristan has unsatisfied deps... runs around looking for 'cliapp' and tries again | 02:31 | |
| paulsherwood | ack | 02:32 |
| gtristan | got it from package manager :) | 02:32 |
| gtristan | hmmm, seems to be doing something | 02:34 |
| gtristan | ok | 02:34 |
| gtristan | I think I got it from here :) | 02:34 |
| gtristan | it's doing git svn fetch on a huge repo | 02:35 |
| gtristan | hopefully we can avoid the webkit tarball | 02:35 |
| * paulsherwood crosses fingers | 02:35 | |
| gtristan | I'm trying this: http://paste.baserock.org/pimafimima | 02:36 |
| gtristan | it's hard to tell how they work, but if i recall how svn works, its all just basically paths | 02:37 |
| gtristan | so releases/WebKitGtk/... is where all the releases are found, they should basically act like branches | 02:37 |
| paulsherwood | should work i think | 02:39 |
| pedroalvarez | gtristan: check how long it takes and how big is the result | 02:44 |
| gtristan | hmmm | 02:45 |
| * gtristan enables timestamps in irc client | 02:46 | |
| gtristan | alright, can do | 02:46 |
| pedroalvarez | :) | 02:46 |
| * gtristan expects something around 6GB | 02:47 | |
| pedroalvarez | hmm.. IIRC that will be around 18G in g.b.o | 02:50 |
| pedroalvarez | maybe not that much | 02:50 |
| * pedroalvarez resurrects mason-x86-32 | 02:51 | |
| pedroalvarez | I'm happy to prepare the release artifacts if we want to do a release this week | 02:51 |
| * gtristan notices "I am fork 1" :) | 02:58 | |
| * gtristan is unsure about limiting max jobs | 03:00 | |
| gtristan | paulsherwood, have you tried just 2 instances but letting them just fight for cores/threads without limiting them explicitly ? | 03:01 |
| * gtristan unsure he understood what instances: 2 does... looks like both instances are trying to build curl at the same time | 03:04 | |
| * gtristan thinks he's just at a part of the build where builds cant be parallel, I've got 90 builds ahead so lets give 'instances' a chance to shine :) | 03:07 | |
| pedroalvarez | gtristan: I believe that paulsherwood said that he had that fixed, so that you don't have various instances building the same | 03:09 |
| pedroalvarez | check if you are using latest ybd | 03:09 |
| paulsherwood | gtristan: please use latest tag | 03:16 |
| paulsherwood | gtristan: max-jobs should be 8 for you | 03:17 |
| gtristan | ah | 03:26 |
| gtristan | I see | 03:26 |
| gtristan | lorry: SVN connection failed somewhere... | 03:26 |
| gtristan | :( | 03:26 |
| gtristan | any idea if it's possible to omit "tags" from the lorry ? | 03:28 |
| gtristan | I think a problem with WebKitGtk is that they work in strange ways, the "tags" in tags/* dont really match up with the branches in releases/WebKitGtk/ | 03:28 |
| gtristan | answer is yes | 03:29 |
| gtristan | I can ignore tags/* | 03:29 |
| * gtristan tries that | 03:29 | |
| gtristan | ybd app.py has reference to undefined variable in cull() | 03:36 |
| gtristan | paulsherwood, am I to trash my artifacts completely with this change, now that the treeserver is uncommented ? | 03:37 |
| gtristan | fixing that yields: Culled 0 artifacts in /home/tristan/ybd/artifacts | 03:40 |
| gtristan | so should I mv everything from ~/.cache/ybd -> /home/tristan ? | 03:40 |
| gtristan | ok, that seems to have fixed it | 03:42 |
| gtristan | ybd master, max-jobs still 4 | 03:44 |
| gtristan | and one zombie python, the other one building | 03:44 |
| gtristan | 1 zombie, and the one that is building I can confirm is limited to 4 gcc's at a time | 03:45 |
| * gtristan kills and tries again | 03:46 | |
| gtristan | ah | 03:46 |
| gtristan | sorry | 03:46 |
| gtristan | "latest ybd" != "latest tag" | 03:46 |
| * gtristan still gets max jobs 4 with tag 15.43 | 03:48 | |
| gtristan | and same situation, 1 zombie and one build limited to 4 jobs | 03:49 |
| gtristan | gah, ok so lorry calls logging.debug() all over the place... which is wonderful | 03:58 |
| gtristan | looking at https://docs.python.org/2/howto/logging.html ... says I should be able to enable it by adding the line: logging.basicConfig (level=logging.DEBUG) | 03:59 |
| * gtristan would have thought an env var would be nice... | 03:59 | |
| gtristan | but even that seems to not work | 03:59 |
| gtristan | ah, --log=DEBUG... gotcha | 04:00 |
| *** brlogger has joined #baserock | 04:19 | |
| paulsherwood | what is 'a zombie'? | 04:19 |
| gtristan | http://paste.baserock.org/oyodoresew | 04:20 |
| gtristan | process 18451 ^^^^^^^^^^^ | 04:20 |
| gtristan | paulsherwood, unreaped child process | 04:20 |
| gtristan | orphaned | 04:20 |
| paulsherwood | gtristan: ok i don't know what that is, i'm afraid | 04:21 |
| paulsherwood | can you paste your console output? | 04:21 |
| gtristan | http://paste.baserock.org/imoyiyeziv <-- ybd console output | 04:23 |
| gtristan | this is without instances: 2 though, I'm afraid | 04:23 |
| * gtristan gave up on that for now, need to get it built... | 04:23 | |
| * gtristan can try with explicit max-jobs: 8 and instances: 2 | 04:24 | |
| paulsherwood | right. ybd forks to do cleanups, for example. maybe that explains the orphan? | 04:24 |
| gtristan | as it will at least pick up where it left off | 04:24 |
| paulsherwood | that console output looks ok to me | 04:24 |
| gtristan | possibly, and it fails to reap the child ? | 04:24 |
| paulsherwood | i don't know what reaping is. the child processes are supposed to exit | 04:25 |
| gtristan | that's odd honestly, I would think that from python, you wouldnt have to take care of that | 04:25 |
| gtristan | reaping is basically trapping SIGCHLD and walling waitpid() on it | 04:25 |
| gtristan | cleanup work that is supposed to be done from the parent | 04:25 |
| paulsherwood | well, i think i'll not worry about it it for now :) | 04:26 |
| gtristan | normally I take it that an orphaned child means that something went wrong, process failed to complete, etc | 04:26 |
| gtristan | but sure, things seem to basically work anyway | 04:26 |
| paulsherwood | :-) | 04:27 |
| * gtristan interrupts and tries with max-jobs explicitly set to 8, and 2 instances | 04:29 | |
| gtristan | paulsherwood, do you expect to see "I am fork 2" ever ? | 04:30 |
| gtristan | or fork 1 is the extra instance ? | 04:30 |
| paulsherwood | yup | 04:31 |
| * paulsherwood has a plan to change the reporting to 'I am instance' but it's more fiddly than you might think | 04:31 | |
| gtristan | paulsherwood, I think you get more juice from instances: 2 when building ci.morph as you have much more orthogonal dependencies there | 04:34 |
| gtristan | make sense ? | 04:34 |
| gtristan | this *cough* again could probably be improved if every package had it's own nicely separate .morph file ;-) | 04:35 |
| * gtristan ducks incoming traffic | 04:35 | |
| * gtristan suspects something is wrong with the lorry process | 05:21 | |
| gtristan | So | 05:23 |
| gtristan | lorry has been running for an hour and a half | 05:23 |
| gtristan | and this is the output (debug enabled): http://paste.baserock.org/okebuzukof | 05:23 |
| gtristan | it's supposedly doing git svn fetch | 05:24 |
| gtristan | http://paste.baserock.org/ekisoroceq <-- lorry related ps axf output | 05:24 |
| gtristan | the git-svn process is apparently doing something | 05:24 |
| gtristan | according to top, it's not dead | 05:25 |
| gtristan | now, what confuses me... is that... I started this process in the lorry checkout directory | 05:25 |
| gtristan | and it created a 'workd' directory | 05:25 |
| gtristan | where I would *suspect* it's putting the ultra huge webkit checkout | 05:25 |
| gtristan | but | 05:25 |
| gtristan | du -hs reveals... that the 'workd' directory, after 1.5 hrs... has accumulated an entire 79MB ! | 05:26 |
| gtristan | nowhere near the 5GB I would have suspected at all | 05:27 |
| gtristan | Any clues ? | 05:27 |
| pedroalvarez | gtristan: but is it still running? | 05:33 |
| gtristan | pedroalvarez, I just got fed up and finally killed it | 05:34 |
| gtristan | think I have to investigate deeper whats going on | 05:34 |
| pedroalvarez | gtristan: it might be that it has a lot of history, and that takes a looong time | 05:35 |
| * gtristan has been cleaning up the rest of the webkit branch in parallel | 05:35 | |
| pedroalvarez | I normally go to the git folder being created and use git log to see how it goes | 05:35 |
| gtristan | pedroalvarez, does it download that separately from payload or smth ? | 05:35 |
| gtristan | hmmm | 05:36 |
| gtristan | ok lets see | 05:36 |
| gtristan | git log | wc -l says... only a mere 2838 lines of git log | 05:37 |
| gtristan | it's tiny :-/ | 05:37 |
| pedroalvarez | in your case that would be: .../workd/WebKitGtk/git | 05:37 |
| pedroalvarez | maybe git-tmp? I'm not sure | 05:37 |
| pedroalvarez | gtristan: I didn't understand your question :/ | 05:38 |
| gtristan | I did that in ./workd/WebKitGtk directly actually | 05:38 |
| pedroalvarez | git log? | 05:38 |
| gtristan | yeah | 05:38 |
| pedroalvarez | that is not a git repo I think | 05:38 |
| gtristan | ah ! | 05:39 |
| gtristan | because lorry puts the git in the current directory | 05:39 |
| gtristan | which was lorry/ | 05:39 |
| gtristan | so that was lorry history | 05:39 |
| gtristan | actually it was 90800 lines of git log | 05:39 |
| gtristan | checking in the git subdir | 05:39 |
| gtristan | so... looks like you are correct... perhaps it's intensively crunching a lot of history before... well before I dont know | 05:40 |
| gtristan | not sure how the payload of the commits can be separate from the history of it, but I suppose that is the case ? | 05:40 |
| gtristan | logs for each commit and then diffs after ? | 05:41 |
| pedroalvarez | I don't know :/ | 05:43 |
| pedroalvarez | I wonder if there is any git mirror already? | 05:44 |
| gtristan | pedroalvarez, yup there is | 05:49 |
| gtristan | Think we can hack around that one ? | 05:49 |
| gtristan | anyway, one problem is lorry is supposed to support svn, so we should be rejecting tarball patches to lorries if there is any VCS | 05:49 |
| gtristan | but, pedroalvarez ... look at this: http://trac.webkit.org/wiki/WebKitGTK/2.10.x#Howtoaddawebkit-2.10branchtoexistinggit-svnclone | 05:50 |
| * pedroalvarez opens link | 05:50 | |
| gtristan | there is a git mirror, but it only mirrors trunk, so what I did to test the webkit build, is clone the git mirror, and run those custom git-svn-foo stuff listed on that page | 05:50 |
| pedroalvarez | i see | 05:51 |
| gtristan | So my thought was hey, they are already basically showing us how to get the branch, there MUST be "the correct" way to do it with the svn type lorry | 05:52 |
| * gtristan wishes he could kill the lorry process and "resume" it later on, but hopes are not high | 05:53 | |
| * gtristan will have to eat lunch at some point | 05:53 | |
| pedroalvarez | I don't think that will work | 05:54 |
| gtristan | no it wont | 05:54 |
| gtristan | lorry patch will probably have to wait for tomorrow, I can run it while I sleep | 05:54 |
| * gtristan doubts that after lunch, the process will even have time to complete | 05:54 | |
| * gtristan doubts it will complete in less than 5 hours | 05:54 | |
| pedroalvarez | I'm happy to give it a try in g.b.o. | 05:54 |
| pedroalvarez | with fast internet, cpus and ram | 05:55 |
| gtristan | I'm not sure what the bottleneck is | 05:55 |
| gtristan | it's not downloading a lot, and it takes like 5% cpu here | 05:56 |
| gtristan | or less than 2.5 even | 05:56 |
| gtristan | pedroalvarez, http://paste.baserock.org/exelicecap | 05:57 |
| gtristan | shall I gerrit that to start with ? | 05:57 |
| gtristan | note the lacking "tags", the tags are actually pointless anyway | 05:58 |
| gtristan | I tried with "tags" and hit some snag, but that was before I discovered how to enable the python DEBUG logging output | 05:58 |
| pedroalvarez | if you do, please add a -1 note saying that you haven't tested yet how long/big could it be lorrying that | 05:59 |
| gtristan | anyway lorry python cases if "tags" in layout so I assume lorry is safe with tagless svn repo | 05:59 |
| gtristan | well, you want to try it on g.b.o ? | 05:59 |
| gtristan | I would appreciate that, at least it might complete before tomorrow | 06:00 |
| pedroalvarez | yes, I'll start it now ish | 06:01 |
| pedroalvarez | I'll let you know how it goes | 06:01 |
| gtristan | it's certainly doing something | 06:02 |
| gtristan | git log keeps showing new logs | 06:02 |
| gtristan | thanks | 06:02 |
| * gtristan is [10/229/300] with a fresh webkit build after the latest patch comments addressed | 06:02 | |
| * gtristan suspects people will still complain that I'm making "too many strata" | 06:03 | |
| gtristan | frustratingly slow this is, I solved building WebKit last week, I should be addressing other dependencies to gnome-initial-setup | 06:04 |
| gtristan | pedroalvarez, assuming that the lorry takes ~24 hours to convert from svn (pessimistically), is it at least reasonable to expect < 1hrs updates in the future ? | 06:07 |
| gtristan | i.e., it will just pull whatever is new ? | 06:07 |
| * gtristan goes for lunch | 06:16 | |
| *** gtristan has quit IRC | 06:17 | |
| pedroalvarez | it will just piull whatever is new | 06:19 |
| pedroalvarez | I failed to run that lorry, fails for me: http://paste.baserock.org/tuhamoduge | 06:19 |
| *** paulwaters_ has joined #baserock | 06:49 | |
| *** straycat is now known as lostduck | 07:01 | |
| *** gtristan has joined #baserock | 07:05 | |
| gtristan | pedroalvarez, http://paste.baserock.org/bovijexuye | 07:16 |
| gtristan | interrupted lorry invocation... resumes ! | 07:16 |
| paulsherwood | gtristan: i think 2+8 should still perform faster for (say) gnome system. if there's a bottleneck (only one thing to build), the other instance will wait | 07:19 |
| gtristan | paulsherwood, yes for sure | 07:20 |
| gtristan | I have noticed a couple of parallelizations | 07:20 |
| gtristan | they are not so frequent when building gnome (or WebKitGtk target), but they happen | 07:21 |
| paulsherwood | gtristan: please paste console output for that, i haven't seen anything like that since a couple of tags ago | 07:27 |
| *** ssam2 has joined #baserock | 07:29 | |
| *** ChanServ sets mode: +v ssam2 | 07:29 | |
| gtristan | paulsherwood, I generally inspect this by just typing ps axf repeatedly in the shell | 07:29 |
| gtristan | paulsherwood, I can't say that there is any ybd output to prove this, if that's what you mean | 07:29 |
| paulsherwood | gtristan: i'm not sure what you're seeing, then. if there's no output from an instance, it's either in mid configure/make/install/artifact-creation, or it's waiting on a timeout/lock-being-freed-on-the-bottleneck-build | 07:33 |
| paulsherwood | (do you have evidence to the contrary?) | 07:33 |
| gtristan | I havent been looking at the output of ybd to make a guess | 07:33 |
| * gtristan inspects his log | 07:34 | |
| gtristan | paulsherwood, when you say "I haven't seen anything like that since a couple of tags ago", you mean - you havent seen... "Infrequent parallelization in a gnome system build" ? | 07:34 |
| gtristan | I'm saying that I have seen some, but just not many, now that I have the latest "tag" | 07:35 |
| gtristan | it behaves generally well with hardcoded max-jobs to 8 as well | 07:35 |
| paulsherwood | ah, i thought you were meaning you'd seen 2 instances building the same thing | 07:35 |
| paulsherwood | gtristan: first char of log line is instance number reporting that line | 07:36 |
| gtristan | I have, I think at least | 07:36 |
| gtristan | shouldnt I ? | 07:36 |
| gtristan | only that its not *very* frequent... my comment from hours ago was just; that I suppose it happens much more when building ci.morph | 07:37 |
| gtristan | oh | 07:37 |
| gtristan | sorry, damn I have to read more carefully | 07:38 |
| * gtristan read building at the same time | 07:38 | |
| paulsherwood | gtristan: no, that shouldn't be possible now. instance 0 (say) claims a lock on building foo. if instance 1 finds itself wanting to build foo,it goes back to the top of the tree... if it finds itself back at foo and foo is still claimed, it waits on the claim or a timeout | 07:38 |
| gtristan | paulsherwood, yeah, I think I saw that at the beginning of the day when I did not update to latest tag yet, that's certainly fixed | 07:38 |
| paulsherwood | cool :) | 07:38 |
| gtristan | can I comment in Gerrit on my own patch ? | 07:40 |
| paulsherwood | i believe so | 07:41 |
| * gtristan just pushed a new branch, at this point, I can vouch for the correctness of ALL of the patches in that queue, _EXCEPT_ the WebKitGtk addition itself | 07:41 | |
| gtristan | oh, I can "Edit the commit message" but I can't seem to add a review | 07:42 |
| gtristan | that's good enough for now | 07:42 |
| paulsherwood | gtristan: shall i build it? | 07:42 |
| gtristan | paulsherwood, the branch wont build because the WebKitGtk does not have a backing lorry | 07:43 |
| paulsherwood | ah. | 07:43 |
| gtristan | I need to test/address the svn WebKitGtk lorry, get that upstream, and then modify the WebKitGtk.morph to point the repo to upstream:WebKitGtk | 07:43 |
| gtristan | for now, I've addressed comments, for jjardon, I've currently settled for upgrading libxml2 in 2 places | 07:44 |
| gtristan | so libxml2 gets built twice with an unpredictable result I guess, they overwrite eachother in the target | 07:44 |
| gtristan | (that should probably produce some error, maybe) | 07:44 |
| gtristan | and I moved ModemManager -> NetworkManager build, per another comment request | 07:45 |
| gtristan | and I folded enchant/hyphen into the WebKitGtk morph itself | 07:45 |
| gtristan | as per another comment | 07:45 |
| gtristan | so I think for now... just about everything in the queue is ready to go except for WebKitGtk itself | 07:45 |
| paulsherwood | gtristan: i'm confused... the webkitgtk page says get the code from http://www.webkit.org/building/checkout.html, which mentions git clone git://git.webkit.org/WebKit.git WebKit | 07:47 |
| gtristan | paulsherwood, that's a mirror | 07:48 |
| paulsherwood | of what? | 07:48 |
| gtristan | paulsherwood, look here: http://trac.webkit.org/wiki/WebKitGTK/2.10.x#Howtoaddawebkit-2.10branchtoexistinggit-svnclone | 07:48 |
| gtristan | its a mirror of the real upstream svn | 07:48 |
| gtristan | but _only trunk_ | 07:48 |
| paulsherwood | ah, i see | 07:48 |
| gtristan | so what I did locally actually to test my definitions build, is checkout that git which you mention | 07:48 |
| gtristan | and then fool around with those other tweaks | 07:49 |
| gtristan | yeah | 07:49 |
| gtristan | git log --oneline | wc -l ... tells my I've got up to almost 5000 commits... and counting | 07:51 |
| * paulsherwood mutters wtf in the general direction of folks who've decided to half-support git | 07:52 | |
| gtristan | they do seem rather disorganized, but it looks like a different crowd of "GTK Port" people who kindof work alongside trunk | 07:53 |
| gtristan | their branches and release tags are not all back in tags/ branches/ and I guess the code is not in trunk/ | 07:54 |
| *** bashrc has joined #baserock | 08:02 | |
| tiagogomes_ | mm, baserock doesn't have iptables (userland) | 08:06 |
| paulsherwood | strata/connectivity.morph:- name: iptables | 08:06 |
| paulsherwood | tiagogomes_: i think that's in the qt systems and the genivi | 08:08 |
| richard_maw | also ppenstack | 08:08 |
| richard_maw | its part of how neutron works | 08:09 |
| paulsherwood | and weston and xfce | 08:09 |
| * paulsherwood had failed to scroll down :) | 08:09 | |
| richard_maw | though i think it daft that openstack has libraries shelling out to iptables rather than making a proper library | 08:09 |
| paulsherwood | interesting... sounds like an opportunity for a contribution :) | 08:10 |
| tiagogomes_ | richard_maw they have a library... that shells out to iptables | 08:10 |
| tiagogomes_ | s/library/private module | 08:11 |
| richard_maw | thats what i meant but failed to unambiguously articulate | 08:11 |
| paulsherwood | in other news, i'm going to be demonstrating baserock ybd workflows on various combinations for a hands-on with ~50 people on thursday... | 08:11 |
| paulsherwood | which leads me to want to add some stuff about ybd on w.b.o homepage and guides | 08:12 |
| *** nowster has joined #baserock | 08:12 | |
| * richard_maw is fine with that | 08:13 | |
| paulsherwood | any thoughts/suggestions on how best to handle it? | 08:13 |
| *** mariaderidder has joined #baserock | 08:14 | |
| * paulsherwood notes we shoulf probably mention CIAT on homepage somewhere, since it is cropping up in conversations too | 08:14 | |
| *** nowster_ has joined #baserock | 08:15 | |
| *** nowster has quit IRC | 08:18 | |
| richard_maw | paulsherwood: I'm not great at documentation, but I think adding the YBD equivalent instructions for all the workflow guides you care about supporting with YBD would be a viable approach. | 08:27 |
| * tiagogomes_ anticipates confusion | 08:28 | |
| *** franred has joined #baserock | 08:29 | |
| wdutch | are trove git-hooks viewable in the webgui? | 08:30 |
| ssam2 | what do you mean by trove git-hooks ? | 08:31 |
| ssam2 | paulsherwood: i removed all the morph-specific branch and merge stuff from the wiki a while ago, so it may be that we can simply replace 'morph' with $build-tool | 08:31 |
| ssam2 | no | 08:31 |
| ssam2 | replace 'morph build' with $buildcommand perhaps | 08:32 |
| ssam2 | not ideal, but neither is having 2 x docs | 08:32 |
| tlsa | you could do a rosetta page like https://wiki.archlinux.org/index.php/Pacman/Rosetta | 08:33 |
| lostduck | that's a neat idea | 08:33 |
| franred | tlsa, +! | 08:34 |
| franred | +1 even | 08:34 |
| wdutch | ssam2: it's a while since I looked at gitano but I seem to remember there being serverside hooks, I think Kinnison used one for CIAT and I'm trying to find it | 08:35 |
| richard_maw | might be handy for explaining how things work compared to other build tools too, so I think it's a good idea, but shorter term paulsherwood needs something up, and the most expedient way to do that would be to s/morph/ybd/ | 08:35 |
| richard_maw | wdutch: they aren't in the standard refs namespace, I'll see if I can find an example for you | 08:35 |
| franred | richard_maw, or s/morph/morph or ybd/ | 08:37 |
| richard_maw | wdutch: part of it is that the repository that the global serverside hooks go in isn't viewable from the cgit, since it requires gitano-admin membership to do it | 08:37 |
| wdutch | :/ | 08:38 |
| wdutch | maybe it's with the mason source? | 08:38 |
| richard_maw | hm, there should be one there | 08:39 |
| wdutch | if I can even find that... | 08:39 |
| richard_maw | hm, and the default trove configuration is to not have anyone be gitano-admin directly, leaving trove-admin to do some operations by proxy | 08:42 |
| richard_maw | wdutch: I'll find you that example | 08:42 |
| richard_maw | wdutch: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/trove-setup.git/tree/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua?id=98cf2ab9c80c8efd0df9086353972e7dc73f40b8 | 08:44 |
| * wdutch curses weechat for being bad with long urls | 08:45 | |
| wdutch | thanks richard_maw | 08:46 |
| *** jonathanmaw has joined #baserock | 08:46 | |
| *** Stanto has joined #baserock | 08:48 | |
| *** Stanto has quit IRC | 08:49 | |
| *** edcragg has joined #baserock | 09:09 | |
| pedroalvarez | hm.. so weston fails to start because http://paste.baserock.org/ekiverifup | 09:14 |
| pedroalvarez | i wonder what will create a /dev/input/event* device | 09:14 |
| pedroalvarez | a mouse? a keyboard? | 09:15 |
| gtristan | pedroalvarez, and a power button | 09:15 |
| gtristan | all of those | 09:15 |
| pedroalvarez | aha | 09:16 |
| pedroalvarez | ok | 09:16 |
| *** Stanto has joined #baserock | 09:16 | |
| * gtristan thinks udev is what is creating those in devfs... and then systemd basically plays the role of pausing them and restricting access to them | 09:16 | |
| * gtristan had an xorg experience with that last week | 09:16 | |
| pedroalvarez | hehe, this is annoying | 09:17 |
| pedroalvarez | I've been testing the rpi, and failed because maybe it didn't have a mouse/keyboard | 09:17 |
| gtristan | I never really got to the bottom of it, installing the display manager properly magically fixed it | 09:17 |
| gtristan | aha, I see | 09:18 |
| pedroalvarez | also i had some fun trying to debug the board.. looks like the rpi hangs if I connect the power pin | 09:19 |
| pedroalvarez | probably because is more power than it needs | 09:20 |
| gtristan | fun | 09:20 |
| *** petefoth_ has joined #baserock | 09:24 | |
| *** fay_ has quit IRC | 09:24 | |
| *** edcragg has quit IRC | 09:24 | |
| *** rdale has quit IRC | 09:24 | |
| *** tiagogomes_ has quit IRC | 09:24 | |
| *** flatmush has quit IRC | 09:24 | |
| *** jonathanmaw has quit IRC | 09:24 | |
| *** CTtpollard has quit IRC | 09:24 | |
| *** bashrc has quit IRC | 09:24 | |
| *** gary_perkins has quit IRC | 09:24 | |
| *** petefoth has quit IRC | 09:24 | |
| *** petefoth_ is now known as petefoth | 09:24 | |
| *** bashrc has joined #baserock | 09:24 | |
| *** jonathanmaw has joined #baserock | 09:24 | |
| *** tiagogomes_ has joined #baserock | 09:24 | |
| *** CTtpollard has joined #baserock | 09:24 | |
| *** edcragg has joined #baserock | 09:24 | |
| *** flatmush has joined #baserock | 09:25 | |
| *** rdale has joined #baserock | 09:25 | |
| *** fay_ has joined #baserock | 09:25 | |
| *** gary_perkins has joined #baserock | 09:26 | |
| *** locallycompact has joined #baserock | 09:34 | |
| * wdutch doesn't know what to do about the CIAT git hook because he can't hack around with gbo | 09:43 | |
| richard_maw | wdutch: what do you need to do? | 09:44 |
| wdutch | whatever it is that cu10-trove does but sending it ciat1.baserock.org | 09:45 |
| jonathanmaw | update to the branch baserock/jonathanmaw/k1.0-fixes, the PowerOff button works | 09:45 |
| richard_maw | wdutch: ok, I can add you to gitano-admin, remember great power comes with great responsibility | 09:46 |
| wdutch | but I don't actually know what to change :( | 09:47 |
| wdutch | for one thing what is a MASON_ID? | 09:48 |
| richard_maw | wdutch: you should be able to clone ssh://git@git.baserock.org/gitano-admin.git now | 09:49 |
| richard_maw | wdutch: if you revert 0967983042b93c257235cd3b1fffff1605612245 you will have the old hook | 09:49 |
| wdutch | unlimited powah! | 09:50 |
| * wdutch will look at the hook | 09:50 | |
| gtristan | mkay | 09:52 |
| gtristan | so... gnome-initial-setup wants libpwquality | 09:52 |
| gtristan | which basically tests password quality | 09:53 |
| gtristan | libpwquality can't live without cracklib | 09:53 |
| gtristan | is not having cracklib intentional ? | 09:53 |
| richard_maw | not as far as I'm aware | 09:53 |
| jjardon | gtristan: why it would be? :) | 09:54 |
| gtristan | jjardon, because I think we are not using it for login/passwd | 09:54 |
| gtristan | so I thought maybe it's intentional | 09:54 |
| gtristan | I *could* just insert it at the gnome/ level, but generally I think a setup with cracklib is usually integrated lower in the OS | 09:55 |
| gtristan | thats just my impression from a few weeks of looking at setups and reading LFS manuals, though | 09:55 |
| jjardon | gtristan: integrate it lower in the stack then | 09:56 |
| gtristan | I think not right away, I suspect that may cause me to lose another day in PAM | 09:57 |
| jjardon | but that will take you more time probably :) | 09:57 |
| gtristan | also I am still wary of the idea that fiddling with the lower level package config for gnome effects the rest of the systems | 09:58 |
| jjardon | gtristan: I would say have secure pasword is beneficial for all the systems, but I see your point | 10:00 |
| * gtristan 's impression that every package should be in it's own individual stratum increases with each passing day | 10:01 | |
| jjardon | gtristan: I have the same feeling when I start to work on this | 10:09 |
| * gtristan remarks that lorry should probably bail out with a user friendly error when git-svn or mercurial-git are not installed | 10:09 | |
| jjardon | lower level stratum seems to more or less work: but it gets more difficult when you go up in the stack | 10:09 |
| gtristan | jjardon, I have some ideas of how it can all come together nicely | 10:09 |
| gtristan | first step would be that the morphologies need to understand that they can A.) Have optional dependencies, B.) Be configured with different flavors, C.) Generate different hash keys for the cache based on which flavor a package was compiled for ... and D.) Packages can also be ambivalent as to what dependency provides a required API, in some but not all cases | 10:11 |
| gtristan | jjardon, I think if we can truly embrace the huge complexity of what this is instead of trying to escape it by grouping things statically together, the potential is great | 10:12 |
| gtristan | jjardon, of course if falls apart with the *current* semantics, as it lacks features - once you start duplicating a morph because of a different compilation flavor/set of dependencies, duplication upwards becomes exponential the further up the stack you go | 10:14 |
| gtristan | but there should not be more than one definition of a given package (and our current approach already does, actually - i.e. libxml2 in GNOME builds twice) | 10:15 |
| *** edcragg has quit IRC | 10:19 | |
| jjardon | gtristan: that applies to entires stratum as well, btw (one stratum can depend on different stuff, depending of what you want to build) | 10:20 |
| gtristan | right | 10:20 |
| gtristan | well, I dont see the difference really | 10:21 |
| gtristan | when you say "entries in a stratum", this is just a question of putting all the definitions into one file instead of splitting them up into separate ones | 10:22 |
| gtristan | jjardon, ah, I see you meant to write "entire stratum", and I misread "entires" as "entries" (and interpreted that as entries in a stratum) | 10:24 |
| gtristan | but really, the stratum serves no real discernable purpose but to generalize these into a group - and in doing so we lose a lot of power and flexibility | 10:24 |
| gtristan | afaics (I'd love to be proven wrong :) ) | 10:25 |
| *** edcragg has joined #baserock | 10:26 | |
| jjardon | gtristan: agree, but we need to decide if we prefer flexibility or simplicity, I guess (if both can not be achieved) | 10:31 |
| *** edcragg has quit IRC | 10:31 | |
| jjardon | one of the major complains about yocto, for example (and advantage in our side) is that is difficult to understand what are you building taking a look to the yocto recipes. Nobody discussed that yocto is very flexible, though | 10:32 |
| *** edcragg has joined #baserock | 10:38 | |
| gtristan | jjardon, I think with the right approach we can get our tooling to tell us exactly what will be built for a given high level recipe | 10:39 |
| gtristan | honestly I have a hard time seeing middle ground | 10:39 |
| gtristan | tools like buildroot duplicate *everything* into a separate build tree iirc | 10:40 |
| gtristan | so at least you have 100% certainty of what it is you are building | 10:40 |
| gtristan | with the present situation, taking libxml2 for example... you know using your common sense that libxml2 will only build an extra module if python is present | 10:41 |
| gtristan | and so the library will be installed twice but not differently | 10:41 |
| gtristan | but we do not *know* that with certainty, that the built library is bit-for-bit exactly the same when built with python present | 10:41 |
| gtristan | this imo is a level of uncertainty that I personally cannot trust | 10:42 |
| jjardon | agree | 10:42 |
| *** edcragg has quit IRC | 10:42 | |
| jjardon | everything should be builded only one time, we agree on that; this case is a special case as I can not think in another solution | 10:43 |
| gtristan | well, there have been others; ICU for instance | 10:44 |
| jjardon | gtristan: talking about that: https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/definitions+branch:master+topic:libepoxy_1_3_1 :) | 10:44 |
| gtristan | grouping these stratum together is prone to this kind of incident | 10:44 |
| jjardon | gtristan: beleive that it used to be much more :) | 10:44 |
| gtristan | and then untangling them becomes more difficult too | 10:44 |
| gtristan | because, now we have lost some information; we know that 'gnome' depends on 'foundation', but we forget exactly why an individual component in 'gnome' depends on a given component below | 10:45 |
| gtristan | jjardon, so; I think it's obvious at any rate that libxml2 & xslt in the present model do *not* belong in core anyway | 10:45 |
| ssam2 | completely agree on the fact that we lose dependency information with the current model, i would like to change it something more flexible | 10:45 |
| ssam2 | i'm not sure I understand your comment about python and libxml.. | 10:46 |
| ssam2 | the cache key of an artifact includes all dependencies | 10:46 |
| ssam2 | so a build of libxml with python available will be a different artifact to a build of libxml without python available | 10:47 |
| jjardon | gtristan: happy to move them out from core | 10:47 |
| ssam2 | that's not a great way of expressing whether or not the python module was built, but it *is* trustworthy | 10:47 |
| gtristan | jjardon, right, the point is I can not efficiently do it because now... other components will be depending on 'core' for libxml2... since we have lost that knowledge base in grouping things, it will take a lot more work to move it out of 'core' | 10:48 |
| gtristan | cause every build need validation again | 10:48 |
| gtristan | ssam2, that's good to know that the cache key computes it's dependency... in this particular case, libxml2 is installed *twice* under a different morph name | 10:49 |
| *** edcragg has joined #baserock | 10:49 | |
| gtristan | ssam2, it's hard to say right now, which of the two will end up in the target image | 10:49 |
| ssam2 | yeah | 10:49 |
| gtristan | but both are certainly built and installed | 10:49 |
| ssam2 | originally the plan was to disallow such "overlaps" completely | 10:49 |
| ssam2 | but that's never proved practical | 10:49 |
| gtristan | it would be amazing if that level of security was possible | 10:50 |
| gtristan | but there are many exceptions to catch | 10:50 |
| ssam2 | Gentoo does it, I thinj | 10:50 |
| ssam2 | *think | 10:50 |
| gtristan | as make install tends to update things in common | 10:50 |
| gtristan | so for instance, ld.so.cache is the first exception | 10:50 |
| jjardon | gtristan: exactly; I think I have the record of breaking the ci so believe I though the same as you are saying several times; looking forward to heard about that propossal to fix the current situation :) | 10:51 |
| gtristan | things which essentially get updated again at system-integration time are normally overlaps | 10:51 |
| gtristan | jjardon, it will need brainstorming, I have a couple of paragraphs but I need to bullet-proof it more | 10:52 |
| gtristan | have discussed it with pedroalvarez casually over beers, but he was obviously exhausted from the long flight :) | 10:53 |
| gtristan | anyway it's good to hear that there is some consensus at least on IRC right now on the things which need fixing - so we can probably move in the same direction in this if a good solution is found & tried :) | 10:54 |
| gtristan | eek, lorried in 12000 webkit commits so far | 10:57 |
| gtristan | only 388MB used | 10:57 |
| gtristan | and that took.... at least 5hrs | 10:58 |
| * gtristan is glad that it can resume where it left off, and hopes it will complete overnight | 10:58 | |
| ssam2 | is that an SVN conversion? | 10:58 |
| ssam2 | ouch | 10:58 |
| gtristan | ssam2, yup | 10:59 |
| gtristan | I dont even know if it *worked* | 10:59 |
| gtristan | ssam2, http://paste.baserock.org/ujivaniten fwiw | 11:00 |
| gtristan | that should, in theory get us a git mirror of trunk and all the related webkitgtk release branches | 11:00 |
| ssam2 | ah... makes sense | 11:01 |
| ssam2 | we would probably have to hack it into working on git.baserock.org, as it kills long running jobs normally | 11:01 |
| ssam2 | if your conversion succeeds maybe we can just rsync that to the correct place :) | 11:01 |
| gtristan | I am hoping that it would only be one shot | 11:02 |
| gtristan | right | 11:02 |
| gtristan | and every consecutive pull would only pull the commits of the day | 11:02 |
| *** edcragg has quit IRC | 11:04 | |
| * gtristan pushes lorry patch for libpwquality: https://gerrit.baserock.org/1272 | 11:06 | |
| gtristan | thats a mercurial, tested with lorry :) | 11:07 |
| * gtristan packs it up and changes locations... | 11:08 | |
| *** gtristan has quit IRC | 11:11 | |
| *** edcragg has joined #baserock | 11:16 | |
| *** gtristan has joined #baserock | 11:39 | |
| gtristan | ok, frustratingly, I lost all that data after resuming the interrupted lorry failed | 11:45 |
| gtristan | I will run it overnight, but can someone run lorry on this: http://paste.baserock.org/ujivaniten... perhaps on a server instead of a workstation ? | 11:46 |
| gtristan | I have no idea how long it will take, it could literally take an entire week if it continues at the given speed | 11:46 |
| pedroalvarez | jonathanmaw: ta for the fixes :) | 12:14 |
| pedroalvarez | btw, did you know that weston in the gdp fails to start if you don't have a keyboard or mouse connected? | 12:16 |
| pedroalvarez | With this error: http://paste.baserock.org/ekiverifup | 12:17 |
| *** paulwaters_ has quit IRC | 12:23 | |
| CTtpollard | I've seen people post similar errors with 1.6 on the genivi-projects / automotive-discussion list pedroalvarez | 12:24 |
| CTtpollard | 'I've tried weston-editor on my rpi2 and it will crash when no real | 12:25 |
| CTtpollard | keyboard is present' | 12:25 |
| pedroalvarez | CTtpollard: good to know. I panicked... | 12:26 |
| persia | CTtpollard: How does it detect a "real keyboard"? | 12:33 |
| CTtpollard | persia: I don't know, I was just quoting from a thread | 12:34 |
| rjek | trying to open a hard-coded /dev/input node? | 12:34 |
| persia | CTtpollard: You can probably work around it with a suitably configured uinput call | 12:35 |
| persia | mod_uinput allows userspace to inject events into the HID event queue, which can be useful for all sorts of things. | 12:36 |
| *** paulwaters_ has joined #baserock | 12:48 | |
| wdutch | richard_maw: how do I get my changes to ssh://git@git.baserock.org/gitano-admin.git take effect? | 12:55 |
| richard_maw | wdutch: should happen on push | 13:00 |
| *** ssam2 has quit IRC | 13:01 | |
| wdutch | is there are process or can I just push whatever? | 13:02 |
| richard_maw | wdutch: if you want me to look over the commit log, I'll do that, otherwise not really | 13:02 |
| wdutch | prepare for everything to break! | 13:04 |
| * CTtpollard braces | 13:04 | |
| wdutch | I'm going to push then run to the nearest doorframe | 13:04 |
| lostduck | gtristan, if you do move libxml and xslt, beware that the docutils stratum expects to find them in core so docutils will most likely need to be updated as well | 13:07 |
| lostduck | i also agree that strata could do with having something like nixos's build-inputs, or else there should be somewhere to make it clear what a chunk's direct dependencies are | 13:08 |
| lostduck | *some way | 13:08 |
| *** ssam2 has joined #baserock | 13:16 | |
| *** ChanServ sets mode: +v ssam2 | 13:16 | |
| wdutch | urg the new ciat has changed it's IP | 13:22 |
| *** paulwaters_ has quit IRC | 13:42 | |
| *** paulwaters_ has joined #baserock | 13:47 | |
| *** ssam2 has quit IRC | 14:10 | |
| *** ssam2 has joined #baserock | 14:24 | |
| *** ChanServ sets mode: +v ssam2 | 14:24 | |
| *** paulwaters_ has quit IRC | 15:01 | |
| wdutch | the ciat hook in gbo is working :) | 15:26 |
| *** mariaderidder has quit IRC | 15:30 | |
| *** nowster_ has quit IRC | 15:34 | |
| pedroalvarez | wdutch: good! :) | 15:35 |
| wdutch | morph is complaining about "ImportError: No module named fs.tempfs", what do I need to fix this? | 15:55 |
| ssam2 | install pyfilesystem | 15:57 |
| *** franred has quit IRC | 15:57 | |
| wdutch | thanks | 15:58 |
| ssam2 | there are a bunch of different things on pypi with similar, some of them even install 'fs' packages but without the right things in | 15:58 |
| ssam2 | 'fs' is the current one I think | 15:58 |
| ssam2 | *correct | 15:58 |
| ssam2 | or it might be in your distro with a differentname | 15:58 |
| wdutch | fs worked | 15:59 |
| * richard_maw wasn't a fan of pyfilesystem last time he saw it | 16:02 | |
| richard_maw | it assumed that all valid file paths are encodable as unicode | 16:03 |
| * pedroalvarez wonders where are we running morph that doesn't have all the dependencies | 16:04 | |
| jmacs_ | richard_maw: (*thinks*) are they not? | 16:04 |
| richard_maw | jmacs_: You could encode some sort of escaping system on top of unicode for them, but you can have paths that don't form valid unicode strings | 16:06 |
| * richard_maw articulated that badly | 16:07 | |
| richard_maw | this wasn't a theoretical problem either, we had builds fail because of it | 16:07 |
| rjek | The only byte values that are not valid in a POSIX pathname are NUL and / | 16:07 |
| rjek | ISTR libgmp had fruity filenames in its build system? | 16:08 |
| richard_maw | yeah, not sure what caused that | 16:08 |
| rjek | I think the issue was that it was assuming they were valid UTF8 or something? | 16:09 |
| jmacs_ | So it had bytes 128-255 in there somewhere? | 16:10 |
| rjek | Something like that, yeah | 16:11 |
| * rjek is fuzzy on the details, but it caused all sorts of problems | 16:11 | |
| *** jmacs_ is now known as jmacs | 16:13 | |
| jmacs | Invalid UTF8, rather than invalid unicode, makes a bit more sense. | 16:15 |
| *** ssam2 has quit IRC | 16:23 | |
| lostduck | can anyone think of a chunk we have that uses submodules? | 16:25 |
| radiofree | gstreamer | 16:25 |
| radiofree | lostduck: http://git.baserock.org/cgi-bin/cgit.cgi/delta/gstreamer.git/log/?h=baserock/morph/1.2 | 16:25 |
| lostduck | radiofree, thanks | 16:26 |
| *** locallycompact has quit IRC | 16:26 | |
| radiofree | are you writing something to automatically do they "Fix .gitmodules to use baserock upstream" thing? :-o | 16:26 |
| radiofree | s/they/the | 16:26 |
| lostduck | no, i just wanted to quickly check that we can use repo aliases there | 16:27 |
| radiofree | ah | 16:28 |
| *** jonathanmaw has quit IRC | 16:35 | |
| *** bashrc has quit IRC | 17:00 | |
| * gtristan wonders if we could 'man-in-the-middle' the git submodule functionality, using some mapping of the existing lorries - and not patch or modify the .gitmodules files at all | 17:05 | |
| persia | breaks signatures, but in practice, git doesn't support a provenance horizon through the results of mirroring, so it isn't worse. | 17:31 |
| *** edcragg has quit IRC | 18:01 | |
| paulsherwood | wdutch: what's going on with ciat, please? it seems to be not actuallty building anything. if it's not used, it shoudl be turned off, or a at least moved to a cheaper box | 18:27 |
| pedroalvarez | paulsherwood: I believe he has started the migration of it to a cheaper box | 18:31 |
| *** radiofree has quit IRC | 18:39 | |
| *** radiofree has joined #baserock | 18:41 | |
| paulsherwood | pedroalvarez: yup. but in the meantime why is the live one sitting there inactive? | 19:15 |
| paulsherwood | (i could understand it sitting there running builds, or being turned off) | 19:16 |
| wdutch | paulsherwood: I haven't turned it off because I thought people were watching it | 21:45 |
| wdutch | I'll happily turn it off whenever | 21:45 |
| wdutch | although not right now because I don't have the login details to hand | 21:46 |
| paulsherwood | wdutch: the mystery is why is it not actually building? i thought it was tracking systemd and linux? | 21:59 |
| paulsherwood | ... people are watching it. and wondering why it's doing nothing | 21:59 |
| wdutch | I'll have a look | 21:59 |
| paulsherwood | tvm | 21:59 |
| wdutch | the last build was an hour ago | 22:00 |
| wdutch | oh my bad | 22:00 |
| wdutch | 10pm yesterday | 22:00 |
| wdutch | trove notifications are happening, just not to systemd or linux | 22:02 |
| wdutch | this is expected behavior | 22:03 |
| paulsherwood | not for me :) | 22:18 |
| persia | wdutch: Are no commits happening upstream? | 22:25 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!