IRC logs for #baserock for Saturday, 2015-03-21

*** zoli__ has quit IRC07:02
*** zoli__ has joined #baserock07:02
*** zoli__ has quit IRC09:04
*** zoli__ has joined #baserock09:04
*** zoli___ has joined #baserock09:06
*** zoli__ has quit IRC09:06
*** zoli___ has quit IRC09:17
paulsherwoodthinking furher about forward-backward compatibility i made
paulsherwoodi'll add other examples if i get time10:25
rjekMorning, paulsherwood.10:25
straycatjjardon, didn't seem to, if there's no version file the function returns None rather than 010:25
* rjek looks10:25
paulsherwoodmorning rjek :)10:26
paulsherwoodit's still fuzzy in my mind - an old system may or may not have an old morph on it10:26
rjekforwards/backwards compatability is always a pain when there are so many variables involved, but it's often worthwhile pain.10:27
paulsherwoodyup. am trying to help clarify policy/approach, given i expect rate of change to have to increase soonish10:28
paulsherwoodand it would be a shame to p*** off users with increasing frequency if we can avoid it :-)10:29
paulsherwoodstraycat: does that make it behave differently, though?10:31
* paulsherwood has morph and definitions in very specific pieces at the moment so can't check trivially10:31
straycatright now, probably not, but it could if we have sections of code that are conditional on the version number i think10:32
paulsherwoodah, correct10:33
rjekWe've had quite a few instances recently where people have come in here and said "I get this error", to which the solution is "upgrade your morph to master"; even having a more helpful error here would be an improvement.10:33
paulsherwoodhmm... i guess you don't mean to suggest that all morph errors include the line 'please upgrade to latest morph' ? ;-)10:35
rjekNo :)10:35
rjekBut if we go ahead with your suggestion that morph at least /try/ in the face of warnings at the top of its output ("I don't understand this VERSION, but I'm going to try anyway"), if the build fails it should probably say at the end of its output ("Try upgrading morph")10:36
rjekWhen things go wrong, people almost never look at the start of output :)10:36
pedroalvarezpaulsherwood: I guess you can rename pylru as something more generic  that implies that Morph needs more things to run. The same thing happened whith cliapp a while ago.10:37
* straycat thought it might be useful to have a mechanism that collects warnings and prints them at the end10:37
rjekstraycat: +110:37
ratmice__I think if the warn and continue approach prevails, a strict option to parse and fail when hitting an unknown version/fields should be considered10:38
* rjek is very proud of his command line parser that does that, having been fed up with so little software that does it :)10:38
rjekratmice__: Or perhaps the inverse; a --ignore-warnings10:38
rjekPeople could always pop the option in their morph.conf; although I'd still suggest the output includes "ignoring some warnings/errors; build may not be reproducible" or similar.10:39
paulsherwoodpedroalvarez: yup. i just wanted to start with specific examples, and then try to infer the generic factor(s)10:44
ratmice__not sure if there is the possibility that an ignored option could turn into a successful compilation but end up with runtime errors, and if this happens how far removed from the build logs could someone end up before someone notices11:01
straycatjust need to check something, is version 0 to be treated as some sort of special version where morph just applies a "default" behaviour, essentially ignoring versions, or is it a first version?12:18
*** zoli__ has joined #baserock12:19
*** zoli__ has quit IRC12:23
* straycat guesses that if lack of version file means 0 then 0 means "current"12:30
paulsherwoodno, lack of version file was to be assumed version 0. version 1 and onwards should only be achievable explicitly in version file12:50
paulsherwood(i believe)12:50
straycatOkay and version 0 is just a first version, so in the absence of a version file morph will assume you're using the first version of definitions, version 013:01
persiaSTating it like that gives me the impression there is something special about the exact format labelled "0", which worries me, as the current morph has some support for variations in the format.13:37
*** zoli__ has joined #baserock13:44
*** pacon has quit IRC13:49
*** pacon has joined #baserock13:51
paulsherwoodstraycat: yes14:00
paulsherwoodpersia: i don't understand what you mean14:00
persiaUnversioned definitions has had several formats.  I seem to recall prior statements that some parts of morph existed to support some features that are not present in the current HEAD of definitions master.14:01
persiaI don't think we'd do well to declare that the format that is formally expressed as version "0" is something in particular, and force morph to handle it explicitly.14:02
persiaRather, I think we ought assume unversioned is a mess, and try to deal with it for a bit, but focus on getting to version 1, which we can declare as a specific format, and mostly worry about versions from that point.14:02
persiaAnd once the transition away from unversioned is complete, I think the entire debate about lack of version becomes meaningless.14:03
persiaBecause very soon it won't make sense for morph to try to handle unversioned, as that will be too old.14:03
persiaPlus, as there are a large number of different formats that were unversioned, we can't clean up the legacy mess if we try to handle them.14:04
paulsherwoodi agree with most of what you're saying, in theory. how do you propose to 'deal with it for a bit' ?14:04
persiaFor that, it makes sense to treat unversioned with essentially similar parsing logic to what we have today, as you suggest.14:06
persiaBut it doesn't make sense to suggest that unversioned is specifically "Version 0", as if that is a defined format.14:06
*** pacon2 has joined #baserock14:06
persiaBecause it becomes tempting to clean up the "version 0" support code, which breaks earlier transitional unversioned definitions.14:06
persiaTo a certain degree, it seems to me that this might be a sensible time to branch morph, so we have a morph that moves to bugfix-only that can handle unversioned+0, and another branch that can handle 0+1 and complains for unversioned.14:08
persiaWhere "0" is specifically the format from the time that we declared version 0.14:08
persiaDIfferent branches can be different major versions, so morph 0.x.y is the one that handles transitionals, and morph 1.x.y handles 1, etc.14:08
*** pacon has quit IRC14:09
paulsherwoodi think the idea was to move on from 0 with tiny changes, as quickly as possible. no attempt to stay with or tidy up 014:09
persiaI specifically don't think we want to increment the major version every time we change definitions, but we probably want to have a new major version every time we drop support for several older formats, so we can tell folk to migrate by using version A, then version B, etc. to get from some old state to current morph.14:10
paulsherwoodat the meetup the broad idea seemed to be integers, not i.j.k14:11
persiaThat makes sense.  I just think it is wrong for any morph except for the transitional morph to assume unversioned == version 0.14:11
persiaI'm talking about morph versions.  Definitions is integers.14:11
persiaI think it would be a mistake to tie these.14:11
persiaSo, I imagine morph 0.x.y being a continuing series with small bugfixes for critical bugs that can handle unversioned+014:11
persiaAnd morph version 1.x.y being a continuing series with feature adds that can handle definitions 0,1,2,3,...14:12
* paulsherwood struggles with the complexity14:12
persiaAnd morph version 2.x.y being a continuing series with feature adds that is introduced when morph 1.x.y moves to bugfix mode that can handle definitions versions k, k+1, k+2, k+3, ...14:13
persiaIt's *less* complex.14:13
persiaA similar situation is deciding which versions of glibc and which versions of linux work together.14:13
* straycat can't see a compelling reason to branch morph like that14:14
persiaAs long as we assert that morph versioning is the same as definition versioning, we have to deal with all sorts of odd complexities about how we deal with compound trasitions.14:14
persiastraycat: The main reason is to allow us to drop support for all sorts of old stuff while still being able to apply bugfixes to a morph that can support the old stuff.14:14
persiaIt matters more when we get to definitions version k.14:15
* paulsherwood backs out of this... confused again14:15
persiaBut it also allows us to escape the unversioned == version 0 mess.14:15
straycatI'm also not sure where morph versioning came into this14:18
persiaPart of why I am excited about definitions versions is that it allows us to remove old compatibility code from morph.14:19
persiaAnother reason I'm excited is that it allows us to modify definitions in a way that morph can have explicit compatibility routines, rather than heuristics.14:20
persiaBut this causes a transition issue, because we need heuristics to handle unversioned definitions.14:20
persiaMost software projects that want to support some features, and also want to drop support for those features, do this by having releases.14:21
persiaAnd there is generally a separte branch for each release to allow for bugfixing to the release without forcing folk to accept all the new features (which may not work in their environment due to things that were deprecated).14:22
persiaI suggest we deal with morph having difficulty understanding how to get from unversioned to version 0 to version 1, etc. by following this model.14:22
straycatI don't follow, morph is already at version 1, the series I have in flight puts us up to 214:24
persiaI thought that was the definitions format.14:25
straycatyes sorry that's what I meant14:26
persiaSo I suggest that morph acquire a version number.14:27
persiaAnd then a new branch be made for stable changes to that version number14:27
persiaAnd then your branch land, giving morph a new stable version14:28
persiaAnd we expect the new version number to be able to handle 3, 4,5, etc. until we need to do a transition.14:28
persiaAnd if folk need to deal with unversioned, they go back and use the morph that was branched as stable prior to your series landing.14:28
persiaAnd then when we get to definitions version k, we do the same branch&increment to handle that transition.14:29
straycatThe alternative is to identify the heuristics we have and instead of removing them attach conditionals to them so that they only run for unversioned/version 0 definitions14:29
persiaYes, but I assert that unversioned != version 0.14:30
persiaWe've had too many different unversioned things, and too many leftover heuristics.14:30
straycatMy series makes unversioned == version 014:30
persiaYes, which is the part that started me complaining about it.14:30
straycatOkay, so the wiki lied to me14:31
persiaNot necessarily: I might disagree with the wiki.14:31
persiaI might even be completely wrong.14:31
persiaAnd I certainly didn't attend the versioning session at the meetup.14:31
straycatNeither did I :/14:31
straycat"The Definitions directory tree contains a file called VERSION. If this file is not present, tooling may assume definitions-V0 or fail gracefully"14:32
straycatSo, the new branch of morph could fail gracefully for unversioned14:33
persiaAnd suggest that folk who need unversioned support try the morph from the stable branch.14:34
persiaSimilarly, a bugfix patch could land on the stable branch that checked for definitions version >1, and suggested upgrading morph.14:34
straycatOkay, this seems like a pretty huge change, guess an RFC should go to the list before anything is done14:34
persiaYou can also do it the other way: I just think branching is easier when we want to support two different models.14:35
persiaThat being easier in code, and easier for people to think about "morph version X, supports definitions A, B, C; morph version Y supports definitions C, D, E; etc."14:36
persiaBut it is a vast change from the prior culture of asserting that "Basrock" has a version, which is implied to be related to multiple components.14:36
straycatI like the idea of a separate branch to support legacy definitions, allowing us to remove code we don't need anymore from current morph14:38
straycatwithout breaking things for older definitions14:38
persiaI eventually imagine us having a new morph branch every year or two, and feeding bugfixes to 2-3 of them at any time.14:39
persiaWhereas I hope definitions formats change more frequently, as we find better ways to express things.14:39
straycatDo you want to send this suggestion or shall I? I know which of us is the better writer :)14:44
*** pacon2 is now known as pacon14:44
persiaI still owe the list of whole bunch of things.  Waiting on me to submit an RFC may delay you unacceptably long.14:47
straycatShould the split be legacy: unversioned,0,1 current: >=2 ? Or legacy: unversioned current: >= 0   ?  We could go back to the checkout before versioning was added and branch from that in the latter case14:50
persiaI would do morph version 0.1 supporting unversioned, 0, 1; and morph version 1.0.1 supporting 1,2 to start.14:51
persiaSo that folk could transition against definitions version 1.14:51
straycatmakes sense14:53
straycati'll make the needed changes to this series then and submit the series as an rfc with this suggestion in the cover letter14:54
persiaSounds like an excellent plan.14:56
persiaIf the RFC is approved, it may make sense to have a separate series targeting the stable branch that detects defintions version > 1, and suggests using a different morph.14:57
persia(which series should never go to master)14:57
* straycat nods15:15
*** zoli__ has quit IRC16:00
*** zoli__ has joined #baserock16:01
*** zoli___ has joined #baserock16:02
*** zoli__ has quit IRC16:05
*** zoli___ has quit IRC16:07
*** CTtpollard has joined #baserock16:56
*** CTtpollard has quit IRC16:58
paulsherwoodanyone know the difference between 'config.status:994: creating Makefile' and 'config.status:1002: creating Makefile'21:49
paulsherwoodis it just a line number?21:49
robtaylorpaulsherwood: i'd expect so, what's the context?21:59
paulsherwoodi'm still hunting the ghost in my machine22:00
paulsherwoodthat's the only difference in config.log - which now makes me wonder if the path to my staging is too long, causing linewrappings22:01
robtaylorpaulsherwood: hmm, that shouldn't happen22:01
robtaylorpaulsherwood: have you got the two config.status's in qurestion22:02
robtaylorpaulsherwood: can you pastebin them?22:02
paulsherwood(i've used sed to replace the staging dirs)22:06
paulsherwoodthe diffs seem to be linewrappings http://paste.baserock.org22:07
robtaylorpaulsherwood: so seems to be something wrong with your STRIP_FOR_TARGET,RANLIB_FOR_TARGET,OBJDUMP_FOR_TARGET,NM_FOR_TARGET, LD_FOR_TARGET, AS_FOR_TARGET,AR_FOR_TARGET, LD,22:09
robtaylorsomething weird going on with TOPLEVEL_CONFIGURE_ARGUMENTS in both cases22:09
robtaylorpaulsherwood: so22:10
robtayloris yours22:10
robtayloris morphs22:10
robtaylorpaulsherwood: so some issue with your concatenation, i guess?22:10
paulsherwoodi'm not concatenating that22:10
paulsherwoodthat's configure, wrapping22:11
robtaylorpaulsherwood: no22:11
robtaylorpaulsherwood: yours has '\<newline>' in22:11
robtaylorbetween ../x86_64-bootstrap-linux-gnu and /bin/strip22:11
paulsherwoodoh, perhaps. but i have no clue what could be doing that22:12
robtaylor(p.s. i just ran the two config.status's though meld to find this quickly, useful tool, but side by side diff would also help)22:12
robtaylorpaulsherwood: probably how you set 'STRIP' in the environment22:13
robtaylorwhen you run configure22:13
* paulsherwood has never heard of STRIP, and neither has YBD, and apparently neither has morph ? :)22:14
robtaylorhmm, configure must be assembling it22:14
robtaylorpaulsherwood: could you pastebin the YBD config.log?22:15
paulsherwoodit's a struggle - it's really long. i can tell you that the only difference vs morhp's appears to be the config.status line numbers...22:19
paulsherwood-config.status:994: creating Makefile22:19
paulsherwood+config.status:1002: creating Makefile22:19
robtaylorooi, does baserock have 'haste' ?22:19
robtaylor(the haste command, that is)22:19
paulsherwoodi'm going to shorten my stage path... the differences i see in that status are mostly odd concatenations for longer lines22:20
robtaylorhum, someone should add haste-client to the devel environment22:20
robtaylorpaulsherwood: where do you pass in "/STAGING/tools/bin/../lib/gcc/x86_64-bootstrap-linux-gnu/4.9.2/../../../../x86_64-bootstrap-linux-gnu" ?22:21
paulsherwoodsame way morph does, afaik22:22
robtaylorwhich is?22:22
robtaylori think you have a newline on the end somehow22:22
robtaylorpaulsherwood: paste it using this command:  curl -F 'sprunge=<-'
robtayloroh sorry, i mean:22:23
robtaylorcat config.log | curl -F 'sprunge=<-' http://sprunge.us22:23
robtaylorand paste the address it returns22:24
paulsherwoodpasssing in is done by a combination of a 'clean environment' and containerisation settings, and the configure-commands22:26
robtaylorpaulsherwood: i *think* you might be missing a final "'" off the end of the configure line22:27
robtaylorconfigured by ../configure, generated by GNU Autoconf 2.64, with options \" '--build=x86_64-unknown-linux-gnu' '--host=x86_64-bootstrap-linux-gnu' '--target=x86_64-bootstrap-linux-gnu' '--prefix=/tools' '--libdir=/tools/lib' '--with-local-prefix=/tools' '--with-build-sysroot=/STAGING/STAGING/STAGING22:28
robtaylorCopyright (C) 2009 Free Software Foundation, Inc.22:28
robtaylorThis config.status script is free software; the Free Software Foundation22:28
robtaylorgives unlimited permission to copy, distribute and modify it."22:28
robtaylorvery odd line that22:28
* robtaylor is a bit bemused what could be happening there22:30
paulsherwoodthat's probably my sed22:30
robtaylorcan i have an un-sedded version? :)22:30
paulsherwoodand the status
paulsherwood(but this is with my shorter path, now)22:32
paulsherwoodand as a result the linenumber is now 994, same as morph's22:34
robtaylorahah, ok, it was the sed introducing the newline22:34
robtaylorno, really, it was22:34
robtaylorrun the sed again and see22:35
paulsherwoodthe 994 was reported in configure's result, before i sed anything22:35
paulsherwoodmy sed makes lines shorter, not longer. and i'm only running it *after* configure so i can compare morph result with ybd22:35
robtaylorline legth is not the issue here22:36
paulsherwoodyou may still be right :)22:36
paulsherwoodbut note there's no sed in the ybd run, only in my attempt to analyse it afterwards22:37
paulsherwood(sed -i 's|src/staging/[^/]*/[^/]*|STAGING|g' /src/staging/build-essential-20150318-190951/stage2-gcc/
robtaylorhmm, i guess configure may have some maximum line length for some old unix shell and that was causing your difference22:41
paulsherwoodi think that's more likely22:41
paulsherwoodif it is that, i'll be a happy camper! :)22:42
robtaylorHP-UX 11.00 and IRIX 6.5 Awk require that input files have a line length of at most 3070 bytes.22:43
robtayloroh, thats just awk22:44
* robtaylor bemused22:54
*** pacon has joined #baserock22:54
paulsherwoodim rebuilding ... j1 takes a while23:01
paulsherwoodwell the output now is different from what it was before, and different from morph... so that's progress of a sort ;)23:15

Generated by 2.15.3 by Marius Gedminas - find it at!