| *** gtristan has quit IRC | 03:50 | |
| *** trn has quit IRC | 04:04 | |
| *** gtristan has joined #baserock | 04:53 | |
| *** trn has joined #baserock | 06:05 | |
| *** paulwaters_ has joined #baserock | 07:28 | |
| *** rdale has joined #baserock | 07:43 | |
| *** ctbruce has joined #baserock | 07:59 | |
| *** toscalix has joined #baserock | 08:04 | |
| *** paulwaters_ has joined #baserock | 09:04 | |
| *** locallycompact has joined #baserock | 09:28 | |
| *** gtristan has quit IRC | 09:32 | |
| *** edcragg has quit IRC | 10:07 | |
| *** edcragg has joined #baserock | 10:09 | |
| *** gtristan has joined #baserock | 10:22 | |
| locallycompact | what is python | 11:01 |
|---|---|---|
| locallycompact | tring to tell me | 11:01 |
| locallycompact | here | 11:01 |
| locallycompact | def load_yaml(filename): | 11:02 |
| locallycompact | with open(filename) as f: | 11:02 |
| locallycompact | text = f.read() | 11:02 |
| locallycompact | contents = yaml.safe_load(text) | 11:02 |
| locallycompact | return contents | 11:02 |
| locallycompact | print(load_yaml(sys.argv[0])) | 11:02 |
| locallycompact | yaml.scanner.ScannerError: mapping values are not allowed here | 11:02 |
| locallycompact | in "<string>", line 5, column 24: | 11:02 |
| locallycompact | def load_yaml(filename): | 11:02 |
| pedroalvarez | Try putting spaces after the colons in your yaml | 11:05 |
| locallycompact | no that's not it | 11:05 |
| pedroalvarez | grr | 11:05 |
| locallycompact | argv[0] -> argv[1] | 11:05 |
| pedroalvarez | hah | 11:05 |
| locallycompact | fs | 11:05 |
| pedroalvarez | that too | 11:05 |
| jmacs | It's still a weird error | 11:06 |
| jmacs | Oh, I get it now | 11:06 |
| pedroalvarez | it's obvious once you get it | 11:06 |
| pedroalvarez | but yes, very confusing | 11:06 |
| pedroalvarez | good catch locallycompact :) | 11:07 |
| richard_maw | o_O it was trying to parse your script as yaml‽ | 11:48 |
| locallycompact | yarp | 11:49 |
| leeming | haha | 11:49 |
| persia | The extra confusing part is that, I presume because of all the colons in the python, it reached line 5 before hitting issues. | 11:59 |
| locallycompact | how do you make python look for files where the program is not where it's run | 12:26 |
| jmacs | You want the run directory or the directory the script is in? | 12:27 |
| locallycompact | relative to script | 12:27 |
| richard_maw | that's normally argv[0] I think | 12:27 |
| jmacs | os.path.realpath(__file__) | 12:27 |
| locallycompact | is that the file file or the main executable | 12:28 |
| locallycompact | because they might be in different dirs | 12:28 |
| jmacs | The file that line occurs in | 12:28 |
| locallycompact | how do I get the main executable dir | 12:28 |
| richard_maw | use os.path.dirname(__file__) in the main execuable, unless you mean the python interpreter | 12:29 |
| locallycompact | well ybd has a main exe, and the includes files in subfolder ybd/ I want to always get the folder the main exe is in | 12:30 |
| jmacs | In that case I think you want os.path.realpath(sys.argv[0]) | 12:32 |
| locallycompact | thanks! | 12:32 |
| locallycompact | paulsherwood, you about? | 12:47 |
| *** gtristan has quit IRC | 13:32 | |
| *** gtristan has joined #baserock | 14:51 | |
| paulsherwood | locallycompact: i am now | 15:23 |
| *** CTtpollard has quit IRC | 15:59 | |
| *** tiagogomes has quit IRC | 16:14 | |
| locallycompact | paulsherwood, I'm going to bring the spec into the build tool to get rid of that maddening parsing logic. Would you prefer it hard copied or submoduled | 16:24 |
| locallycompact | this gives all the error tracing you could want https://gitlab.com/baserock/ybd/commits/lc/009 | 16:25 |
| paulsherwood | i don't like submodules in general. what do you mean by hardcoded? | 16:32 |
| paulsherwood | s/submodules/git submodules/ | 16:32 |
| paulsherwood | maybe you didn't mean that | 16:33 |
| locallycompact | hard copied, as in those commits | 16:34 |
| locallycompact | the list of valid fields things is confusing, jsonschema is very robust and gives decent errors, and morphs.py is just unintelligable | 16:35 |
| paulsherwood | agreed for all three points | 16:36 |
| locallycompact | so I can polish the schema for version 9 and we can copy it in and change the code, or add a submodule and update it as the spec channges but then you have to clone recursive | 16:37 |
| paulsherwood | or have a separate spec-parser python library? | 16:38 |
| locallycompact | mm, sure | 16:38 |
| paulsherwood | i'm not sure which is best tbh. i wanted to bring in the spec, but ssam argued strongly for splitting it out | 16:39 |
| locallycompact | really? what did he say? | 16:39 |
| locallycompact | all that's happened is we've makeshifted a poorer version of jsonschema.validate(), reusing the spec for that purpose is an implementation detail of ybd imo | 16:40 |
| * paulsherwood can't remember the details, but we ended up with a separate repo for it | 16:40 | |
| locallycompact | that was to split it out from definitions | 16:40 |
| locallycompact | it was in definitions and now it isn't, that makes sense | 16:40 |
| locallycompact | but the build tool should still leverage it | 16:41 |
| paulsherwood | by duplicating the code from the spec repo? | 16:42 |
| locallycompact | I'm fine with that | 16:43 |
| paulsherwood | in my dreams the spec is a self-consistent library that can validate any given version of definitions, and present it for direct consumption by ybd or other tools | 16:43 |
| locallycompact | I like that too | 16:43 |
| locallycompact | I'll see about trying to publish a library | 16:44 |
| paulsherwood | if you copy the code into ybd, we have the usual possibility of getting out-of-step | 16:44 |
| locallycompact | well, no, ybd still has juresdiction over what version of definitions it is capable of parsing at what point in history | 16:44 |
| locallycompact | it would be no more out of step than it is now | 16:44 |
| locallycompact | we have to update ybd to be able to parse V9, the schema can already do it | 16:45 |
| paulsherwood | right. but in part morphs.py is unintelligible because it handles all of the previous versions too | 16:46 |
| locallycompact | ok, well we can still aim for that with a library but morphs.py is unsustainable | 16:47 |
| locallycompact | I doubt it could handle this and more changes, with a full library we could technically duck type based on the definitions version | 16:47 |
| locallycompact | so grab the version, you get this interface of parsing implementations in return | 16:48 |
| paulsherwood | iiuc it would be desirable for the spec-library to read datasets at any/all historic schema versions, write converted data to the current schema version, and present an in-memory representation for consumption by other tools | 16:48 |
| locallycompact | ok | 16:49 |
| persia | I'd argue that it should write converted data to any/all historic schema | 16:49 |
| paulsherwood | why? | 16:49 |
| persia | Ideally with some mechanism where the read/write aspects are bound, to make it easy to add new ones. | 16:49 |
| locallycompact | that's not possible, it's not necessarily bijective between adjacent versions | 16:50 |
| * paulsherwood doesn't see the need. the only direction is forward | 16:50 | |
| persia | paulsherwood: A) Someone might have a hacked version of a build tool that needs version 6, and so wants to write out an old one. B) it means that the entire output module doesn't have to be rewritten every time a new version is created, just a writer for that version from some consistent internal store. | 16:50 |
| persia | Oh, and C) it validates that the internal store can be usefully mapped to version X, so that one can be more confident about consuming version X, because round-trip is a no-op. | 16:51 |
| paulsherwood | persia: in the scenario above, build tools use (a given version of) the library, and work internally with the memory representation it gives them. | 16:51 |
| persia | (if round-trip isn't a no-op, then the tool would be introducing something (probably buggy) | 16:51 |
| persia | Yes. | 16:52 |
| persia | But that hasn't always been true. | 16:52 |
| persia | So, if anyone is using a build tool in production today, and they have modified it in any way, this would be an epoch if there isn't bidirectional compatibility. | 16:53 |
| paulsherwood | only if they continue using that build tool? | 16:54 |
| persia | Yes. | 16:54 |
| persia | But there aren't any other tools that meet their custom need, or they wouldn't have modified it. | 16:54 |
| persia | So if they are changing tooling and data format, what makes this new state you describe interesting? | 16:54 |
| persia | Unless they beleive in following mainline, they are much more likely to fork and live with it until it becomes too painful, then evaluate available tooling/systems at that future point in time. | 16:55 |
| locallycompact | no hope for them | 16:55 |
| paulsherwood | i'm interested in ensuring that new tools can build old definitions. not that old tools can build new definitions | 16:56 |
| locallycompact | I don't know how well that's going to work off the top of my head because I don't remember definitions versions older than 7, but I can start with a library that works on 7+ and work backwards | 16:56 |
| persia | Given that there needs to be a mapping between a given definitions format and the internal representation *anyway*, and given that there is an expectation of there being an infinite number of definitions versions, I don't understand why it is especially hard to preserve the writer for each version vs. rewriting one "perfect" writer each time, and making all the old stuff immediatly invalid. | 16:56 |
| locallycompact | it might not be much work, 7 and 8 parse identically fr ex | 16:57 |
| *** toscalix has quit IRC | 16:57 | |
| paulsherwood | locallycompact: the library could call morphs.py for everything up to 8, and do your new magic for everything newer. in that way, morphs.py would be (kind of) sustainable | 16:57 |
| persia | SO, yeah, if it's especially difficult, I could be convinced it isn't necessary. If it's not done just because people want to chase the future, I imagine that to be a poor choice for a broad ecosystem. | 16:57 |
| paulsherwood | s/8/8 or some earlier cutoff/ | 16:58 |
| locallycompact | yeah alright | 16:58 |
| paulsherwood | persia: why would preserving the (infinite number of) writers in the current version of the library be better than saying just use an old version of the library? | 17:07 |
| persia | Because I don't expect the build tool to work with an old version of the library, so I end up hunting for an old version of the build tool, and have to research the bugs it has, etc. Given this mess, easier to never upgrade. | 17:08 |
| persia | Since I think people should upgrade all the time, I think we should make it really easy to do so. | 17:08 |
| locallycompact | It still reads stuff | 17:09 |
| persia | and given the need to preserve the (infinite number of) readers, I don't expect it is much more code. | 17:10 |
| persia | I may be wrong, and if it is lots more code, then I am happy to give up the argument. | 17:11 |
| locallycompact | It's not possible, there is no functional mapping from 9 -> 8 | 17:11 |
| persia | locallycompact: Why not, assuming there is no variance in build dependencies for a given strata over the set of systems? | 17:12 |
| persia | s/strata/stratum/ | 17:12 |
| locallycompact | that's why it's not functional | 17:13 |
| persia | It's functional: the function just sometimes returns "undefined" | 17:14 |
| persia | (depending on the input data) | 17:14 |
| locallycompact | that's not mathmo functional | 17:15 |
| * persia fails to parse "mathmo" | 17:15 | |
| locallycompact | mathmonk | 17:16 |
| persia | Still no, and my search engines don't help. | 17:17 |
| locallycompact | high priest mathmage | 17:17 |
| locallycompact | I'm going to bed | 17:17 |
| persia | https://en.wikipedia.org/wiki/Functional_(mathematics) ? | 17:18 |
| locallycompact | that too but I was more | 17:18 |
| locallycompact | https://en.wikipedia.org/wiki/Functional_dependency | 17:19 |
| * locallycompact sleep | 17:20 | |
| persia | Sleep well. When you wake, consider that the page you linked doesn't require that the target set contain unique values, as long as there is a value for each member of the set. For example, VIN->engine_capacity works fine, even for cases where there is no engine, so engine_capacity is undefined. | 17:21 |
| *** locallycompact has quit IRC | 17:25 | |
| *** rdale has quit IRC | 18:27 | |
| *** ctbruce has quit IRC | 20:24 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!