IRC logs for #baserock for Tuesday, 2016-07-05

*** gtristan has quit IRC00:01
*** gtristan has joined #baserock00:01
*** gtristan has quit IRC03:18
*** gtristan has joined #baserock04:12
*** gtristan has quit IRC06:47
*** gtristan has joined #baserock07:15
*** rdale has joined #baserock07:20
*** ctbruce has joined #baserock07:28
*** paulwaters_ has joined #baserock07:54
*** toscalix has joined #baserock08:03
*** franred_ has joined #baserock08:11
jjardonpaulsherwood: hi, can you reopen https://github.com/devcurmudgeon/ybd/issues/218 ? Seems I can not do it myself08:24
paulsherwooddone. however i have no clue how to debug it, let alone fix it, i'm afraid08:25
*** jonathanmaw has joined #baserock08:32
paulsherwoodleeming: any chance you could take a look, please?08:43
leemingpaulsherwood, sure I can switch back over to looking at ybd09:27
paulsherwoodleeming: i was hoping you'd solve it in minutes :)09:30
*** locallycompact has joined #baserock09:30
* leeming tries to figure out the sarcasm to expectation ratio09:31
*** locallycompact has quit IRC09:32
leeminganyway, at a glance sounds like it _might_ be a funny python string thing (unicode strings)09:36
*** locallycompact has joined #baserock09:36
paulsherwoodit was't sarcasm, it was desperate optimism.09:38
paulsherwoodyes, it's a funny python string thing. unfortunately i still don't understand that dark corner of python09:39
paulsherwoodleeming: if you're 'in the zone' on what you're doing, i can ask someone else09:41
leemingwell I was going to add some more advanced things to spec2def... I got it in a usable state (by others) yesterday09:43
franred_paulsherwood, I think you can escape, decode and filter unicode strings. I think morph has deal with this in the past09:43
leemingbut i havent started anything considerable yet, so i can09:43
locallycompactpython3 makes this a non-issue, I'm led to believe09:44
paulsherwoodpython2 is lowest-common-denominator, so i believe(d) it's best to stick with that09:45
leemingyes, it is a 2.7 issue09:45
leemingpart of the friction between python 2 and 3 camps :)09:45
locallycompactlowest-common-denominator is why we can't have nice things09:45
richard_mawpython3 makes the type conversions more explicit so they fail earlier at a point where it's easier to fix, unfortunately it has incorrect opinions on what should be unicode and what should be bytes09:46
richard_mawfile paths for example09:47
richard_mawin Linux file paths are bytes with the only restriction that you can't embed NUL bytes and / is the path component separator, unicode is much more strict09:48
richard_mawso you can make some python3 programs fail by having non-unicode-compatible file names09:48
richard_mawthere's os.fs{en,de}code for mangling bytes into something unicode compatible, which most of the filesystem APIs de-mangle under the hood, but I'm told fsencode and fsdecode can mangle strings so badly that they embed NULs09:50
richard_mawthere's a pathlib module in python3's standard library. it is wholly incapable of handling non-unicode file paths, so should be ejected from orbit as not fit for purpose09:51
richard_mawRust gets it right by having file paths be a different type entirely so the appropriate conversions *can* be handled transparently09:51
rjekIt sounds like Python is designed and written by Python language lawyers, not language lawyers of the language they've implemented it in.09:52
*** ctbruce has quit IRC09:52
richard_mawdon't get me started on time representation09:52
richard_mawrjek: nor developers on the platform it must target.09:53
rjekNod09:53
* locallycompact prods richard_maw about time representation09:55
richard_mawlocallycompact: time is represented as floating point. A double lacks sufficient precision to represent linux time stamps. The reason why this wasn't fixed was that "nobody has clocks of that precision". This is no longer true, and regardless it means that you can't accurately copy a file's metadata from python without messing about with ffi.10:00
locallycompactoh lord10:01
*** ctbruce has joined #baserock10:10
rjekFire.10:11
leemingjjardon, can you give me instructions of replicating your error please? I think I know what is up, but just want to go through some debug testing10:56
jjardonleeming: simply build a system10:57
jjardonall of them are failing: https://gitlab.com/baserock/definitions/pipelines/364299510:57
leemingLatest build -> https://gitlab.com/baserock/definitions/builds/216806311:08
leeming In file included from ext/_yaml.c:343:0:11:08
leeming    ext/_yaml.h:6:0: warning: "PyUnicode_FromString" redefined11:08
leemingyaml seems not to be building pre system build11:08
leemingoops sorry that was a warning11:09
leemingthere are a few errors tho11:09
leemingBut since they are for PyUnicode*, seems to raise questions11:10
* tiagogomes learns his number of commits to definitions11:17
leeminghow do I tell gitlab to use a different branch ? Assuming in .gitlab-ci.yml, but only examples of cloning, not using alternative branches11:22
leemingor do I just stick in a 'git branch new/branch' line?11:23
leemingunable to push to test.. just looking into the alternatives atm11:46
*** gtristan has quit IRC12:48
jjardonleeming: you can have a personal fork of the repo, then you are free to do any change you want13:38
leemingalready onto it :)13:42
leemingI was trying to push a dev branch originally13:42
jjardonleeming: great! :)13:51
*** cosm has joined #baserock14:31
*** ctbruce has quit IRC15:22
*** gtristan has joined #baserock16:08
leemingjjardon, is this expected? https://gitlab.com/leeming/definitions/builds/218850416:09
leemingcontext, fixed (i think, just checking) the issue from eariler16:10
jjardonleeming: cool! let me take a look16:10
richard_maw"tar: ./sbin: Directory renamed before its status could be extracted16:10
richard_mawO_O16:10
leemingah, looking at the main builds view, they are all deploys16:11
leeminghttps://gitlab.com/leeming/definitions/builds16:11
leemingi dont think i know the difference between build/deploy16:11
jjardonoh, so deploying is failing16:12
jjardonleeming: maybe you found another bug :)16:12
richard_mawThe "Directory renamed before its status could be extracted" sounds like a terrifying race condition to me.16:12
leemingissue was to do with unicode magic... a cheap hack fixed it, but I guess the whole of ybd would have this issue :P16:12
jjardonyeah, its definetely a problem in deployment: https://gitlab.com/leeming/definitions/pipelines/364990316:12
jjardonleeming: how difficult you think would be to port it to python3?16:13
richard_maw"16-07-05 00:16:25 [454/508/508] [initramfs-x86_64] ERROR: No install-commands, manual build-system" looks like an internal logic error.16:14
leemingwell, it went thru my mind when i was reading up on python unicode/encoding stuff16:14
richard_maws/thru/through/16:14
* richard_maw ducks16:14
leemingrichard_maw, yeah I saw that. thus 'expected' question :D16:14
* leeming quacks16:14
leemingjjardon, I have a set of possible suggestions... but im not sure if I want to set myself up for being responsible16:15
leemingim going to continue to add verbose logging to ydb for personal dev/understanding. I will see what I can come up with16:16
leeminganywho... paulsherwood want me to submit a merge request for issue 218 now, or once I've cleaned up the code a bit more16:27
leeminge.g. proper fix vs quickpatch16:28
paulsherwoodleeming: yes please16:29
leemingto a or b?16:29
jjardonleeming: quick patch now, keep working in a proper fix? ci is broken atm16:30
paulsherwoodleeming: your call16:30
leemingok, think i will throw in the quick patch now and raise an issue as a note for further fix/reference16:31
paulsherwoodok16:31
paulsherwoodleeming: pep816:32
* leeming googles pep816:32
paulsherwoodleeming: never mind16:33
leemingpython 'standard' style guide... your code uses mixed camel and snake case you know ;)16:35
* leeming so does mine16:35
* richard_maw understood the standard python style was Camel for classes and snake for variables and members16:37
* leeming nods at richard_maw 16:37
* rjek learns that one of the big reasons PEP8 advises against bare except: apart from making the code undebuggable is that it also stops Ctrl-C working16:37
leemingmy python is a mix of my usual camel and then remembering python likes snake case16:38
richard_mawaye, it's a PITA when someone's bare-except causes safe killing it to fail16:38
richard_mawleeming: too much Java programming?16:38
leemingrjek, wait, it does? I thought python treat exception as the base, instead of catching runtime16:38
leemingor am i mixing it with java?16:38
rjekleeming: The difference between except Exception: and except BaseException:16:39
leemingyes, I am a java person16:39
rjekYou'll get better16:39
leemingrjek, yes i was going to put BaseException, but was lazy as i'd already typed exception16:39
leemingnow im confused to which exception model is in my head16:39
paulsherwood17:14 <+richard_maw> "16-07-05 00:16:25 [454/508/508] [initramfs-x86_64] ERROR: No install-commands, manual build-system" looks like an internal logic error.16:40
leeming"Throwable"16:40
paulsherwoodrichard_maw: does it make sense for a chunk with a manual build system to have no install-commands? how could that be useful?16:40
leemingah that must be java, because python aptly calls it 'raise'16:40
richard_mawpaulsherwood: initramfs-x86_64 is not a chunk16:41
richard_mawor at least wasn't when I wrote it16:41
paulsherwoodrichard_maw: oh... but if dn.get('kind', 'chunk') == 'chunk': ?16:41
richard_mawI have no idea what's going on there, in the definitions I see: `kind: system`.16:43
paulsherwoodok, must be me, then :)16:43
* richard_maw would guess that YBD is not loading the definition properly,16:45
richard_mawsince to get chunk out there, it would need to not have a kind present16:45
richard_mawthat, or it's decided that you can't look up a unicode string with a byte string16:46
richard_mawpython3 will treat them differently, but in python3 you'll be asking for the unicode string, and the yaml parser will have put a unicode string in there, so that's not the problem16:48
leemingoh16:49
leemingmaybe i need to do a proper fix then :d16:49
leemingthe patch I did was inside a function that does some file opening/saving. Unsure what it does though (no comments) - _process_list(srcdir, destdir, filelist, actionfunc)16:51
leemingI did some unicode hackery16:52
leemingthis is python 2.716:53
leemingif someone can give me a pointer to where this might slip up, i can look into it16:54
*** jonathanmaw has quit IRC16:54
leemingFor the specific definitions used, it was some mozilia(?) certificate that had accented characters, therefore throwing the  original error16:55
paulsherwoodleeming: just create a pull request, please... i'll investigate the 'No install-commands' thing16:56
paulsherwood"Directory renamed before its status could be extracted"16:56
paulsherwoodi think this is something to do with gitlab runners... see https://github.com/meteorhacks/meteord/issues/6416:57
leemingok paulsherwood 2 secs, let me figure out the github interface16:57
paulsherwoodi don't see how there can be a race condition... ybd is single threaded at that point16:57
leemingdone17:04
leemingi guess that is my first official contribution to open source? \o/17:04
paulsherwoodw00t! :)17:07
jjardonleeming: nice, thanks for the patch!17:09
paulsherwoodleeming: is there some reason for this being 4 patches?17:09
leemingis incompetence a valid answer?17:09
leeming4 patches? or 4 commits?17:09
paulsherwood4 commits17:09
leemingah, you prefer me to squish them?17:10
paulsherwoodnot if there is a reason for them to be separate. but 'Added in utf-8 encode. no seriously did it this time' makes me think there isn't17:10
leemingso that is a "yes please" right?17:10
paulsherwoodyup :)17:11
paulsherwoodanother reason would be '4 commits looks like more work than 1' :)17:11
leemingi will stick to my original honest answer17:12
paulsherwood:)17:12
leemingi've already shown my cards to the table17:12
paulsherwoodand to the whole internet. forever17:12
* leeming wonders if github has magic to do this rebase for him17:12
leemingprobs now17:13
leemingnot*17:13
paulsherwoodleeming: could you re-run your ci that failed please?17:14
paulsherwoodoh actually, never mind17:14
paulsherwoodleeming: i have a squash already if you like... http://paste.baserock.org/zosodububi17:26
leemingah sorry, i had rebased it my side, then got distracted by coffee and escalated17:29
paulsherwoodif you push your rebase, i'm happy to go with it17:30
leeming^sorry, it is already there17:30
leemingi was going to ping you17:31
leemingbut got distracted, sorryu17:31
paulsherwoodmerged17:31
leemingfollowing up on something persia said... that was actually my 2nd contribution... forgot about adding some lorries to baserock :D17:45
*** locallycompact has quit IRC18:15
*** toscalix has quit IRC18:19
*** janderJLR has joined #baserock18:46
*** janderJLR has quit IRC18:47
*** janderJLR has joined #baserock18:48
*** janderJLR has left #baserock18:48
*** locallycompact has joined #baserock20:11
*** rdale has quit IRC20:55
*** locallycompact has quit IRC21:17

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!