IRC logs for #baserock for Thursday, 2016-07-14

*** toscalix has joined #baserock00:25
*** toscalix has quit IRC00:52
*** toscalix has joined #baserock01:02
*** toscalix has quit IRC01:31
jjardonpaulsherwood: is it possible to reduce the output of ybd when downloading tarballs? It's making the logs so big that is reaching the gitlab limit; https://gitlab.com/baserock/definitions/builds/235653006:33
*** fay has joined #baserock06:51
*** fay is now known as Guest9826206:52
*** rdale has joined #baserock07:56
paulsherwoodjjardon: that's GitLab output, not ybd?08:14
paulsherwood(i think... i don't normally see anything like that when I run ybd)08:14
*** franred has joined #baserock08:18
pedroalvarezjjardon: we had a similar problem in morph08:25
pedroalvarezin morph with mason*08:26
pedroalvarezbut the solution we made might not work for this case08:27
paulsherwoodwhat was the solutiom?08:31
pedroalvarezhere: http://git.baserock.org/cgit/baserock/baserock/morph.git/commit/?id=0f00fbbed38757a2b0628341f172c69d9c4dd60a08:32
pedroalvarez(took me a bit to find it)08:32
*** CTtpollard has quit IRC08:33
pedroalvarezthat might work with  gitlab, and shouldn't be complicated to implement08:34
*** jonathanmaw has joined #baserock08:37
*** CTtpollard has joined #baserock08:42
* paulsherwood still can't retry builds, for some reason.08:44
paulsherwoodjjardon: pls can you retry?08:45
pedroalvarezpaulsherwood: i think you should redirect stderr to null, not stdout08:47
pedroalvarezbut yes, try that first just in case08:48
paulsherwoodpedroalvarez: the messages i'm trying to lose are stdout?08:48
pedroalvarezi think the messages are stderr08:48
paulsherwoodreally? how odd.. they aren't errors, obviosly08:48
pedroalvarezquick test: `wget https://github.com/ 2> file`08:49
paulsherwoodi can do both, in any case... it's in a try block, falling back to git clone08:49
pedroalvarezsome commands send to stderr information08:50
paulsherwoodactually, my logic leaves something to be desired08:50
paulsherwoodi've dropped both, in any case, if jjardon can retry08:52
pedroalvarezright08:52
paulsherwoodtvm08:54
jjardonpaulsherwood: thanks, it will rerun in https://gitlab.com/baserock/definitions/builds/236534209:02
*** Guest98262 is now known as faybrocklebank09:07
*** locallycompact has joined #baserock09:16
*** tiagogomes has quit IRC10:39
*** tiagogomes has joined #baserock10:49
*** edcragg has joined #baserock10:55
*** paulwaters_ has joined #baserock12:40
paulsherwoodjjardon: one step forward, one step back12:49
paulsherwoodleeming: https://gitlab.com/baserock/definitions/builds/2365342 is that a different example from the one you fixed?12:49
*** paulwaters_ has joined #baserock12:50
leemingpaulsherwood, looks like another example yes. Internally YBD uses byte arrays (ascii strings) instead of unicode strings13:17
*** anahuelamo has quit IRC13:57
*** gtristan has joined #baserock13:57
gtristanjjardon, are you here ??13:58
paulsherwoodleeming: that sounds like the kind of thing that should be configurable once-and-for-all, rather than futzing with specific code lines?14:01
richard_mawin this case it's because pyfilesystem is crap and assumes file paths are unicode objects when they are actually arbitrary byte strings14:04
paulsherwood:/14:05
richard_mawthe right thing to do is to have a clear understanding of what types your interfaces actually are, but the implicit conversion rules in python2 mean that you're not immediately told when you've gotten it wrong and it takes until you have unexpected unicode data or data that can't be converted to unicode14:06
richard_mawpython3 is more strict about the types… but it got file paths wrong so it's also doing the wrong thing14:06
leeming^ what he said14:07
paulsherwoodso for a mere mortal like me, i guess the only solution is futzing around, or OSFS14:08
*** edcragg has quit IRC14:08
richard_mawOSFS being the pyfilesystem thing?14:08
richard_mawif that is what was meant then futzing around is more correct14:08
paulsherwoodyes, so ok14:09
paulsherwoodleeming: pls would you consider doing the futzing for this too?14:09
richard_mawpyfilesystem suffers even more heavily from the assumption that file paths are unicode, and converts all your carefully preserved bytestrings into unicode… the *wrong* way14:10
* paulsherwood was in an odd discussion this week... where the relative impacts of my death versus Lennart's death were considered14:10
paulsherwoodnet result, even though very few people use ybd, i'm trying to encourage other people to fix it :)14:10
locallycompactwho14:10
paulsherwoodwho? anyone14:11
paulsherwoodanyone competent, that is ;)14:11
locallycompactwho is Lennart14:11
rjekMr Avahi, PulseAudio, and systemd14:12
locallycompactoh that gyu14:12
rjekA triptych of controversy14:12
leemingim not quite sure what pyfilesystem is, or the expected time to go through all of the YBD code to fix it :\ what priority is this?14:13
paulsherwoodleeming: never mind14:13
locallycompactpyfilesystem is a python filesystem abstraction that lets you walk with folds and simpler iterators14:14
*** gtristan has quit IRC14:14
locallycompactbut apparently causes all this unicode problems14:14
leemingfolds?14:14
locallycompacthttps://wiki.haskell.org/Fold14:14
richard_mawpython's equivalent is called reduce14:14
richard_mawand I'm sure locallycompact is about to chime in and tell me they are completely different14:15
locallycompactoh14:15
locallycompactoh oh14:15
* paulsherwood gets popcorn14:15
leemingI thought locallycompact was writing ybd in rust anyway?14:15
richard_mawrust is multi-paradigm I think14:15
locallycompactI do have a few thoughts14:16
locallycompacthttps://gitlab.com/baserock/morph-rs14:16
locallycompactearly days yet14:16
locallycompactbe interested to know if it builds everywhere14:16
* leeming started to go through some rust tutorials the other evening14:16
richard_mawthe defining your command-line parser as a yaml document and having a macro generate the code for it is terrifyingly clever14:17
* leeming gets confused and toddles off back to work on his git repo generator14:19
locallycompactI do like that thing14:19
*** edcragg has joined #baserock14:19
* richard_maw wonders if it can also generate config file readers14:20
locallycompactit's doing it14:21
* tiagogomes wonders if rust will actually succeed or just become the new "D" language14:21
locallycompactwell serde is doing that14:21
locallycompact#[derive(Deserialize)] makes serde do magic against a type all of whose members derive Deserialize14:23
locallycompacttiagogomes, the bar for success imo is replacing all the c code in linux14:29
locallycompactat a minimum14:29
*** fay has joined #baserock14:40
*** fay is now known as Guest2838314:40
richard_mawha! I'd settle for usurping the throne of node.js, ruby or Go14:41
*** faybrocklebank has quit IRC14:44
*** Guest28383 is now known as faybrocklebank14:50
jjardonreading https://www.azavea.com/blog/2014/03/24/solving-unicode-problems-in-python-2-7/ , seems a possible solution is to only use Unicode objects internally15:12
jjardonpaulsherwood: about the python unicode problem ^15:13
richard_mawjjardon: how does that handle file paths which aren't unicode strings?15:13
jjardonyou can still treat them as unicode internally?15:14
rjeknot losslessly IIRCC15:14
richard_mawnot if the strings contain bytes that aren't valid unicode code points15:14
richard_mawthere's os.fsencode and os.fsdecode, but they have some strange notion that the user's language and localisation settings are relevant15:19
locallycompactat what point does pyfilesystem start to misbehave, because there is this maybe http://docs.pyfilesystem.org/en/latest/wrapfs/base.html15:20
locallycompactoh but wraps existing15:20
locallycompactmaybe pointles15:20
richard_mawI think fs{encode,decode} fall back to using the "surrogateescape" encoding so it hex encodes characters it can't represent, but that has the terrifying implication that if you're just joining arbitrary unicode strings together you can end up picking up user input that encodes a NUL in the decoded string, and truncating the string when the python filesystem API translates it back into bytes for calling the syscall15:24
richard_mawso while python3 will catch you if you combine byte strings and unicode strings, it won't help you if you combine unicode strings with file paths15:24
richard_mawsince they decided that file paths can be unicode strings15:25
locallycompactI'm sure they'll fix it in python 415:26
richard_mawha!15:26
richard_mawthere's not going to be a python415:26
* paulsherwood doesn't understand how they messed up going from 2 to 3, but they clearly did15:27
richard_mawthere may be a successor language that's pythony, but it's not going to be directly descended from python15:27
richard_mawpaulsherwood: it's a topic of hot debate in the python community, lwn.net has quite a few good articles about it, but the general problem is the difficulty porting *because* it was a breaking change15:27
richard_mawpaulsherwood: if it were ABI compatible, so you could use the same python interpreter and safely run both python3 and python2 code, so you could migrate your applications piecemeal then it would have been ok15:28
paulsherwoodack15:28
richard_mawpaulsherwood: but the biggest problem was that libraries weren't ported quickly enough15:28
richard_mawyou can't justify a rewrite of your whole codebase15:28
locallycompactI can justify that all day long.15:29
richard_mawdo you have thousands of users who depend on you to provide bug fixes for every version of your codebase?15:33
locallycompactNo but I can complain just as loudly15:41
*** franred has quit IRC16:07
*** CTtpollard has quit IRC16:17
*** jonathanmaw has quit IRC16:42
*** edcragg has quit IRC17:17
*** edcragg has joined #baserock17:24
*** locallycompact has quit IRC17:44
pedroalvarezIm curious about the not booting problem you hit the other day. Is that fixed? I'll need to do some baserocking next week and I'm hoping it goes smoothly19:58
pedroalvarezjjardon: nevermind, genivi-baseline (with initramfs) seems to boot fine22:55
*** rdale has quit IRC23:17

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!