IRC logs for #baserock for Thursday, 2016-11-03

*** gtristan has joined #baserock07:11
*** franred has joined #baserock07:44
*** perryl has joined #baserock07:54
*** benbrown_ has joined #baserock08:06
*** ctbruce has joined #baserock08:31
*** ctbruce has quit IRC08:34
*** ctbruce has joined #baserock08:37
*** paulw has joined #baserock08:39
*** mdunford has quit IRC09:01
*** mdunford has joined #baserock09:15
*** ChrisPolin has joined #baserock09:18
*** rdale has joined #baserock09:26
*** toscalix has joined #baserock09:29
*** ChrisPolin has quit IRC09:51
*** ChrisPolinCT has joined #baserock09:53
*** paulsherwood has joined #baserock10:09
*** locallycompact has joined #baserock10:32
*** jmacs has joined #baserock10:32
locallycompactis there any way to make ybd use a local collection of site-packages so it is not fighting over dependency versions with anything else installed11:19
* SotK uses tox to do things like that11:21
pedroalvarezvirtualenv11:23
pedroalvarezbut tox makes it easier, yes11:23
locallycompactit this invisible to the user11:24
locallycompact*is11:24
leemingtox? i thought that was for testing11:24
locallycompactit looks a bit testy11:25
SotKyes, but it also conveniently makes running things in a virtualenv nice and easy11:25
locallycompactI want for someone to clone ybd and then for pip install -r requirements.txt be local to that app only11:27
locallycompactlike cargo and stack does it11:27
locallycompactcan it be that simple?11:27
pedroalvarezwith tox11:29
pedroalvarezto make it simple11:29
pedroalvarezlocallycompact: this would be using a plain virtualenv http://paste.baserock.org/pafavosahi11:34
pedroalvarezbut it didn't work11:34
paulsherwoodlocallycompact: have you seen actual problems with the current install_dependencies.sh approach?11:34
locallycompactpaulsherwood, ofc11:35
locallycompactybd breaks if any dependency changes upstream11:35
locallycompacthttps://gitlab.com/baserock/ybd/commit/0abf4cdf0df74ca8dcc4ed5647cc3def71a771eb11:36
locallycompactthe further problem is that it installs those everywhere11:36
paulsherwoodwhat does ofc stand for?11:37
pedroalvarezlocallycompact: you can also, with pip, install the packages in the home folder of the user, and then add that to PYTHONPATH11:37
rjekof course11:37
locallycompactpaulsherwood, of course11:37
paulsherwoodthanks11:37
* rjek has never run install_dependancies.sh11:38
pedroalvarezbtw, should be "fs" included in requirements?11:38
pedroalvarezybd didn't work without it11:38
locallycompactsee above branch11:39
pedroalvarez:)11:39
pedroalvarezthat patch makes a lot of sense11:40
paulsherwooddoes the requirements.txt mandate specifying version for each dependency? if not, can't you just relax that?11:40
thinkl33tIt doesnt11:41
thinkl33tyou can specify an exact version, a package name, or say greater than a certain version11:41
persiaIt permits expressions of ranges of versions or precise versions.  If these are overridden, often the software doesn't work (as best practice for requirements.txt is to only specify versions if/when things would otherwise break)11:41
locallycompactpaulsherwood, we don't know what will change in new versions of those dependencies11:41
locallycompactif you specify >= x11:42
locallycompactsomething will stop master spontaneously11:42
locallycompacteventually11:42
persialocallycompact: Depends on how firmly upstream maintains the ABI, but yes, usually.11:42
paulsherwoodlocallycompact: true, but does that matter, really?11:42
* rjek would rather the instructions recommended virtualenv or similar, rather than installing stuff globally11:43
persiapaulsherwood: More important is when requirements.txt specifies a set of versions that is not met by upstream master, in which case the developers are expressing that they believe the software will not work with current master.11:43
paulsherwoodrjek: patches welcome :)11:44
locallycompactpaulsherwood, yes that matters11:45
locallycompactthat master doesn't break suddenly11:45
paulsherwoodright... but our ci can check for that, then?11:45
rjekGrabbing unversioned things from pip can mean your thing can unpredictably stop working11:46
rjekalso see: Dockerfiles that curl | bash11:46
pedroalvarezheh11:46
pedroalvareztbh I've seen requirements without limitations in other projects11:47
pedroalvarezand that works fine11:47
pedroalvarezwhenever they see a problem, they either adjust their code, or limit the version until they can adjust it11:47
persiapaulsherwood: Maybe.  Because something builds doesn't mean that it can run.  Because something runs limited tests doesn't mean it won't break horridly.11:47
rjekWhen you need to fix your system in 2025, I think installing ybd as it currently is may be exciting.11:47
rjek(This is a general problem with *lots* of software, and I don't think anybody has found a pleasing solution yet)11:48
persiaThe most insidious sort of change is when a function changes what the parameters mean without changing the function signature, so that the same types are consumed, but used differently.  Many sorts of tests pass in this situation, but the software behaviour may be very different to the intent of the developers.11:48
paulsherwoodhence my preference that the tooling should continue to build old things, rather than saying (in 2025) 'you need to use the old tool from 2016'11:49
locallycompactthat's not what this is about11:50
paulsherwoodno, agreed11:50
persiaThat preference is the only sane way to use a 2025 system to build 2016 software.11:50
rjekBut it does mean that tooling needs to be supported for as long as the product they build is.11:50
paulsherwoodyup11:51
persia"supported" is a funny word in open source.  Most folk can cause software to be "supported" by asserting the identity of "developer".11:51
persia(assuming they have source)11:51
rjekOr you support an emulator that can run your 2016 software on 2025 hardware :)11:52
* locallycompact wonders if there's anything quicker we can put in premerge than gcc three times.11:53
rjekMy concern is that things bitrot; would we expect somebody who wants to support their product for 20 years to maintain CI infrastructure that is constnatly running for those 20 years?  What if they never need to make a fix until year 18?11:54
rjekI don't like it when important things are expensive :)11:54
jmacspersia: The process from having source to being able to support a piece of software can take years now, and I'd expect that to have grown exponentially by 202511:54
locallycompactI'd expect that to have shrunk drastically11:55
jmacsI fully expect the phrase "you have source, so you can fix it" to be untrue by then11:55
* rjek wonders how many people think about even ten years into the future when designing and writing software11:55
locallycompactI think that's untrue now11:55
persiajmacs: Why?11:55
rjekGetting stuff written ten years in the past running can be tricky and I don't see any reason to think that it's not going to get trickier.11:55
jmacspersia: Source code will be as complicated and difficult to understand as binaries are now.11:55
persiaI suspect the number of folk comfortable with software archeology will increase to offset that change.  I may be wrong.11:56
locallycompactime code is simpler as languages get more sophisticated11:56
paulsherwoodlocallycompact: i suggest that maybe what you need is an *additional* way to get ybd, rather than changing one of the current ones?11:56
locallycompactwhat?11:56
pedroalvarezyeh, it would be possible to have 2 different requirement files11:57
pedroalvarez(one way to do it)11:57
* paulsherwood wonders if the 'what?' is aimed at him...11:57
locallycompactyes11:57
franredlocallycompact,  ime code is simpler as languages get more sophisticated -- so, you rely more and more in other people code and controlling less and less what your code does in reality11:58
pedroalvarezin any case, I think that moving all of the pip dependencies to the requirements file is good11:58
paulsherwoodok11:58
SotKpedroalvarez: +111:58
paulsherwoodok, so currently there are two documented ways to install ybd: install_dependencies.sh, and vagrant up11:59
locallycompactfranred, no? I certainly have more control in more sophisticated languages11:59
paulsherwoodit may be better, for both locallycompact and rjek, to have a way that involves virtualenv and precise versions12:01
paulsherwoodbut other users may still benefit from the current two ways12:01
rjekI got bitten by pip the otherday removing a load of perfectly functional libraries and replacing them with newer shinier versions that broken everything else.12:01
locallycompactno user benefits from a dependency update that breaks execution surprisingly12:01
locallycompactif someone publishes a new sandboxlib that passes *it's* ci and breaks ybd then it will break in any production definitions that is tracking ybd master12:02
rjeklocallycompact: Or breaks execution subtly12:02
locallycompactor that12:02
pedroalvarezthat was clear, and the suggestion was to make something else to install ybd, but not change the current approach12:03
paulsherwoodi'm ok with a new installation that does virtualenv and whatever other improvements. i just see no reason to remove the current approach, or make ybd *require* virtualenv12:04
rjekpinning installed dependancy versions globally is also bad, as it may break other software on the system, of course.12:05
rjekSo such pins only really make sense in situations where tools like virtualenv are used12:06
paulsherwoodthat's my issue12:06
locallycompactpip install --help12:06
locallycompact  -U, --upgrade               Upgrade all specified packages to the newest available version. The handling of dependencies depends12:06
locallycompact                              on the upgrade-strategy used.12:06
locallycompactso one requirements.txt12:07
locallycompactshove -U where yo uwant12:07
pedroalvarezI don't understand what you mean12:07
locallycompactpip install -U -r requirements.txt12:07
locallycompactwill ignore the pins12:07
locallycompactIIUC12:07
pedroalvarezno12:08
pedroalvarezI mean, I believe that nope, it will get the latest possible, but without ignoring the pins12:08
SotKthat is my understanding of what will happen too12:08
locallycompact  --upgrade-strategy <upgrade_strategy>12:09
locallycompact                              Determines how dependency upgrading should be handled. "eager" - dependencies are upgraded regardless12:09
locallycompact                              of whether the currently installed version satisfies the requirements of the upgraded package(s).12:09
locallycompact                              "only-if-needed" -  are upgraded only when they do not satisfy the requirements of the upgraded12:09
locallycompact                              package(s).12:09
locallycompactyeah ok it's not doing that idno what hat's doing12:10
pedroalvareznope, that doesn't change my understanding12:10
paulsherwoodaiui the current install_dependencies.sh does not change any versions already installed, but installs things that are missing. it's been working ok so far, and it shouldn't *break* anything else on the system12:11
locallycompactit will break anything else on the system that requires a specific version of a dependency upgraded by ybd12:12
paulsherwoodybd doesn't upgrade anything12:12
pedroalvarezhm.. I'm not sure about that12:13
paulsherwoodpip runs, recommends upgrades, but doesn't do them, afaik12:13
pedroalvarezi think it might, but not sure12:13
paulsherwoodCollecting cherrypy Downloading CherryPy-8.1.2-py3-none-any.whl (456kB)12:15
paulsherwoodRequirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python3.5/site-packages (from cherrypy)12:15
paulsherwood(for example)12:15
persiaThe potential breakage comes when software package X has dependency K that must be <=4.9.3 and software package Y has dependency K that must be >=5.7.1412:18
persiaUnless one can find a couple such packages on pypi today, it's probably easier to consider it theoretically.12:18
pedroalvarezAlso, I don't know what happens if dependency X requires K >=5 and system has K < 512:19
pedroalvarezI'd say that pip will upgrade that package without asking, but not sure12:20
paulsherwoodyup. i'm not against having a virtualenv + pinned list approach, as a way to be sure to avoid such issues. however i'm not happy to drop the current simple approach, since it's been working fin for some usecases12:20
pedroalvarezand i think thats fair12:20
paulsherwoodtvm :)12:21
*** ctbruce has quit IRC13:06
*** ctbruce has joined #baserock13:27
*** ctbruce has quit IRC13:27
*** ctbruce has joined #baserock13:27
jjardonlocallycompact: https://gitlab.com/baserock/ybd/commits/jjardon/py3.413:30
*** toscalix has quit IRC14:28
*** gtristan has quit IRC14:33
*** toscalix has joined #baserock14:59
*** faybrocklebank has quit IRC16:45
*** ctbruce has quit IRC16:57
*** franred has quit IRC18:10
*** ctbruce has joined #baserock18:28
*** rdale has quit IRC18:36
*** locallycompact has quit IRC18:50
*** toscalix has quit IRC18:58
*** gtristan has joined #baserock19:14
*** locallycompact has joined #baserock19:43
*** gtristan has quit IRC22:02
*** locallycompact has quit IRC22:26
*** ctbruce has quit IRC23:23

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!