| *** gtristan has joined #baserock | 07:11 | |
| *** franred has joined #baserock | 07:44 | |
| *** perryl has joined #baserock | 07:54 | |
| *** benbrown_ has joined #baserock | 08:06 | |
| *** ctbruce has joined #baserock | 08:31 | |
| *** ctbruce has quit IRC | 08:34 | |
| *** ctbruce has joined #baserock | 08:37 | |
| *** paulw has joined #baserock | 08:39 | |
| *** mdunford has quit IRC | 09:01 | |
| *** mdunford has joined #baserock | 09:15 | |
| *** ChrisPolin has joined #baserock | 09:18 | |
| *** rdale has joined #baserock | 09:26 | |
| *** toscalix has joined #baserock | 09:29 | |
| *** ChrisPolin has quit IRC | 09:51 | |
| *** ChrisPolinCT has joined #baserock | 09:53 | |
| *** paulsherwood has joined #baserock | 10:09 | |
| *** locallycompact has joined #baserock | 10:32 | |
| *** jmacs has joined #baserock | 10:32 | |
| locallycompact | is there any way to make ybd use a local collection of site-packages so it is not fighting over dependency versions with anything else installed | 11:19 |
|---|---|---|
| * SotK uses tox to do things like that | 11:21 | |
| pedroalvarez | virtualenv | 11:23 |
| pedroalvarez | but tox makes it easier, yes | 11:23 |
| locallycompact | it this invisible to the user | 11:24 |
| locallycompact | *is | 11:24 |
| leeming | tox? i thought that was for testing | 11:24 |
| locallycompact | it looks a bit testy | 11:25 |
| SotK | yes, but it also conveniently makes running things in a virtualenv nice and easy | 11:25 |
| locallycompact | I want for someone to clone ybd and then for pip install -r requirements.txt be local to that app only | 11:27 |
| locallycompact | like cargo and stack does it | 11:27 |
| locallycompact | can it be that simple? | 11:27 |
| pedroalvarez | with tox | 11:29 |
| pedroalvarez | to make it simple | 11:29 |
| pedroalvarez | locallycompact: this would be using a plain virtualenv http://paste.baserock.org/pafavosahi | 11:34 |
| pedroalvarez | but it didn't work | 11:34 |
| paulsherwood | locallycompact: have you seen actual problems with the current install_dependencies.sh approach? | 11:34 |
| locallycompact | paulsherwood, ofc | 11:35 |
| locallycompact | ybd breaks if any dependency changes upstream | 11:35 |
| locallycompact | https://gitlab.com/baserock/ybd/commit/0abf4cdf0df74ca8dcc4ed5647cc3def71a771eb | 11:36 |
| locallycompact | the further problem is that it installs those everywhere | 11:36 |
| paulsherwood | what does ofc stand for? | 11:37 |
| pedroalvarez | locallycompact: you can also, with pip, install the packages in the home folder of the user, and then add that to PYTHONPATH | 11:37 |
| rjek | of course | 11:37 |
| locallycompact | paulsherwood, of course | 11:37 |
| paulsherwood | thanks | 11:37 |
| * rjek has never run install_dependancies.sh | 11:38 | |
| pedroalvarez | btw, should be "fs" included in requirements? | 11:38 |
| pedroalvarez | ybd didn't work without it | 11:38 |
| locallycompact | see above branch | 11:39 |
| pedroalvarez | :) | 11:39 |
| pedroalvarez | that patch makes a lot of sense | 11:40 |
| paulsherwood | does the requirements.txt mandate specifying version for each dependency? if not, can't you just relax that? | 11:40 |
| thinkl33t | It doesnt | 11:41 |
| thinkl33t | you can specify an exact version, a package name, or say greater than a certain version | 11:41 |
| persia | It permits expressions of ranges of versions or precise versions. If these are overridden, often the software doesn't work (as best practice for requirements.txt is to only specify versions if/when things would otherwise break) | 11:41 |
| locallycompact | paulsherwood, we don't know what will change in new versions of those dependencies | 11:41 |
| locallycompact | if you specify >= x | 11:42 |
| locallycompact | something will stop master spontaneously | 11:42 |
| locallycompact | eventually | 11:42 |
| persia | locallycompact: Depends on how firmly upstream maintains the ABI, but yes, usually. | 11:42 |
| paulsherwood | locallycompact: true, but does that matter, really? | 11:42 |
| * rjek would rather the instructions recommended virtualenv or similar, rather than installing stuff globally | 11:43 | |
| persia | paulsherwood: More important is when requirements.txt specifies a set of versions that is not met by upstream master, in which case the developers are expressing that they believe the software will not work with current master. | 11:43 |
| paulsherwood | rjek: patches welcome :) | 11:44 |
| locallycompact | paulsherwood, yes that matters | 11:45 |
| locallycompact | that master doesn't break suddenly | 11:45 |
| paulsherwood | right... but our ci can check for that, then? | 11:45 |
| rjek | Grabbing unversioned things from pip can mean your thing can unpredictably stop working | 11:46 |
| rjek | also see: Dockerfiles that curl | bash | 11:46 |
| pedroalvarez | heh | 11:46 |
| pedroalvarez | tbh I've seen requirements without limitations in other projects | 11:47 |
| pedroalvarez | and that works fine | 11:47 |
| pedroalvarez | whenever they see a problem, they either adjust their code, or limit the version until they can adjust it | 11:47 |
| persia | paulsherwood: Maybe. Because something builds doesn't mean that it can run. Because something runs limited tests doesn't mean it won't break horridly. | 11:47 |
| rjek | When you need to fix your system in 2025, I think installing ybd as it currently is may be exciting. | 11:47 |
| rjek | (This is a general problem with *lots* of software, and I don't think anybody has found a pleasing solution yet) | 11:48 |
| persia | The most insidious sort of change is when a function changes what the parameters mean without changing the function signature, so that the same types are consumed, but used differently. Many sorts of tests pass in this situation, but the software behaviour may be very different to the intent of the developers. | 11:48 |
| paulsherwood | hence my preference that the tooling should continue to build old things, rather than saying (in 2025) 'you need to use the old tool from 2016' | 11:49 |
| locallycompact | that's not what this is about | 11:50 |
| paulsherwood | no, agreed | 11:50 |
| persia | That preference is the only sane way to use a 2025 system to build 2016 software. | 11:50 |
| rjek | But it does mean that tooling needs to be supported for as long as the product they build is. | 11:50 |
| paulsherwood | yup | 11:51 |
| persia | "supported" is a funny word in open source. Most folk can cause software to be "supported" by asserting the identity of "developer". | 11:51 |
| persia | (assuming they have source) | 11:51 |
| rjek | Or you support an emulator that can run your 2016 software on 2025 hardware :) | 11:52 |
| * locallycompact wonders if there's anything quicker we can put in premerge than gcc three times. | 11:53 | |
| rjek | My concern is that things bitrot; would we expect somebody who wants to support their product for 20 years to maintain CI infrastructure that is constnatly running for those 20 years? What if they never need to make a fix until year 18? | 11:54 |
| rjek | I don't like it when important things are expensive :) | 11:54 |
| jmacs | persia: The process from having source to being able to support a piece of software can take years now, and I'd expect that to have grown exponentially by 2025 | 11:54 |
| locallycompact | I'd expect that to have shrunk drastically | 11:55 |
| jmacs | I fully expect the phrase "you have source, so you can fix it" to be untrue by then | 11:55 |
| * rjek wonders how many people think about even ten years into the future when designing and writing software | 11:55 | |
| locallycompact | I think that's untrue now | 11:55 |
| persia | jmacs: Why? | 11:55 |
| rjek | Getting stuff written ten years in the past running can be tricky and I don't see any reason to think that it's not going to get trickier. | 11:55 |
| jmacs | persia: Source code will be as complicated and difficult to understand as binaries are now. | 11:55 |
| persia | I suspect the number of folk comfortable with software archeology will increase to offset that change. I may be wrong. | 11:56 |
| locallycompact | ime code is simpler as languages get more sophisticated | 11:56 |
| paulsherwood | locallycompact: i suggest that maybe what you need is an *additional* way to get ybd, rather than changing one of the current ones? | 11:56 |
| locallycompact | what? | 11:56 |
| pedroalvarez | yeh, it would be possible to have 2 different requirement files | 11:57 |
| pedroalvarez | (one way to do it) | 11:57 |
| * paulsherwood wonders if the 'what?' is aimed at him... | 11:57 | |
| locallycompact | yes | 11:57 |
| franred | locallycompact, ime code is simpler as languages get more sophisticated -- so, you rely more and more in other people code and controlling less and less what your code does in reality | 11:58 |
| pedroalvarez | in any case, I think that moving all of the pip dependencies to the requirements file is good | 11:58 |
| paulsherwood | ok | 11:58 |
| SotK | pedroalvarez: +1 | 11:58 |
| paulsherwood | ok, so currently there are two documented ways to install ybd: install_dependencies.sh, and vagrant up | 11:59 |
| locallycompact | franred, no? I certainly have more control in more sophisticated languages | 11:59 |
| paulsherwood | it may be better, for both locallycompact and rjek, to have a way that involves virtualenv and precise versions | 12:01 |
| paulsherwood | but other users may still benefit from the current two ways | 12:01 |
| rjek | I got bitten by pip the otherday removing a load of perfectly functional libraries and replacing them with newer shinier versions that broken everything else. | 12:01 |
| locallycompact | no user benefits from a dependency update that breaks execution surprisingly | 12:01 |
| locallycompact | if someone publishes a new sandboxlib that passes *it's* ci and breaks ybd then it will break in any production definitions that is tracking ybd master | 12:02 |
| rjek | locallycompact: Or breaks execution subtly | 12:02 |
| locallycompact | or that | 12:02 |
| pedroalvarez | that was clear, and the suggestion was to make something else to install ybd, but not change the current approach | 12:03 |
| paulsherwood | i'm ok with a new installation that does virtualenv and whatever other improvements. i just see no reason to remove the current approach, or make ybd *require* virtualenv | 12:04 |
| rjek | pinning installed dependancy versions globally is also bad, as it may break other software on the system, of course. | 12:05 |
| rjek | So such pins only really make sense in situations where tools like virtualenv are used | 12:06 |
| paulsherwood | that's my issue | 12:06 |
| locallycompact | pip install --help | 12:06 |
| locallycompact | -U, --upgrade Upgrade all specified packages to the newest available version. The handling of dependencies depends | 12:06 |
| locallycompact | on the upgrade-strategy used. | 12:06 |
| locallycompact | so one requirements.txt | 12:07 |
| locallycompact | shove -U where yo uwant | 12:07 |
| pedroalvarez | I don't understand what you mean | 12:07 |
| locallycompact | pip install -U -r requirements.txt | 12:07 |
| locallycompact | will ignore the pins | 12:07 |
| locallycompact | IIUC | 12:07 |
| pedroalvarez | no | 12:08 |
| pedroalvarez | I mean, I believe that nope, it will get the latest possible, but without ignoring the pins | 12:08 |
| SotK | that is my understanding of what will happen too | 12:08 |
| locallycompact | --upgrade-strategy <upgrade_strategy> | 12:09 |
| locallycompact | Determines how dependency upgrading should be handled. "eager" - dependencies are upgraded regardless | 12:09 |
| locallycompact | of whether the currently installed version satisfies the requirements of the upgraded package(s). | 12:09 |
| locallycompact | "only-if-needed" - are upgraded only when they do not satisfy the requirements of the upgraded | 12:09 |
| locallycompact | package(s). | 12:09 |
| locallycompact | yeah ok it's not doing that idno what hat's doing | 12:10 |
| pedroalvarez | nope, that doesn't change my understanding | 12:10 |
| paulsherwood | aiui the current install_dependencies.sh does not change any versions already installed, but installs things that are missing. it's been working ok so far, and it shouldn't *break* anything else on the system | 12:11 |
| locallycompact | it will break anything else on the system that requires a specific version of a dependency upgraded by ybd | 12:12 |
| paulsherwood | ybd doesn't upgrade anything | 12:12 |
| pedroalvarez | hm.. I'm not sure about that | 12:13 |
| paulsherwood | pip runs, recommends upgrades, but doesn't do them, afaik | 12:13 |
| pedroalvarez | i think it might, but not sure | 12:13 |
| paulsherwood | Collecting cherrypy Downloading CherryPy-8.1.2-py3-none-any.whl (456kB) | 12:15 |
| paulsherwood | Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python3.5/site-packages (from cherrypy) | 12:15 |
| paulsherwood | (for example) | 12:15 |
| persia | The potential breakage comes when software package X has dependency K that must be <=4.9.3 and software package Y has dependency K that must be >=5.7.14 | 12:18 |
| persia | Unless one can find a couple such packages on pypi today, it's probably easier to consider it theoretically. | 12:18 |
| pedroalvarez | Also, I don't know what happens if dependency X requires K >=5 and system has K < 5 | 12:19 |
| pedroalvarez | I'd say that pip will upgrade that package without asking, but not sure | 12:20 |
| paulsherwood | yup. i'm not against having a virtualenv + pinned list approach, as a way to be sure to avoid such issues. however i'm not happy to drop the current simple approach, since it's been working fin for some usecases | 12:20 |
| pedroalvarez | and i think thats fair | 12:20 |
| paulsherwood | tvm :) | 12:21 |
| *** ctbruce has quit IRC | 13:06 | |
| *** ctbruce has joined #baserock | 13:27 | |
| *** ctbruce has quit IRC | 13:27 | |
| *** ctbruce has joined #baserock | 13:27 | |
| jjardon | locallycompact: https://gitlab.com/baserock/ybd/commits/jjardon/py3.4 | 13:30 |
| *** toscalix has quit IRC | 14:28 | |
| *** gtristan has quit IRC | 14:33 | |
| *** toscalix has joined #baserock | 14:59 | |
| *** faybrocklebank has quit IRC | 16:45 | |
| *** ctbruce has quit IRC | 16:57 | |
| *** franred has quit IRC | 18:10 | |
| *** ctbruce has joined #baserock | 18:28 | |
| *** rdale has quit IRC | 18:36 | |
| *** locallycompact has quit IRC | 18:50 | |
| *** toscalix has quit IRC | 18:58 | |
| *** gtristan has joined #baserock | 19:14 | |
| *** locallycompact has joined #baserock | 19:43 | |
| *** gtristan has quit IRC | 22:02 | |
| *** locallycompact has quit IRC | 22:26 | |
| *** ctbruce has quit IRC | 23:23 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!