| *** narispo has quit IRC | 00:01 | |
| *** narispo has joined #buildstream | 00:02 | |
| *** narispo has quit IRC | 00:33 | |
| *** narispo has joined #buildstream | 00:33 | |
| *** narispo has quit IRC | 01:03 | |
| *** narispo has joined #buildstream | 01:03 | |
| *** narispo has quit IRC | 01:34 | |
| *** narispo has joined #buildstream | 01:34 | |
| *** tristan has quit IRC | 01:48 | |
| *** narispo has quit IRC | 02:04 | |
| *** narispo has joined #buildstream | 02:05 | |
| *** narispo has quit IRC | 02:34 | |
| *** narispo has joined #buildstream | 02:34 | |
| *** narispo has quit IRC | 03:05 | |
| *** narispo has joined #buildstream | 03:06 | |
| *** narispo has quit IRC | 03:36 | |
| *** narispo has joined #buildstream | 03:36 | |
| *** lantw44 has quit IRC | 04:00 | |
| *** lantw44 has joined #buildstream | 04:02 | |
| *** narispo has quit IRC | 04:06 | |
| *** narispo has joined #buildstream | 04:07 | |
| *** narispo has quit IRC | 04:36 | |
| *** narispo has joined #buildstream | 04:36 | |
| *** narispo has quit IRC | 05:07 | |
| *** narispo has joined #buildstream | 05:07 | |
| *** narispo has quit IRC | 05:38 | |
| *** narispo has joined #buildstream | 05:38 | |
| *** narispo has quit IRC | 06:08 | |
| *** narispo has joined #buildstream | 06:08 | |
| *** narispo has quit IRC | 06:39 | |
| *** narispo has joined #buildstream | 06:39 | |
| *** narispo has quit IRC | 07:08 | |
| *** narispo has joined #buildstream | 07:09 | |
| *** qinusty has joined #buildstream | 07:16 | |
| *** samkirkham has joined #buildstream | 07:27 | |
| *** rdale has joined #buildstream | 07:31 | |
| gitlab-br-bot | coldtom opened issue #1107 (Filter elements do not preserve integration commands) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1107 | 07:34 |
|---|---|---|
| *** narispo has quit IRC | 07:39 | |
| *** narispo has joined #buildstream | 07:39 | |
| *** benschubert has joined #buildstream | 07:57 | |
| *** narispo has quit IRC | 08:09 | |
| *** narispo has joined #buildstream | 08:10 | |
| *** tme5 has joined #buildstream | 08:25 | |
| *** phil has joined #buildstream | 08:34 | |
| *** narispo has quit IRC | 08:40 | |
| *** narispo has joined #buildstream | 08:40 | |
| *** Becky has joined #buildstream | 08:42 | |
| *** Becky has quit IRC | 08:43 | |
| *** tpollard has joined #buildstream | 08:56 | |
| *** alexandrufazakas has joined #buildstream | 09:01 | |
| *** Becky has joined #buildstream | 09:02 | |
| *** narispo has quit IRC | 09:13 | |
| *** narispo has joined #buildstream | 09:13 | |
| *** alexandrufazakas has left #buildstream | 09:18 | |
| *** lachlan has joined #buildstream | 09:29 | |
| *** traveltissues has joined #buildstream | 09:30 | |
| *** jonathanmaw has joined #buildstream | 09:35 | |
| gitlab-br-bot | traveltissues approved MR !1547 (jennis/add_dep_name_to_cache_key->master: Add the name of build deps to the strict cache key) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1547 | 09:41 |
| *** lachlan has quit IRC | 09:46 | |
| *** toscalix has joined #buildstream | 09:46 | |
| *** lachlan has joined #buildstream | 09:50 | |
| gitlab-br-bot | juergbi opened issue #1108 (Consolidate helpers to run tests in subprocess) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1108 | 10:08 |
| *** narispo has quit IRC | 10:08 | |
| tme5 | hi everyone, so, i've started to sketch out some of the work for the local sourcing of downloadablefileplugins | 10:08 |
| *** narispo has joined #buildstream | 10:09 | |
| gitlab-br-bot | tlater approved MR !1547 (jennis/add_dep_name_to_cache_key->master: Add the name of build deps to the strict cache key) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1547 | 10:10 |
| tme5 | the first two parts of my email proposal, i think, are uncontroversial? should I just make the changes and MR and then you will decide at review? | 10:15 |
| tme5 | just wondering how to ensure you are happy with it :) | 10:15 |
| juergbi | tme5: maybe ping tristan when he's around to check whether he is happy with your ML reply. just making the changes and opening an MR is fine, of course. however, if there is no consensus on the approach yet, there is a risk that larger changes will be required before a merge | 10:21 |
| gitlab-br-bot | tlater approved MR !1548 (jennis/add_project_to_proto->master: Add project name to the proto and the cache key) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1548 | 10:24 |
| tme5 | juergbi, alright, thank you! | 10:24 |
| juergbi | benschubert, jennis, tlater[m], traveltissues: thanks for your review of !1499. I've addressed or at least replied to all current comments. Can you please check the threads that you started? | 10:25 |
| gitlab-br-bot | MR !1499: Use buildbox-casd for CAS access https://gitlab.com/BuildStream/buildstream/merge_requests/1499 | 10:25 |
| traveltissues | looking now | 10:25 |
| benschubert | juergbi: on it! | 10:25 |
| juergbi | ta | 10:25 |
| jennis | juergbi, will do it soon :) | 10:28 |
| *** narispo has quit IRC | 10:38 | |
| benschubert | juergbi: would it make sense installing the buildbox binaries in the docker images directly? | 10:38 |
| benschubert | instead of pulling them everytime? | 10:38 |
| *** narispo has joined #buildstream | 10:39 | |
| juergbi | benschubert: they are already built as part of the docker image CI. not done in every buildstream CI run | 10:40 |
| juergbi | with the exception of WSL as that doesn't use docker, there the binary is downloaded and installed in buildstream CI | 10:40 |
| juergbi | or are you suggesting to download and install the buildbox-casd binary when creating the docker images instead of building it from source? | 10:41 |
| benschubert | Oh I see, the pulling is only for the WSL run, I misunderstood the ci config :) | 10:42 |
| benschubert | I didn't realize it was not for all tests | 10:42 |
| benschubert | thanks | 10:42 |
| benschubert | juergbi: I have no further comments on the MR, once the others are happy with it, I am! | 10:44 |
| juergbi | great, thanks | 10:45 |
| juergbi | I think Kinnison also wanted to take a look when everything pending is resolved | 10:45 |
| *** lachlan has quit IRC | 10:47 | |
| * Kinnison would like to; but is also okay for it to merge without that if he'd be a blocker | 10:52 | |
| * Kinnison has meetings through until 14Z00 today so wouldn't be able to look before then | 10:52 | |
| juergbi | Kinnison: if you have time to review it at some point today, I can certainly wait. let me know | 10:59 |
| * Kinnison shall plan to review around 14Z00 and will let you know before then if it'll not be possible | 11:00 | |
| juergbi | ta | 11:00 |
| juergbi | there seems to be a CI issue with integration-tests-base.v1.x86_64.tar.xz downloads from digital ocean | 11:00 |
| juergbi | I'm seeing: File downloaded from https://bst-integration-test-images.ams3.cdn.digitaloceanspaces.com/integration-tests-base.v1.x86_64.tar.xz has sha256sum 'cfc626ca2f2d5186a056969e0df7c38d12e88652346d8d6a1ce653bb9f650ac4', not '3eb559250ba82b64a68d86d0636a6b127aa5f6d25d3601a79f79214dc9703639'! | 11:00 |
| juergbi | in multiple jobs with varying hexstrings for the first sha256sum | 11:00 |
| juergbi | manual download locally seems to result in a file with the right sha256sum, though | 11:01 |
| tlater[m] | juergbi: Sounds more like there's a problem with buildstream | 11:03 |
| juergbi | it seems to happen in multiple branches, though | 11:04 |
| tlater[m] | Maybe we're accidentally ignoring some sort of abort signal and writing a half-complete file? | 11:04 |
| juergbi | it could definitely be a buildstream bug but it just started happening | 11:04 |
| tlater[m] | Hm | 11:04 |
| tlater[m] | We can't get access to the artifacts directory on one of the runners, can we? | 11:04 |
| tlater[m] | Check what the local tar looks like... | 11:05 |
| jennis | tlater[m], traveltissues, if you're ok with it, I'll close the first MR and just merge the second, as it includes the changes of the first? | 11:05 |
| traveltissues | jennis, ga | 11:05 |
| tlater[m] | jennis: Yup | 11:06 |
| jennis | tlater[m], this will address your BST_ARTIFACT_VERSION bump discussion | 11:06 |
| tlater[m] | Not entirely | 11:06 |
| tlater[m] | I'm talking about the whole renaming parts of the cache key thing, removing the empty dictionaries and hardcoded values. | 11:07 |
| gitlab-br-bot | jennis closed MR !1547 (jennis/add_dep_name_to_cache_key->master: Add the name of build deps to the strict cache key) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1547 | 11:07 |
| benschubert | juergbi: I'm also seeing this on a branch I just rebased, that was passing last week | 11:07 |
| tlater[m] | But well, I've reconsidered how quickly I think we can resolve that, so feel free to merge anyway. | 11:07 |
| laurence | jennis, thanks for the benchmarks updates! the dates format on the second table is a bit confusing | 11:08 |
| benschubert | but the bug doesn't happen all the time | 11:08 |
| tlater[m] | juergbi: Maybe we should run a couple tests on master | 11:08 |
| laurence | jennis, looking at it, I think the very bottom row should be at the top?? | 11:08 |
| benschubert | jennis: thanks for the update! | 11:09 |
| * tlater[m] wonders if someone's got ahold of our machine and is trying to feed us malicious base images | 11:09 | |
| *** narispo has quit IRC | 11:09 | |
| *** narispo has joined #buildstream | 11:09 | |
| gitlab-br-bot | traveltissues approved MR !1499 (juerg/casd->master: Use buildbox-casd for CAS access) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1499 | 11:09 |
| laurence | we seem to be getting slower with more builders... | 11:10 |
| laurence | sigh | 11:10 |
| tlater[m] | Not consistently, laurence | 11:11 |
| * tlater[m] is happy to chalk that up to noise | 11:11 | |
| tlater[m] | We are getting slower in general though | 11:11 |
| benschubert | jennis: on mybinder, is it possible to see the commits info still? It doesn't show the popup | 11:14 |
| jennis | laurence, no | 11:16 |
| jennis | benschubert, should show the commit when you click on it | 11:17 |
| jennis | tlater[m], sorry, are you happy to merge with just the one BST_ARTIFACT_VERSION bump | 11:17 |
| jennis | ? | 11:17 |
| tlater[m] | jennis: Yeah, that's ok | 11:18 |
| jennis | laurence, ah, I just forgot to change the dates on the last row of that second table /o\ turns out my triple checking was not enough | 11:18 |
| jennis | benschubert, oh the MR number is missing on the information... I will change this | 11:19 |
| benschubert | cheers :) | 11:19 |
| benschubert | now it works, after realoding | 11:20 |
| laurence | jennis, ah ok, it's just the dates | 11:25 |
| jennis | benschubert, you should now see the MR number too | 11:28 |
| benschubert | jennis: thanks a lot! | 11:29 |
| *** lachlan has joined #buildstream | 11:36 | |
| jennis | tlater[m], traveltissues, I've assigned to marge | 11:41 |
| *** lachlan has quit IRC | 11:42 | |
| *** lachlan has joined #buildstream | 11:45 | |
| *** lachlan has quit IRC | 11:49 | |
| *** lachlan has joined #buildstream | 12:03 | |
| *** narispo has quit IRC | 12:09 | |
| *** narispo has joined #buildstream | 12:09 | |
| *** lachlan has quit IRC | 12:19 | |
| tpollard | hitting the checksum issue a lot | 12:20 |
| tpollard | locally and in the pipelines | 12:23 |
| juergbi | oh, happening locally as well? interesting | 12:28 |
| tpollard | yep | 12:28 |
| tpollard | source determinism tests seem to be the first to hit it, and just hang indefinitely for me | 12:29 |
| benschubert | tpollard: that's weird, for me the error is in test_fallback_platform_fails | 12:30 |
| tpollard | I think it can be any tests that triggers a download of that tarball | 12:31 |
| tpollard | I've seen it on the buildtree tests too | 12:31 |
| tpollard | like https://gitlab.com/BuildStream/buildstream/-/jobs/275069912 | 12:31 |
| benschubert | right | 12:33 |
| juergbi | I can't reproduce it locally so far | 12:33 |
| benschubert | One bug I definitely noticed before is that we silently download failures (like if it's a tarball that does require login and we get redirected to the login page, we'll end up downloading the logging page) | 12:34 |
| benschubert | so we might have troubles reaching the file | 12:34 |
| juergbi | tpollard: can you reproduce it frequently/easily enough (running one particular test multiple times) to be able to do some debugging? maybe bisect? | 12:34 |
| benschubert | why, I have no idea | 12:34 |
| juergbi | maybe add a bit of code that keeps the file with the wrong checksum somewhere | 12:35 |
| juergbi | so we could take a look what was downloaded instead of the right file. i.e., truncated or error page | 12:35 |
| tpollard | juergbi: I'll try the next time it happens | 12:35 |
| juergbi | or we could at least print out the size of the wrong file as well, I suppose | 12:36 |
| benschubert | Kinnison: https://gitlab.com/BuildStream/buildstream/merge_requests/1537/ has been updated to show the correct kind name, if you want to have another look | 12:39 |
| benschubert | and all the source tests are run as part of the pipeline | 12:39 |
| tpollard | https://gitlab.com/BuildStream/buildstream/-/jobs/274997414 so I don't always get it to hit with the git type, but more often than not at least one type will have for test_deterministic_source_umask for me | 12:42 |
| Kinnison | benschubert: I'll look soon | 12:44 |
| Kinnison | benschubert: Yep, 👍 | 12:51 |
| *** narispo has quit IRC | 13:00 | |
| *** narispo has joined #buildstream | 13:00 | |
| *** lachlan has joined #buildstream | 13:08 | |
| *** bochecha has joined #buildstream | 13:15 | |
| *** narispo has quit IRC | 13:30 | |
| *** narispo has joined #buildstream | 13:30 | |
| *** lachlan has quit IRC | 13:53 | |
| Kinnison | juergbi: I like how -ve this MR is, lots of code gone \o/ | 13:58 |
| juergbi | yes, me too :) net negative line count if we ignore the generated code | 13:59 |
| tlater[m] | Kinnison: To be fair, it's all hidden in a magic binary now | 13:59 |
| Kinnison | :D | 14:00 |
| tlater[m] | But I agree - I especially like that we can now clean caches without giving up on parallelism :) | 14:00 |
| tlater[m] | Is there any reason to avoid creating multiple multiprocessing.Queue? | 14:03 |
| * tlater[m] sees some code here that's going to great lengths to avoid creating more than one of them. | 14:03 | |
| Kinnison | Every time you make a queue, it uses FDs up | 14:04 |
| Kinnison | for as long as it's alive | 14:04 |
| tlater[m] | So if I create and close a couple it's not an issue? | 14:04 |
| tlater[m] | Sequentially, that is | 14:04 |
| *** narispo has quit IRC | 14:05 | |
| Kinnison | so long as they're properly chucked away, no issue at all | 14:05 |
| Kinnison | the main queue issue we had was that element jobs would create them in __init__ before | 14:05 |
| Kinnison | which meant when you had 70k pull jobs, you ran out of FDs | 14:06 |
| *** narispo has joined #buildstream | 14:06 | |
| tpollard | urk | 14:06 |
| Kinnison | because the scheduler process tried to create 140k FDs | 14:06 |
| tlater[m] | Hah, no, I'm just looking at a loop of probably 6 that's trying to share a queue for each element, although it's not kept alive for more than a couple of seconds at most. | 14:07 |
| tlater[m] | It's bleeding into the signatures of at least 4 functions, so let's clean that a little :) | 14:08 |
| Kinnison | juergbi: a few nits/queries, but otherwise I think that looks excellent. | 14:25 |
| juergbi | ok, thanks, will reply | 14:27 |
| *** cs-shadow has joined #buildstream | 14:33 | |
| *** lachlan has joined #buildstream | 14:38 | |
| gitlab-br-bot | traveltissues approved MR !1546 (tpollard/frontendelement->master: Don't directly handle Elements and Queues in App) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1546 | 14:39 |
| *** lachlan has quit IRC | 14:41 | |
| jennis | Are we having CI problems? | 14:44 |
| jennis | Can't seem to reproduce this umask test failure locally and it's only failed on a few of the jobs | 14:44 |
| tlater[m] | jennis: juergbi mentioned some shenanigans with potentially breaking downloads | 14:44 |
| jennis | :( | 14:45 |
| tlater[m] | Sounds like a different issue though | 14:45 |
| tpollard | jennis: yep, fails for me locally often two though | 14:45 |
| jennis | Looks like it might be a download problem: FileNotFoundError | 14:45 |
| tpollard | or any test that fetches the integration tarball | 14:45 |
| * tlater[m] remembers thinking he should include that tarball in the repo because network issues would happen two years ago | 14:46 | |
| tlater[m] | Now you hear me! | 14:47 |
| tlater[m] | ;p | 14:47 |
| tpollard | jennis: checksums not matching https://gitlab.com/BuildStream/buildstream/-/jobs/274997414 | 14:47 |
| tlater[m] | jennis: If you want to, the credentials for that server should be somewhere on the wiki | 14:53 |
| tlater[m] | You could double check if something weird's happening :) | 14:53 |
| tpollard | we've got access | 14:55 |
| tpollard | but the checksum matches | 14:55 |
| tlater[m] | Have you checked server logs? | 14:55 |
| tlater[m] | Maybe connections are being interrupted | 14:55 |
| * tlater[m] imagines it's some sort of apache server | 14:56 | |
| tlater[m] | Although we shouldn't see the issue that consistently then... | 14:56 |
| tlater[m] | Maybe someone is MITM us | 14:56 |
| cs-shadow | this may not be related but in the past I've seen such mismatches when the server returns an error pages with a 200 error code, so we end checksuming text like "500 internal server error" | 14:56 |
| tpollard | yep I would suspect it something like that | 14:57 |
| tlater[m] | Oh, that would make sense | 14:57 |
| tpollard | but I would expect that to be a consistent wrong hash we'd get maybe? | 14:57 |
| tlater[m] | Could contain a timestamp | 14:57 |
| juergbi | I improved the error message in a test branch to include the file size | 14:58 |
| tpollard | true | 14:58 |
| juergbi | and it was 1 and 3 MB in failing tests | 14:58 |
| juergbi | so unlikely to be an error page | 14:58 |
| tlater[m] | That's a big error page | 14:59 |
| tpollard | I'll try to get on the droplet hosting it | 15:02 |
| * tpollard notes it's not actually a dedicated droplet | 15:04 | |
| tpollard | nothing looks out of the ordinary | 15:07 |
| tpollard | digital ocean's status page isn't reporting any networking problems | 15:08 |
| *** bochecha has quit IRC | 15:13 | |
| gitlab-br-bot | marge-bot123 merged MR !1548 (jennis/add_project_to_proto->master: Cache key calculation changes: add project name and name of dependencies) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1548 | 15:18 |
| *** lachlan has joined #buildstream | 15:18 | |
| *** lachlan has quit IRC | 15:23 | |
| *** lachlan has joined #buildstream | 15:29 | |
| *** lachlan has quit IRC | 15:34 | |
| tlater[m] | So, I've been using the fancy new get_enum: https://hastebin.com/rofujulane.md | 15:45 |
| tlater[m] | benschubert: I think you'd know more about this? | 15:45 |
| tlater[m] | For reference, here's how I'm accessing that: https://hastebin.com/igujiyefeb.js | 15:47 |
| benschubert | tlater[m]: could you post gitlab snippets? corporate proxies :'D | 15:50 |
| tlater[m] | Ah, sec | 15:50 |
| tlater[m] | benschubert: https://gitlab.com/BuildStream/buildstream/snippets/1887072 | 15:52 |
| benschubert | oh, BOTH = "both", INDEX = "index", etc should fix it | 15:53 |
| benschubert | we match on the value of the enum :) | 15:53 |
| tlater[m] | AH! | 15:54 |
| tlater[m] | Ta benschubert, I thought the magic __str__ I found elsewhere did that. | 15:54 |
| benschubert | ah no, it's only for display | 15:55 |
| benschubert | If you have an idea on how to improve the docs, let me know, it would be good to help others :D | 15:55 |
| tlater[m] | I'll have a read and let you know :) | 15:56 |
| benschubert | cheers | 15:56 |
| gitlab-br-bot | marge-bot123 merged MR !1546 (tpollard/frontendelement->master: Don't directly handle Elements and Queues in App) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1546 | 15:57 |
| *** tpollard has quit IRC | 16:11 | |
| *** lachlan has joined #buildstream | 16:17 | |
| *** toscalix has quit IRC | 16:24 | |
| *** lachlan has quit IRC | 16:26 | |
| *** narispo has quit IRC | 16:28 | |
| *** narispo has joined #buildstream | 16:28 | |
| *** raoul has quit IRC | 16:34 | |
| *** tme5 has quit IRC | 16:34 | |
| gitlab-br-bot | jennis opened MR !1553 (jennis/load_artifact_dependencies->master: Add the ability to load (build) deps from an artifact ref) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1553 | 16:35 |
| *** lachlan has joined #buildstream | 16:36 | |
| adds68 | has anyone ever seen: SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE. | 16:42 |
| adds68 | when setting up a bst-artifact cache? | 16:42 |
| adds68 | I am using a valid certificate provided by lets encrypt | 16:42 |
| tlater[m] | No! | 16:43 |
| tlater[m] | Haha | 16:43 |
| tlater[m] | adds68: Are you running the docker image? | 16:43 |
| adds68 | tlater[m] yea | 16:43 |
| adds68 | tlater[m], i'm also trying to debug it on my machine and passing: https://paste.gnome.org/pm1ujj1rz | 16:44 |
| adds68 | which gives the same error | 16:44 |
| tlater[m] | adds68: give me a minute, I have a suspicion as to what might cause that | 16:45 |
| adds68 | tlater[m], thanks! | 16:45 |
| tlater[m] | adds68: It'll go into http mode if server-key isn't set | 16:46 |
| tlater[m] | Yours is | 16:46 |
| tlater[m] | So i'm not sure after all :( | 16:47 |
| adds68 | =/ | 16:47 |
| tlater[m] | If you'd like to debug manually, it should be pretty easy to find - it's in the click definition of _cas/casserver.py | 16:48 |
| tlater[m] | I suspect that this: https://gitlab.com/BuildStream/buildstream/blob/master/src/buildstream/_cas/casserver.py#L138 | 16:49 |
| tlater[m] | Evaluates to false somehow | 16:49 |
| *** lachlan has quit IRC | 16:50 | |
| adds68 | tlater[m], i wonder if this is needed? root_certificates=client_certs_bytes, | 16:50 |
| tlater[m] | You look to be specifying client certs though? | 16:51 |
| tlater[m] | I've definitely run with those empty before | 16:51 |
| tlater[m] | But well, you obviously can't push then | 16:52 |
| tlater[m] | adds68: IME these kinds of issues aren't actually caused by the server though. | 16:53 |
| adds68 | they are definitely not empty, the actual certificate also has LEs root certificate inside as a chain | 16:53 |
| tlater[m] | It'll be some apache default message or somesuch, and you just don't have your ports set up right | 16:53 |
| adds68 | i'm not using apache? | 16:53 |
| adds68 | or does the cache use apache? | 16:53 |
| tlater[m] | Well, some other service that's returning on a port you don't think is running anything | 16:54 |
| tlater[m] | Have you tried telneting the address? | 16:54 |
| tlater[m] | You can try `telnet localhost 1101`, if you type anything non-ssl the server should throw a warning and disconnect you. | 16:56 |
| tlater[m] | And if you go "hi" it'll start ssling :) | 16:56 |
| adds68 | tlater[m], doing telnet to my url on port 1101 or 1102 returns "welcome to nginx" | 16:57 |
| adds68 | could that be the case? | 16:57 |
| tlater[m] | No | 16:58 |
| tlater[m] | You'll need to set up nginx to forward to your bst-artifact-server | 16:58 |
| adds68 | hmm i didn't have to do this before, does bst not handle incoming requests itself? | 16:59 |
| tlater[m] | It does, but nginx sits on top of your system and manages all connections | 17:00 |
| tlater[m] | It's a reverse proxy | 17:00 |
| tlater[m] | If you ran bst raw, it'd do things for you | 17:00 |
| tlater[m] | But alas, that's not how your host works :) | 17:00 |
| * tlater[m] personally runs https://github.com/jwilder/nginx-proxy | 17:00 | |
| tlater[m] | But this is way beyond what I can really help you with right now, I don't know your server requirements :) | 17:01 |
| adds68 | tlater[m], i just turned off nginx and it still fails | 17:01 |
| tlater[m] | What does telnet say? | 17:01 |
| adds68 | tlater[m], it connects, but if i type hi twice, it disconnects | 17:03 |
| tlater[m] | If it's not connecting at all, how did you configure docker to set up your ports? | 17:03 |
| tlater[m] | Ah, that seems correct | 17:03 |
| tlater[m] | Any warnings from the bst artifact server? | 17:03 |
| adds68 | tlater[m], once the connection closes, it prints: " SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER." | 17:04 |
| tlater[m] | It's working then | 17:04 |
| tlater[m] | You should be able to point buildstream at that | 17:04 |
| tlater[m] | If that doesn't work, double check your project.conf? | 17:05 |
| *** lachlan has joined #buildstream | 17:05 | |
| adds68 | WARNING Failed to initialize remote https://cache.alvarezpiedehierro.com:1102: failed to connect to all addresses | 17:05 |
| adds68 | That is the error i get back from bst | 17:05 |
| adds68 | and every time bst tries, the server prints "OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE" | 17:06 |
| tlater[m] | Is this set up as push/pull? | 17:06 |
| adds68 | tlater[m], yea | 17:07 |
| tlater[m] | Oh, wait, I thought that was the client messge | 17:07 |
| tlater[m] | Hmm, did you set the correct CN? | 17:07 |
| adds68 | tlater[m], yea subject=CN = cache.alvarezpiedehierro.com | 17:08 |
| tlater[m] | It frankly sounds like you're not using the certbot key, but the auto-generated one, and that your client is rejecting it because the CN is wrong. | 17:08 |
| tlater[m] | If you try to recreate the docker image with -e CN=cache.alvarezpiedehierro.com, does it work? | 17:09 |
| adds68 | I suspected that in Docker, but not on my host machine? | 17:09 |
| tlater[m] | You're running bst raw then? | 17:10 |
| adds68 | tlater[m], at the moment yes, just to make it easier to debug | 17:10 |
| tlater[m] | Gotcha | 17:11 |
| adds68 | the server is provisioned with ansible, so anything on this machine is also used by the docker container | 17:11 |
| tlater[m] | I also assume you've quadruple-checked the client cert already? | 17:11 |
| adds68 | as in the cert is actually there? | 17:12 |
| tlater[m] | Well, that the client certificate is correct and there. | 17:12 |
| tlater[m] | Like, removed the old one, regenerated one, readded it, etc. | 17:12 |
| adds68 | i assumed for pull, if my certificate is provided by LE then i don't need a copy of that cert on my machine | 17:12 |
| adds68 | tlater[m], yea i'm getting close to being rate limited by LE :( | 17:12 |
| tlater[m] | The client will need a certificate to push. | 17:12 |
| tlater[m] | You don't need to regenerate the LE cert | 17:13 |
| adds68 | yea which is fine | 17:13 |
| tlater[m] | Just the client cert | 17:13 |
| adds68 | i only want to be able to pull as a test atm, push will be done via CI | 17:13 |
| tlater[m] | I'm pretty sure that if you specify push it'll fail to set up the remote unless you have the cert setup correctly, even if you don't intend to push. | 17:13 |
| *** narispo has quit IRC | 17:13 | |
| tlater[m] | So set up buildstream to treat it only as a pull remote? | 17:13 |
| *** narispo has joined #buildstream | 17:13 | |
| tlater[m] | This is why the server is complaining about the client not giving a cert | 17:14 |
| adds68 | i've tried that also and get the same error regarding no peert certificate | 17:14 |
| tlater[m] | Ugh, annoying. That might be a bug. Could you try setting up push? | 17:14 |
| adds68 | hahahahaha | 17:15 |
| adds68 | it is tlater[m] | 17:15 |
| tlater[m] | Grmbl | 17:15 |
| adds68 | tlater[m], if you remove --enable-push but keep --client-certs then it still assumes you want to enable push | 17:15 |
| adds68 | it's my fault and i should have removed both, however just using 1101 seems to be working correctly now | 17:16 |
| tlater[m] | Cool | 17:16 |
| tlater[m] | i hope you'll file the bug, too :) | 17:16 |
| adds68 | yes, i am wondering though | 17:16 |
| adds68 | If i run this via Docker | 17:16 |
| adds68 | will i need to have two instances running for port 1101 and 1102 to stop this happening to people who *just* want to consume? | 17:16 |
| tlater[m] | That was the original design iirc | 17:17 |
| tlater[m] | The error messages could be more clear | 17:17 |
| adds68 | ack, ok thanks! | 17:17 |
| tlater[m] | But I feel like it'd be pretty easy to make that not happen, and we should document it better... | 17:18 |
| tlater[m] | ta for the patience adds68 ;) | 17:18 |
| adds68 | No thanks for your help, it knew it would be something stupid and tripple checked permissions etc | 17:18 |
| adds68 | i'll open an issue now to try and explain how i feel into this problem | 17:19 |
| tlater[m] | tyvm :) | 17:19 |
| *** lachlan has quit IRC | 17:24 | |
| *** jonathanmaw has quit IRC | 17:42 | |
| gitlab-br-bot | adds68 opened issue #1109 (bst-artifact-server misconfigured options) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1109 | 17:44 |
| *** traveltissues has quit IRC | 17:56 | |
| *** lachlan has joined #buildstream | 18:14 | |
| *** lachlan has quit IRC | 18:58 | |
| *** lachlan has joined #buildstream | 19:00 | |
| *** bochecha has joined #buildstream | 19:03 | |
| *** lachlan has quit IRC | 19:11 | |
| *** lachlan has joined #buildstream | 19:16 | |
| *** lachlan has quit IRC | 19:27 | |
| *** lachlan has joined #buildstream | 19:56 | |
| *** lachlan has quit IRC | 20:05 | |
| *** narispo has quit IRC | 20:06 | |
| *** lachlan has joined #buildstream | 20:06 | |
| *** narispo has joined #buildstream | 20:06 | |
| *** lachlan has quit IRC | 20:19 | |
| *** bochecha has quit IRC | 21:16 | |
| *** rdale has quit IRC | 21:38 | |
| *** narispo has quit IRC | 22:37 | |
| *** narispo has joined #buildstream | 22:37 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!