| *** jjardon has joined #baserock | 00:01 | |
| *** Stanto has quit IRC | 00:04 | |
| *** Stanto has joined #baserock | 00:07 | |
| *** petefoth has joined #baserock | 00:07 | |
| *** kejiahu has joined #baserock | 00:28 | |
| *** tlsa has joined #baserock | 00:30 | |
| *** jjardon_ has joined #baserock | 00:33 | |
| *** pacon has quit IRC | 00:33 | |
| *** jjardon has quit IRC | 00:36 | |
| *** tlsa_ has quit IRC | 00:36 | |
| *** sam_ has quit IRC | 00:36 | |
| *** kejiahu_ has quit IRC | 00:36 | |
| *** sambishop has joined #baserock | 00:36 | |
| *** jmacs_ has joined #baserock | 00:45 | |
| *** jjardon_ is now known as jjardon | 00:46 | |
| *** jmacs has quit IRC | 00:55 | |
| *** Guest19028 has quit IRC | 01:18 | |
| *** Guest19028 has joined #baserock | 01:18 | |
| *** Guest19028 is now known as persia_ | 01:18 | |
| *** persia__ has quit IRC | 01:19 | |
| *** persia__ has joined #baserock | 01:19 | |
| *** persia__ is now known as persia | 01:19 | |
| *** edcragg has quit IRC | 01:24 | |
| jeak_ | ? | 07:17 |
|---|---|---|
| *** edcragg has joined #baserock | 08:20 | |
| *** Kinnison_ is now known as Kinnison | 10:03 | |
| *** pacon has joined #baserock | 10:47 | |
| *** pacon has quit IRC | 11:49 | |
| *** pacon has joined #baserock | 11:50 | |
| *** pacon has quit IRC | 12:34 | |
| *** jmacs has joined #baserock | 13:01 | |
| *** jmacs_ has quit IRC | 13:08 | |
| *** petefotheringham has quit IRC | 13:20 | |
| *** petefotheringham has joined #baserock | 13:21 | |
| *** petefotheringham has quit IRC | 13:49 | |
| *** petefotheringham has joined #baserock | 13:49 | |
| paulsher1ood | hi jeak_ ? | 15:12 |
| paulsher1ood | does anyone know if there's an established pattern for cleaning up tmp files for re-entrant programs that may fail leaving tmp files behind? | 15:36 |
| paulsher1ood | eg if i were able to run n instances of ybd on one machine, and some fail, leaving their staging areas... next time i run a ybd i'd like it to clean up dead ones, but not the tmp files being used by live instances of ybd | 15:38 |
| Kinnison | One option is to inject a lockfile into a tempdir | 15:45 |
| Kinnison | if you can't lock it, you can't use/clean it up | 15:45 |
| Kinnison | typically, tmp files are not cleaned up if a program dies, so that investigation can happen | 15:46 |
| juergbi | paulsher1ood: depending on how they're used, you might be able to get away with unlinking them right after creation and just keeping the file descriptors open | 15:51 |
| juergbi | (or O_TMPFILE if you can depend on recent kernel) | 15:51 |
| juergbi | may not be an option when the temp files are to be used by external processes where you can't just pass the fd | 15:51 |
| juergbi | using systemd and tmpfiles.d you can configure automatic cleanup after N hours or days of last modification. default for /tmp is 10d | 15:54 |
| persia | Can't depend on recent kernel, if we're trying to make it work for arbitrary unices (enterprise linux distros, OS X, etc.) | 16:32 |
| juergbi | unlinking right after file creation is not much worse in practice. in the off chance that you hit the race condition, you'll leave one empty temp file around | 16:40 |
| juergbi | i suspect the bigger limitation is interaction with external processes | 16:41 |
| rjek | Also if the data to be written to them is sensitive, make sure you fstat the open fd and count the number of hardlinks it has and confirm it is zero before writing to it. | 16:42 |
| juergbi | rjek: isn't it typically sufficient to use mkstemp? | 16:43 |
| rjek | I assumed that wasn't an option for some reason due to all the discussion about doing it :) | 16:44 |
| juergbi | ah, i thought this was just about cleanup, which mkstemp doesn't directly solve | 16:44 |
| rjek | Indeed not | 16:45 |
| rjek | But writing your own mkstemp that does unlink (opening O_EXCL, unlink, fstat to confirm zero links) probably would. | 16:46 |
| juergbi | i would simply used unlink after mkstemp, if i can't use O_TMPFILE | 16:46 |
| juergbi | *use | 16:46 |
| persia | The issue there is that someone else with filesystem accesss could potentially link the file between those two calls. | 16:48 |
| juergbi | should only be possible for process of the same user, right? and protecting against that is somewhat futile with regular unix user-based access to processes | 16:49 |
| persia | If you are unconcerned with the ability to capture the data, that's fine. If you are concerned (e.g. you are storing plaintext of something typically enciphered), you need to do the zero check after unlinking to confirm there is no consistent log. | 16:49 |
| persia | Whether it needs to be the same user depends on umask. 002 is surprisingly popular. | 16:50 |
| juergbi | shouldn't mkstemp's 0600 permission help here | 16:50 |
| juergbi | also, hardlink only works on the same filesystem. to avoid persistence, you will therefore create a tempfile on tmpfs and thus, the hardlink itself can't cause persistent storage | 16:51 |
| juergbi | it might still be accessed by other processes running as the same user but that's true with or without the zero link check | 16:53 |
| persia | The permission value is libc dependent. | 16:54 |
| persia | But mkostemp(foo, bar, O_EXCL) might be a reduction of steps in the formula above. | 16:55 |
| juergbi | persia: well, i think we can ignore ancient glibc versions (2.06 and earlier). mac also claims to handle it correctly and recent POSIX mandates it | 17:10 |
| persia | Ah, I wasn't aware of the recent POSIX mandate. | 17:11 |
| juergbi | since 2008 | 17:12 |
| persia | musl seems to have used 600 since mkstemp and mkostemp were first supported. | 17:12 |
| persia | FreeBSD's libc also. | 17:13 |
| juergbi | the issue in glibc was in 1997 | 17:14 |
| persia | Maybe I'm being overparanoid, or showing my age when I suggest we can't trust libc impementation for 0600 :) | 17:14 |
| juergbi | there could well be some broken libc still out there, however, i wouldn't work around such OS bugs in my software | 17:15 |
| persia | I'm only concerned about recent systems running. I suspect it is safe. | 17:16 |
| persia | I can't confirm uclibc is safe, but in practice, uclibc is an unlikely libc for a development environment. | 17:16 |
| persia | bionic also does the right thing, completing my list of platforms to check. | 17:18 |
| juergbi | uclibc is safe as well | 17:18 |
| juergbi | return __gen_tempname (template, __GT_FILE, 0, S_IRUSR | S_IWUSR); | 17:18 |
| persia | Oh, cool. I was just looking for the string 0600, rather than doing semantic analysis. | 17:19 |
| persia | I don't find any mention of permissions for _mktemp_s(), but there are other things that would need changing to run on a non-POSIX OS, and the documentation of the PCL is limited, at best. | 17:22 |
| juergbi | yes, no idea about Windows in this regard out of the top of my head | 17:25 |
| persia | Yeah, it wasn't really on my list of potential platforms, except I got carried away :) | 17:25 |
| *** robtaylo1 is now known as robtaylor | 18:44 | |
| paulsher1ood | Kinnison: my plan is to leave the files for invesigation. but then on re-running ybd, it should clean first | 22:03 |
| * paulsher1ood reads the backscroll, and sadly fails to understand most of if | 22:06 | |
| *** wdutch has quit IRC | 22:17 | |
| *** wdutch has joined #baserock | 22:30 | |
| *** paulw has quit IRC | 23:43 | |
| *** paulw has joined #baserock | 23:43 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!