IRC logs for #baserock for Tuesday, 2015-09-01

WalkerdineI got a SSD drive for my jetson today so I'll finally be able to build my own image00:37
*** zoli__ has joined #baserock00:54
*** zoli__ has quit IRC00:58
*** zoli__ has joined #baserock04:52
*** petefoth has quit IRC05:10
*** zoli__ has quit IRC05:40
*** zoli__ has joined #baserock05:59
*** petefoth has joined #baserock06:24
*** zoli__ has quit IRC06:50
paulsherwoodWalkerdine: i like your optimism :)07:12
*** CTtpollard has joined #baserock07:31
*** petefoth has quit IRC07:33
*** petefoth has joined #baserock07:35
*** zoli__ has joined #baserock07:41
*** rdale has quit IRC07:47
*** tiagogomes_ has joined #baserock08:01
*** mdunford has joined #baserock08:03
*** edcragg has joined #baserock08:04
*** Kinnison has joined #baserock08:19
*** franred has joined #baserock08:22
*** jonathanmaw_ has joined #baserock08:25
*** rdale has joined #baserock08:29
*** toscalix_ has joined #baserock08:36
wdutchKinnison: I know you were not working but I don't suppose you learned anything useful this weekend regarding LAVA?08:41
KinnisonSadly I didn't learn anything significant other than that one of the primary LAVA developers very much likes Milton Pegasus08:41
wdutchhehe08:41
* paulsherwood learned that there is a new contender... https://github.com/qca/boardfarm08:41
* wdutch will continue researching buildbot for now08:42
KinnisonHmm, boardfarm looks fairly lightweight08:52
* paulsherwood prefers lightweight, until heavy is proved necessary08:53
KinnisonA brief scan through boardfarm's code leaves me worried that it's really quite brittle08:58
paulsherwooddefine brittle?08:58
KinnisonA lot of its core behaviour appears to be 'expect' driven which isn't the most flexible of behaviours08:59
KinnisonThe networking setups it does seem to be hard-coded into its source08:59
Kinnisonrather than as configuration08:59
paulsherwoodeasy enough to fix the latter? and maybe the former is a benefit. as they say, tests should be simples https://github.com/qca/boardfarm/blob/master/README.md#general-guidelines-for-automated-tests09:00
KinnisonI'm fine for tests to be expect driven09:01
Kinnisonit's the way it batters around with shells etc in that fashion which upsets me09:01
paulsherwoodi'm a little disappointed it's a four-commit dump09:02
KinnisonIt's very clearly something which the author uses and has put little effort into making it nice for others to use09:02
toscalix_Kinnison, agree09:03
paulsherwoodKinnison: actually that's incorrect. from my understanding it's widely used in qualcomm - their board farm is spread across many locations09:03
paulsherwoodand iiuc there are many 'users'09:03
KinnisonSo the 'author' is Qualcomm in their internal setup09:03
paulsherwoodcorrect09:04
KinnisonI think you misunderstand me09:04
paulsherwoodok09:04
KinnisonIt's something the author clearly has put little effort into making it nice for people outside their particular environment to use09:04
Kinnisonif that helps09:04
paulsherwoodunderstood, now.09:04
KinnisonThis is a very *very* common failing in software which gets "opened up"09:05
paulsherwoodack09:05
KinnisonNot at all surprising09:05
* paulsherwood hasn't had time to give it a spin09:05
paulsherwoodi got ybd building on scaleways at the weekend :)09:05
paulsherwoodand i got it publishing its artifacts09:05
paulsherwoodbut this exposed some issues :)09:06
*** wschaller has joined #baserock09:08
paulsherwoodso https://github.com/devcurmudgeon/ybd/blob/master/kbas.py is atomic, applying your directory switch idea Kinnison09:09
paulsherwoodbut i didn't get time to try it with cherrypy or gevent09:09
paulsherwood(to make it non-blocking)09:09
*** nowster has quit IRC09:47
*** wschaller has quit IRC09:49
*** wschaller has joined #baserock09:55
*** nowster has joined #baserock10:01
*** straycat_ has left #baserock10:17
*** edcragg has quit IRC11:04
*** edcragg has joined #baserock11:05
*** wschaller has quit IRC11:24
radiofreepaulsherwood: i need to check through the logs again11:37
radiofreehowever i'm pretty sure both the jetson + mustang generated the same cache key (correct), but their reporting of the cache key didn't match reality11:37
radiofreei'll check next time i have the chance11:37
paulsherwoodweird, radiofree. but that log you sent me was consistent11:38
*** nowster has quit IRC11:39
*** nowster has joined #baserock12:03
*** wschaller has joined #baserock12:15
*** nowster has quit IRC12:35
*** nowster has joined #baserock12:39
*** petefoth_ has joined #baserock12:58
*** petefoth has quit IRC13:00
*** petefoth_ is now known as petefoth13:00
wdutchI've made an, admittedly awful, flowchart of how I think the main orchestration of CIAT could behave, I'd appreciate it if somebody could have a look in case I've got the wrong end of the stick. Flowchart: http://i.imgur.com/LiDoKbO.png13:07
wdutchrichard_maw pdar Kinnison ^13:07
* paulsherwood notices he's not on wdutch's list, and scowls :)13:08
wdutchI did not want to bother you13:08
paulsherwoodwhy? lots of folk enjoy bothering me... :)13:08
Kinnisonwdutch: that's not quite how I see it13:09
Kinnisonwdutch: pretty much everything before "start firehose" is what firehose is13:09
* SotK wonders what "start firehose" refers to in that chart. My understanding of the current state of things was that "firehose" was the bit doing the watching and forking.13:09
richard_mawsnap!13:09
SotKheh13:09
wdutchah okay13:10
wdutchso instead I should be looking for new artifacts appearing?13:10
richard_mawwdutch is not wrong in that it's the flow involved, but we've got pre-existing tooling that we're fitting together, so the boundaries of what we want to do differ13:10
Kinnisonaye13:10
richard_mawwdutch: that would be one way to do it, I was assuming we'd have some tool ping the test runner to run tests rather than polling on new artifacts though13:11
richard_mawwdutch: as you don't need to worry about atomically having a set of artifacts available then13:12
* SotK would expect something like "start build ---> if successful, start tests ---> if successful, do something (eg vote on a change)!"13:13
* SotK butts out13:13
wdutchSotK: you make a valid point13:13
*** petefoth has quit IRC13:15
wdutchrichard_maw: I don't understand your point on not worrying about atomicness13:15
paulsherwoodatomicity :)13:15
* Kinnison drew https://awwapp.com/b/uyoktn1vx/ but that possibly doesn't help13:15
*** petefoth has joined #baserock13:15
wdutchKinnison: it looks like something from /r/dinosaurdrawings :P13:16
KinnisonYou honour me13:16
richard_mawwdutch: unless you only ever have one artifact produced per build, then if you trigger further testing on the presence of an artifact, you might start a test before every artifact it needs has been uploaded13:17
wdutchokay13:17
Kinnisonpretty much every operation is a many->many activity13:17
richard_mawwdutch: if you have the service that uploads the artifacts ping the follow up service that it should do something with the new artifacts, then they have all been uploaded before it tries to pick things up13:17
Kinnisonmany inputs -> activity -> many outputs13:18
wdutchrichard_maw: so another component holding the artifacts with a list of what artifacts are needed for a given test which then triggers orchestration?13:19
KinnisonI think that's likely overcomplicating things for now13:20
KinnisonI'd simply have the orchestration watch the artifact cache and when artifacts matching certain patterns turn up, trigger certain activities13:21
paulsherwood+113:21
Kinnisone.g. when an artifact matching genivi-demo-platform-test-*.img turns up, trigger the genivi-demo-platform-tests suite13:21
wdutchthat just sounds like doing the same thing but giving orchestration an extra job13:21
richard_mawhrm, defined order of upload, and having a different set of trigger files to that of those needed to perform a test would satisfy atomicity13:21
Kinnisonorchestration's one and only job is to take input data and trigger behaviours13:22
wdutchokay13:22
KinnisonOtherwise we may as well glue each component directly to the next component in the pipeline and forget about orchestrating anything13:22
Kinnisonwhich, sadly, will open us up to a lot of pain wrt. scaling13:22
wdutchand fungibility13:23
Kinnisonindeed13:23
nowsterwdutch: how mushroom does it need?13:27
paulsherwoodrichard_maw: can you expand on your defined order idea?13:27
* paulsherwood doesn't understand it yet13:27
* Kinnison reads it as "if a test has multiple inputs then a defined order in which those inputs become available would mean the activity could trigger from the last of those inputs, rather than needing to express some kind of complex dependency"13:28
richard_maw^ that13:28
paulsherwoodok13:31
*** wschaller has quit IRC13:32
paulsherwoodcan someone provide an example of a test requiring multiple inputs?13:37
wdutchwhy does it need an ordering rather than being a set?13:39
richard_mawpaulsherwood: testing that a specific appliance works when run as a container in a specified system, or a multi-system communications test13:46
paulsherwoodrichard_maw: thanks13:47
richard_mawwdutch: it could be a set of files to watch for if you want to put the complication into the polling logic, rather than the upload logic13:47
paulsherwoodrichard_maw: for those cases, wouldn't you want to test any new foo with existing versions of other bits, by default?13:48
paulsherwoodie grab new appliance, test with existing container13:49
paulsherwoodand same for multi-system, unless flagday13:50
richard_mawit's possibly too early to speculate about that, though I'm nervy about the tests statefully defining the "latest" system13:50
paulsherwood'latest' is not special. any existing13:51
paulsherwoodi didn't say latest13:51
wdutchrichard_maw: I don't see how any way other than a set could be sensible, what happens if two artifacts are built in paralell and come out in the wrong order?13:54
SotKwdutch: then the one that should be uploaded second but finished first waits until the one that should be uploaded first finishes I guess13:55
richard_mawpaulsherwood: how would you select it from the existing systems then13:55
* SotK also prefers the idea of a set rather than relying on ordering though13:55
paulsherwoodrichard_maw: doesn't matter much, unless flag day13:56
wdutchSotK: in that case you could end up fork bombing yourself...13:56
paulsherwoodrichard_maw:  in the real world, users have jrandom versio nof system, new thing is hoped to work on all of them13:56
* SotK doesn't see how that could cause a fork bomb?13:58
wdutchyou're right, wrong term but you could end up with threads sat around forever13:59
richard_mawpaulsherwood: and if it doesn't they may opt to try a new version. I'm even more nervy of selecting something at random than the "latest", since that makes it impossible to make it behave deterministically14:00
radiofreejetson build time for devel system (not including download) 15-08-31 19:26:10 [282/282/282] [TOTAL] Elapsed time 07:03:1514:00
SotKwdutch: surely you'd just kill them along with the thing they depend on if it doesn't finish?14:01
paulsherwoodrichard_maw: finding problems is the purpose of this solution14:01
*** jonathanmaw__ has joined #baserock14:01
paulsherwoodartificially restricting things will hide issues14:01
wdutchSotK: how do you know when to do that?14:01
SotKsame way you do if something hangs and you're uploading stuff in a random order I guess14:02
*** jonathanmaw___ has joined #baserock14:04
*** jonathanmaw_ has quit IRC14:04
paulsherwoodrichard_maw: note i'm not saying test new-foo against all existing versions, just any. then if there's a problem, git bisect back to last known good14:05
*** jonathanmaw__ has quit IRC14:06
*** jonathanmaw___ is now known as jonathanmaw_14:07
* wdutch thinks having artifacts sat around is much better than having threads hanging14:07
* SotK does too :)14:08
richard_mawpaulsherwood: I see a distinction between "wanting it to work on all versions, and compromising by testing against a possibly randomly selected sample set" and "not caring about which version it is tested with". The former is a deliberate choice, while the latter is an absence of choosing.14:10
richard_mawI'm not sure we're close enough to having something to show that this discussion is useful at the moment.14:11
* richard_maw gets back to looking at firehose14:11
*** edcragg has quit IRC14:18
wdutcham I right in thinking firehose updates definitions but doesn't trigger a build?14:28
richard_mawwdutch: I'm currently investigating that, the original design had it do candidate builds and push the branch if it "built", leaving further testing to later down the pipeline14:29
SotKI think so, I believe the idea when it was last looked at was that it would push the proposed definitions update to Gerrit where Mason would run automated tests on the change.14:30
SotKs/tests/builds and tests/14:31
*** jonathanmaw_ has quit IRC15:01
*** jonathanmaw_ has joined #baserock15:13
*** zoli__ has quit IRC15:13
richard_mawwdutch, pdar, Kinnison: Ok, so AIUI the latest version of firehose requires a gerrit rather than a regular git repository. If we're going to put together this CIAT prototype using a pre-gerrit Trove, then we need to use a version that's rolled back to before that, and define how the test-builder knows what branches it should test.15:29
richard_mawWe can put a hook in the trove to have it ping a service, or the service can poll the definitions repository periodically.15:31
* richard_maw double checks how firehose is started15:32
KinnisonI think the trove should ping an orchestration core (hence wdutch looking at buildbot)15:32
Kinnisonfirehose as was when I left it was a program to run, not a daemon or service15:32
richard_mawhm, and later versions pretty much just run on a timer15:34
paulsherwoodcan we use http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/infrastructure.git/tree/systems/gerrit-system-x86_64.morph15:35
paulsherwoodand have gerrit as part of the soln?15:35
KinnisonAt some point, yes.  But not for now15:35
paulsherwoodwhy go back in time, when fh + gerrit + trove has already been 'done' for a certain value of 'done'?15:38
KinnisonLimited time to understand the pipeline15:38
paulsherwoodi'm ok with it, just interested15:38
richard_mawAlso, AIUI short-term we're not interested in merging in the candidate branches, which was the purpose of the Gerrit integration. If we were to use the Gerrit'd form, then we'd have to deal with the added complication of fetching the candidate branches from the gerrit, rather than the Trove.15:45
KinnisonIt would certainly complicate matters15:45
paulsherwoodack15:46
richard_mawok, so the orchestrator doesn't need to do anything interesting with firehose, as it can just wait for the process to terminate and pass the resultant candidate branch on to the builder15:51
Kinnisonthat's a likely approach yes15:53
richard_mawright, so we need to decide whether the orchestrator cares that it was triggered for one branch and resulted in candidate branches from other branches that also happen to have changed, or whether we need to add a mode to firehose to make it only process branches specified on the command-line16:00
*** mdunford has quit IRC16:19
*** franred has quit IRC16:26
*** rdale has quit IRC16:29
*** jonathanmaw_ has quit IRC16:32
*** mdunford has joined #baserock16:35
*** mdunford has quit IRC17:18
*** toscalix_ has quit IRC18:08
WalkerdineAlright now its time to start working on building my own image for the jetson. Turns out I didnt have time to yesterday18:24
*** petefoth has quit IRC19:34
*** petefoth has joined #baserock19:35
*** petefoth has quit IRC19:47
*** paulsherwood has quit IRC19:50
*** pdar has quit IRC19:50
*** pdar has joined #baserock19:50
*** paulsherwood has joined #baserock19:51
WalkerdineIs there an easy way to clone the whole source of the genivi demo platform and to put it on a github19:52
*** petefoth has joined #baserock19:52
WalkerdineDoes the cycle.sh script only work on x8620:34
Walkerdinepaulsherwood: do you happen to be around?20:37
SotKWalkerdine: I think cycle.sh should work on ARM too20:52
SotKyou'll have to give it `clusters/jetson-upgrade.morph` as the cluster on a Jetson though20:53
WalkerdineI was wondering how to get off of the genivi demo platform once it boots into it21:00
WalkerdineI'm hoping that is the case21:00
WalkerdineI need another monitor21:00
*** Walkerdine__ has joined #baserock22:08
*** Walkerdine__ has quit IRC23:00

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!