IRC logs for #baserock for Wednesday, 2015-09-02

*** zoli__ has joined #baserock06:54
*** rdale has joined #baserock07:00
*** mdunford has joined #baserock08:03
paulsherwoodWalkerdine: sorry, i was not... did you get it working?08:15 should work on jetson08:15
radiofreeWalkerdine: you can use system-version-manager to set the system back to factory08:16
radiofreesystem-verison-manager set-default factory08:16
radiofreethen reboot, or select "factory" from the boot menu (via serial)08:16
*** petefoth_ has joined #baserock08:17
*** petefoth has quit IRC08:17
*** petefoth_ is now known as petefoth08:17
radiofreeregarding system-version-manager, which version do i have to use for "--json" to work?08:18
* paulsherwood has no idea, sorry08:19
paulsherwoodsome tidyup is required here, i think08:19
radiofree seems to require it08:19
Kinnisonradiofree: it was something ripsum was working on08:19
Kinnisonradiofree: it's possible the corresponding change is still in gerrit08:20
radiofreeah, yeah, tbdiff master didn't work08:20
*** gary_perkins has quit IRC08:34
*** gary_perkins has joined #baserock08:39
*** mdunford has quit IRC08:45
*** jonathanmaw_ has joined #baserock08:53
*** mdunford has joined #baserock08:59
*** flatmush has quit IRC09:04
*** flatmush has joined #baserock09:17
*** mdunford has quit IRC09:33
*** mdunford has joined #baserock09:45
*** toscalix has joined #baserock10:32
*** mdunford has quit IRC10:40
*** fay_ has quit IRC10:42
*** mdunford has joined #baserock10:54
*** fay_ has joined #baserock10:57
paulsherwoodi've started writing some stuff about ybd on the wiki...
paulsherwoodscaleway is slow, but interesting as an elastic cloud for arm builds :)11:00
Kinnisonwhat is scaleway?11:00
paulsherwoodthere was a strange compile error using their debian + ubuntu images though11:00
paulsherwoodhence fedora is the recommended choice there11:00
KinnisonWelcome to the combinatorial explosion11:01
petefothKinnison: accordign to their web site 'Scaleway is the first IaaS provider worldwide to offer an ARM based cloud'11:01
paulsherwooda load of baserock slabs in the cloud, basically :)11:01
paulsherwoodwell, same cpu at least11:01
paulsherwoodtheir own design11:02
Kinnisonpetefoth: well that's ultra-informative¬°11:02
paulsherwoodyou get baremetal 4 core 2GB build machine11:02
Kinnisonyeah that's not going to be the quickest of build farms11:03
Kinnisonbut handy11:03
paulsherwoodeasy to use, with an api11:03
radiofreethe mustang on my desk could be used to provide arm artifacts?11:03
paulsherwoodKinnison: i've not had chance to try splitting workloads across multiple machines there yet, but i expect it to be 'interesting'11:04
radiofreeit builds a devel image in 4 1/2 hours compared to our current fastest option, the jetson, which takes 7 hours11:04
rjekISTR that the storage on Scaleway is Network Block Device.11:04
rjekSo it's not going to be fast.11:04
* Kinnison lunches11:04
paulsherwoodand i keep thinking abot to previous discussions with radiofree and others about distcc11:04
paulsherwoodrjek: correct. but SSD11:04
rjekpaulsherwood: The SSD doesn't really matter when you involve the latency of TCP11:04
rjekSSDs are fast because they don't have a seek time.11:05
radiofreecould we use a moonshot node for building armv7 images?11:06
radiofreethe steps to do it would be exactly the same as the mustang11:06
paulsherwoodradiofree: yes, we could11:10
paulsherwoodi haven't got round to timing anything on moonshot11:11
paulsherwoodradiofree: thanks for your log, i'll add it to the collection :)11:11
radiofreei have one from a jetson if you want that11:13
paulsherwoodyes please. i couldn't connect to jetson last time i tried11:13
paulsherwoods/jetson/my jetson/11:13
paulsherwoodi've added them to
* paulsherwood wants to do some comparative graphs11:40
*** fay_ has quit IRC12:24
*** fay_ has joined #baserock13:38
paulsherwoodKinnison: not sure if you noticed the ybd artifact publication discussion13:50
Kinnisonpaulsherwood: I noticed you mention you'd written a basic artifact service13:50
Kinnisonpaulsherwood: but that you'd not tested any kind of multithreaded behaviour13:50
paulsherwoodit's an interesting possibility that any ybd machine could permanently or temporarily become an artifact server13:50
paulsherwoodKinnison: i did test multithreaded, and proved it doesn't fully work yet :)13:51
KinnisonI think bottle has a simple multithreaded server support13:51
Kinnisonor possibly multiprocess13:51
paulsherwoodbut my point is that there would be advantages to this concept... eg radiofree could do a build, and then offer artifacts from his desktop to be collected by interested server(s)13:52
KinnisonWithin a security domain, sure13:52
Kinnisonbeyond that, I have reservations about safety13:52
paulsherwoodKinnison: bottle is multithreader but blocking13:52
Kinnisondisk IO is likely blocking13:52
paulsherwoodit can be run with gevent or cherrypy to be non-blocking (config flag)13:53
Kinnisonmultiprocess will help with that13:53
paulsherwoodKinnison: my current design is that the ybd server (kbas) needs to be explicitly run. it has a default password. it won't accept uploads unless the user has specified a different password from the default in its config13:54
paulsherwoodis that still too unsafe?13:54
Kinnisonplaintext passwords don't interest me13:54
paulsherwoodack. i need to sort https13:55
paulsherwoodor we could require keys i guess?13:55
KinnisonMy gut feeling is either HTTPS with certificates, or ssh of some kind13:56
*** zoli__ has quit IRC14:00
paulsherwoodKinnison: i want something as trivial as possible, for jrandom ybd instance if at all possible14:00
paulsherwoods/trivial/trivial to setup/14:00
Kinnisonhttps and shared secrets is likely easiest then14:01
* Kinnison stares somewhat blankly at an AWS console14:01
Kinnisonwow they have a lot of features14:01
paulsherwoodclick on EBS topleft14:02
Kinnisonwhich AMI did you use for your ybd testing? I'd like to replicate that as closely as possible14:03
paulsherwoodsorry EC214:03
Kinnison"Amazon Linux AMI 2015.03.1 (PV) " ?14:04
Kinnisonor "Amazon Linux AMI 2015.03.1 (HVM), SSD Volume Type" ?14:04
paulsherwoodm4.10xlarge with the first option Amazon AMI14:04
Kinnisonthat's the HVM one14:05
paulsherwoodi can find the history of my commands if you like14:05
paulsherwoodAmazon Linux AMI 2015.03.0 x86_64 HVM GP214:05
Kinnisondid you slap a big EBS on yours for building?14:06
Kinnisonor just increase the root size?14:06
Kinnison(it appears to be 8G by default on the m4.10xlarge)14:06
paulsherwoodi added a separate volume 100GB14:07
* Kinnison now just needs to wait for his account verification to complete before he can launch14:10
paulsherwoodKinnison: lots of noise in this, but it's everything i've ever done on that machine
paulsherwood(after sudo su)14:11
toscalixKinnison, Paul, what code should I put to the project, CT201?14:20
toscalixpaulsherwood, sorry14:21
* Kinnison assumes toscalix is aware he's leaking internals?14:21
wdutchfor CIAT, will definitions be in trove? I'm not sure what trove is and isn't14:21
Kinnisonwdutch: yes, it will14:21
wdutchKinnison: ta14:21
Kinnisonwdutch: I currently have my ops team sorting out a trove for us to play wish, so we don't have to disturb for our experimentation14:21
wdutchtoscalix: this is the diagram of orchestration, Kinnison paulsherwood and pdar might want to sanity check it14:36
toscalixwdutch, thanks. I will add it to the wiki.14:38
Kinnisonwdutch: poll for changes //-> triggered on changes14:38
Kinnisonwdutch: we can make trove ping the orchestration framework14:38
KinnisonBut that's essentially reasonable14:38
wdutchI don't understand what you mean by //->14:38
Kinnisonwdutch: orchestration can both poll and be triggered wrt. git changes14:39
wdutchthe buildbot way of doing things afaict is for buildbot to do what it calls polling and refers to as 'magically knowing' when the repo changes14:40
Kinnisonso long as it supports being externally triggered14:40
Kinnisonbecause having it poll what will eventually become 1000s of repos would be a bad thing and might influence us away from buildbot14:40
wdutchI was under the impression firehose would deal with the 1000s of repos and then update definitions, orchestration would be triggered by this change in definitions14:41
KinnisonSomething needs to trigger firehose14:41
KinnisonI'm kinda hoping we can make that all the same orchestration framework14:42
Kinnisonto make life easier14:42
* wdutch will have to rethink then14:43
wdutchI thought firehose stoodalone in that respect14:43
Kinnisonnope, it needs (currently) to be caused to run14:43
Kinnisoneither on a timer, or by triggers14:43
pdarShould the orchestration set up the aws machine for building things. In teh diagram that would meant the builder was the aws machine?14:49
KinnisonDon't think about "the aws machine"14:50
Kinnisonremember we're aiming for elastic behaviour14:50
Kinnisonso there'll be "some number of machines"14:50
wdutchI suppose the builders and testers will need to change size elastically, trove, the fileserver and orchestration not so much14:53
pdarSo, to me, will builder just be some scripts that do stuff somewhere then?14:57
wdutchthat describes the whole thing :P14:58
*** Walkerdine_ has joined #baserock15:06
Walkerdine_Apparently I'm in the wrong timezone for this irc15:10
wdutchI didn't know that was a thing15:11
Walkerdine_Just to be able to reach people when they are around15:12
Walkerdine_Do I need to do anything special to use a sata disk with my jetson15:14
Walkerdine_I flashed the baseline image onto my jetson and I have a sata drive hooked up but I want to work off of the SSD cause its faster15:15
radiofreei use the ssd as my /15:16
radiofreeyou'll have to manually partition the ssd for now, but when you deploy change "ROOT_DEVICE" to /dev/sdax instead of /dev/mmblk0p215:17
radiofreeBOOT_DEVICE should probably always be /dev/mmblk0p115:18
paulsherwoodWalkerdine_: most folks here are on eurotime, most of the time15:18
paulsherwoodbut there are exceptions15:18
paulsherwoodhow are you getting on, Walkerdine_ ?15:18
Walkerdine_Yeah I'm on Eastern time15:21
Walkerdine_I'm honestly not sure how to go about doing that15:22
Walkerdine_paulsherwood: I'm sorry I'm not sure if I understand your question15:24
paulsherwoodWalkerdine_: i'm mainly worried that this is turning into a 'slowstart' for you rather than a quick start :)15:25
Walkerdine_its mostly my fault because I'm rather new to linux15:25
paulsherwoodWalkerdine_: well, hopefully we can help... we were all new at some point :)15:26
paulsherwoodradiofree: do we have any simpler instructions for configuring storage for jetsons?15:27
Walkerdine_I'm sure once I get the change build repeat cycle going I'll be able to do everything on my own15:27
paulsherwoodi'm sure of that too :-)15:27
radiofreepaulsherwood: i do not15:28
radiofreebut that's the type of stuff you'd do outside of baserock anyway15:28
paulsherwoodWalkerdine_: how big is your ssd?15:29
*** fay_ has quit IRC15:51
Walkerdine_Is manually partitioning the SSD sort of the same process of how the eMMC was partitioned during the flash?16:03
*** fay_ has joined #baserock16:11
*** jonathanmaw_ has quit IRC16:33
*** petefoth has quit IRC16:38
*** pdar has quit IRC16:59
*** bwh has quit IRC17:00
*** fay_ has quit IRC17:00
*** SotK has quit IRC17:01
*** kejiahu has quit IRC17:01
*** perryl has quit IRC17:01
*** petefotheringham has quit IRC17:01
*** benbrown_ has quit IRC17:01
*** jmacs has quit IRC17:01
*** paulsherwood has quit IRC17:01
*** zoli__ has joined #baserock17:01
*** Zara has quit IRC17:01
*** sebh has quit IRC17:01
*** bwh has joined #baserock17:03
*** petefotheringham has joined #baserock17:03
*** sebh has joined #baserock17:03
*** gary_perkins has quit IRC17:03
*** gary_perkins has joined #baserock17:04
*** zoli__ has quit IRC17:05
*** zoli__ has joined #baserock17:09
*** flatmush has quit IRC17:10
*** jmacs has joined #baserock17:13
*** petefoth has joined #baserock17:50
*** mdunford has quit IRC17:51
*** toscalix has quit IRC18:08
*** lachlan75 has joined #baserock18:14
*** lachlan75 has quit IRC18:33
*** SotK has joined #baserock19:10
*** Walkerdine_ has quit IRC19:41
*** paulsherwood has joined #baserock19:45
paulsherwoodWalkerdine: sorry i missed your question, i got dumped off irc19:54
paulsherwoodyes, i believe it's the same process19:54
paulsherwoodbasically you're aiming to treat the SSD as your ROOT_DEVICE, while booting from the kernel you've flashed to the eMMC19:57
paulsherwood may or may not be clear enough, it's been a long time since i followed those instructions19:59
*** zoli__ has quit IRC21:23
Walkerdinepaulsherwood: So when I deploy an upgrade using the does it boot temp from the SSD or the eMMC?23:11
WalkerdineI'm probably too late for anyone to respond23:14
radiofreeWalkerdine: u-boot will look for things in the emmc23:15
radiofreeso you *always* want to set BOOT_DEVICE to /dev/mmcblk0p123:16
radiofreehowever, ROOT_DEVICE in that cluster can be anything23:16
radiofreeso if you've partitioned your SSD to have... let's say a 50G sda1 partition for / and RESTOFSPACE for /src, you set ROOT_DEVICE to /dev/sda123:16
radiofreethe flashing script you used to setup the baserock instance will create /dev/mcblk0p1, so you never have to worry about that once it's flashed23:17
WalkerdineThen I create my workspace in /src and23:17
radiofreeit's usually better to create /src on a different partition, since / is supposed to be a btrfs one23:18
radiofreeand /src can be used between upgrades23:18
radiofreethe layout i use is a 50G / btrfs partition on sda123:18
radiofreeand RESTOFSPACE sda2 for /src23:18
radiofreebut 50G is too much, i use 50G because i do a *lot* of upgrades/deployments23:19
radiofreethis could probably do with a wiki article actually..... if i have some free time tomorrow i'll write it up23:19
WalkerdineI will probably be doing alot of upgrades23:20
radiofreehow big is your ssd?23:20
radiofreeok, i'd say 20G / and 100G /src23:21
WalkerdineI've got it up and running now how should I partition it out23:21
radiofreemake sure /src is a ext4 partition though23:21
radiofreeactually now that i think about it i don't think it's that easy with this tooling :\ I only know how to do this because i wrote the jetson flashing script23:21
radiofreelet me take a look tomorrow, I'll see if i can write an "experts" guide that explains what to do23:22
WalkerdineCan I build just off of the eMMC23:22
radiofreeone of the things the flashing script does is to setup the emmc (/dev/mmcblk0*) in the correct way23:22
radiofreeWalkerdine: yes, if you used the flashing script that will be fine23:23
radiofreejust set BOOT_DEVICE to /dev/mmcblk0p1 and ROOT_DEVICE to /dev/mmcblk0p223:23
WalkerdineHow long does it usually take23:23
radiofreesadly, on a jetson these days it takes about 7 hours to build the gdp from scratch23:23
radiofreewe used to provide ARM caches of these builds, but that got lost23:24
radiofreei am actively working on restoring some infrastructure to provide that though23:24
radiofreein an ideal world it would take you ~10 minutes to deploy a new devel image or gdp image23:24
radiofreealthough if you're trying to deploy a GDP image to your jetson, then it should be pretty quick23:25
radiofreesince i uploaded the cache for that to the cache server23:25
Walkerdine7 hours!23:26
WalkerdineOh no23:26
radiofreeare you attempting to build the gdp image?23:26
radiofreeok hold on23:27
radiofreeit should be a lot quicker than 7 hours since i uploaded the cache for that23:27
radiofree~10 minutes23:27
radiofree(to 'build' it)23:27
radiofreeso add this line to /etc/morph.conf23:28
radiofreeartifact-cache-server =
radiofreethen follow the "Build it yourself" instructions on
radiofreeif the cache hasn't been deleted it should download the prebuilt cache and it'll be quick23:29
WalkerdineI gotta make sure to use the when I do build it23:29
radiofreesure, is just a wrapper around morph deploy though23:29
WalkerdineBut doesn't it make it so I can boot back into the baseline?23:30
radiofreedo you have a serial connection to the board?23:30
radiofreeif you have that you'll get a boot menu23:30
radiofreeso you can always select previous deployments when you reboot23:30
radiofreehowever, as long as you can login and get a console, you can use the `system-version-manager` command to set the default23:31
radiofreesystem-version-manager list23:31
radiofreeusually for the devel system you'd do23:31
WalkerdineWhat about when you just flash the gdp on it23:31
radiofreesystem-version-manager set-default factory23:31
radiofreethen there is no devel image, i can build a GDP + devel image though23:31
WalkerdineAh I see23:31
WalkerdineOkay that makes sense23:31
radiofreeGDP image is just... well the GDP23:31
radiofreebaserock is great, but we certainly don't explain it well :)23:32
radiofreei'll try and rectify that with the GDP at some point in the next two weeks (currently busy on other things), certainly a GDP + devel image would make sense23:33
WalkerdineI really think its just cause I'm still new to this whole thing23:37
radiofreei don't know, i personally think it should be easier, I think it would be massively beneficial (at least for me) if you would post something to the mailing list ( about your experiences with baserock23:41
radiofreeand your expectations as well23:41
WalkerdineYeah i could do that23:46
WalkerdineMaybe once I get this up and going23:46
radiofreeit would be much appreciated :)23:47
radiofreebut yes, timezones dictate i have to head off now, hopefully the cache is still there23:47
radiofreeif not, it's a bug and i'll try and fix it tomorrow23:47
WalkerdineDarn well thanks for the help23:50
Walkerdinehopefully I can get this going today23:50
radiofreeyeah, if it's taken more than a day then there's clearly something wrong with the workflow/instructions, so it would be nice if you could share/rant on the mailing list about that23:51
radiofreeso we can fix it23:51
WalkerdineThe flashing instructions can use an update too but Ill tell you guys about that in the post23:52
WalkerdineI had a few issues with the TEGRA_TOOLS_DIR that I took it out of the script23:53

Generated by 2.15.3 by Marius Gedminas - find it at!