14:04:21 <ltu> #startmeeting Monthly Team Meeting
14:04:21 <GNOMie> Meeting started Tue Dec 12 14:04:21 2017 UTC.  The chair is ltu. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:04:21 <GNOMie> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:04:37 <ltu> please announce your presence
14:04:44 <ssam2> hi
14:04:46 <juergbi> o/
14:04:48 <jonathanmaw> hello
14:04:53 <tlater> o/
14:04:53 * sstriker mumbles about a dodgy irc client
14:04:53 <angelos> present o/
14:04:57 <dplayle> Hello
14:05:04 <sstriker> Sander Striker - present o/
14:05:11 <gokcennurlu> hello
14:05:28 <AntoineWacheux> Antoine Wacheux - present o/
14:05:58 <ltu> alright, i sent out an agenda yesterday, let's start by going through the actions from last time
14:06:06 <ltu> #topic Actions from last meeting
14:06:08 * ssam2 hides
14:06:27 <ltu> ssam2 Work out new proposal for GPG signing
14:06:53 <ssam2> still not done, i guess I should make some time for that next
14:07:11 <ssam2> i think the worry was that we wanted to know about changes to the internal artifact format before 1.0
14:07:19 <ltu> I think it's fair to say that, once Multiple Cache Support is landed, from your point of view the next priority is to look at performance issues?
14:07:44 <ssam2> i don't really mind; but we do seem to have performance issues
14:07:59 <ssam2> both are quite important really
14:08:10 <ltu> I know angelos is looking at perfromance issues also atm.
14:08:21 <sstriker> +1 on focus on performance
14:08:30 <angelos> +1 perf
14:08:39 <sstriker> I think it's clear from a user experience perspective that it needs TLC
14:08:42 <ssam2> ok. so let's assume GPG signing can be done post 1.0
14:08:48 <ltu> Last meeting we said GPG signing is something that needs to land early in the next cycle
14:09:51 <ssam2> well, i was already assuming that; but let's also assume we can do it for 1.2, not 2.0. since we haven't claimed that the internal artifact storage format is a point of public API in the end, I think that's fine
14:10:04 <juergbi> we'll have a pretty long cycle. there should be sufficient time to land it
14:10:15 <ltu> indeed
14:10:39 <ltu> cool, so i think we should carry the action over so as not to lose sight of it
14:11:02 <ltu> I think we'll discuss performance more in the next topic so can cover that there though
14:11:25 <ltu> #action ssam2 Work out new proposal for GPG signing
14:12:05 <ltu> #info GPG signing to be carried forward and land in the next cycle
14:12:19 <ltu> ok next up was one on tristan
14:12:29 <ltu> tristan Update HACKING.rst with feature acceptance policy
14:13:26 <juergbi> the file hasn't been updated yet
14:13:33 <ltu> #info tristan Update HACKING.rst with feature acceptance policy: not done yet
14:13:44 <ltu> ok so action is carried over again
14:13:50 <ltu> #action tristan Update HACKING.rst with feature acceptance policy
14:13:58 <ltu> ok so onto the next topic...
14:14:12 <ltu> #topic Key Features currently in flight
14:14:33 <ltu> I listed first: Build-compile cycle time
14:15:28 <ltu> Which is kind of a catch all term for 'faster builds', I think
14:15:37 <ssam2> yeah
14:15:45 <juergbi> yes, one part is the WIP incremental build support !126
14:16:10 <juergbi> i'm looking into improvements in element state handling to help here as well
14:16:48 <ssam2> angelos, are you just focusing on the performance of staging dependencies ? or did you do some broader profiling work ?
14:17:15 <angelos> We started some broader work, it's not ready for general release though
14:17:27 <ssam2> ok; is it something we could collaborate on ?
14:17:48 <angelos> Quite possibly, as soon as we have a first version to go with
14:17:51 <ssam2> i have some ideas on places to optimize, but am also interested in being a bit more rigorous
14:18:02 <ssam2> ok, great
14:18:09 <ssam2> i am interested to try it out :-)
14:18:45 <angelos> You can get a rough idea of where we're going here: https://gitlab.com/BuildStream/buildstream/merge_requests/136
14:19:04 <ssam2> aha
14:19:25 <dplayle> We also have !176 for review, as part of the larger !126 objective
14:20:20 <juergbi> yes, will review this as soon as possible. not sure whether we want to land this before 1.0, though
14:21:50 <ltu> #info angelos is currently looking, in a broad sense, at performance, see https://gitlab.com/BuildStream/buildstream/merge_requests/136 for the general idea. ssam2 to collaborate once this is ready for general release
14:21:59 <sstriker> Maybe a controversial question, but should we branch?
14:22:13 <sstriker> So that the blockers for 1.0 can be taken care of independently?
14:22:32 <angelos> (@ssam2, for more background on that branch, there's this on the mailing list: https://mail.gnome.org/archives/buildstream-list/2017-November/msg00001.html)
14:22:47 <ssam2> thanks, i remember that one :-)
14:23:07 <juergbi> tristan didn't want to branch before the release. he is planning to roll the release very soon
14:23:36 <ssam2> yes, i think we have no more real blockers
14:24:05 <sstriker> Ok.
14:24:06 <ssam2> although if we don't get 1.0 fairly soon, i guess we will have to branch. but tristan is back next week i think
14:25:02 <ltu> #info improvements in element state handling and WIP incremental build support !126 are contributing towards a quicker build-compile cycle time
14:25:23 <ltu> #info we'll not branch before 1.0, which will be soon as there are no real blockers now
14:25:40 <ltu> ok, ready to move onto the next topic?
14:25:40 <ltu> I added these topics in order to 'open the floor' to the discussion, to see if anyone has any vital points to raise on each topic
14:25:45 <sstriker> ltu: s/Build-Compile/Edit-Compile/
14:26:16 <ltu> sstriker, thanks
14:26:40 <ltu> #topic Build orchestration and distribution
14:27:07 <ltu> again, another term which captures a few different features / enhancements
14:27:48 <ltu> distributed builds is something we've not yet started designing properly
14:28:07 <sstriker> Right.  In the current design we have shared caches, but not distribution
14:28:38 <sstriker> We've talked about adding a "plugin" API to interact with the scheduler, but we haven't gotten to a design yet
14:29:16 <juergbi> we talked about exporting the pipeline in a machine-readable format
14:29:24 <sstriker> I think it would be good at this point to also have a look at the Bazel Remote Execution API (https://docs.google.com/document/d/1AaGk7fOPByEvpAbqeXIyE8HX_A3_axxNnvroblTZ_6s)
14:29:26 <juergbi> such that external orchestration may be used
14:30:33 <ltu> did we say that the plugin will have to work with a specific scheduler, or be adaptable to be used with any scheduler?
14:31:25 <ssam2> it should really work with multiple schedulers, i don't think there's a "golden standard" here that everyone uses
14:31:45 <sstriker> I think that point was to only have one scheduler within buildstream; this is what juergbi was referring to with exporting the pipeline
14:32:06 <sstriker> At least that is what I used the scheduler term for in this context
14:32:16 <ssam2> hmm, yes too many things called scheduler
14:32:26 <ssam2> my understanding was we want to make buildstream integrate with an "external scheduler"
14:32:57 <sstriker> Let's phrase that as letting the buildstream scheduler interface with remote execution? :)
14:33:16 <ssam2> ok
14:33:22 <juergbi> i expect the actual scheduling be done from outside buildstream, though
14:33:49 <juergbi> i.e., buildstream simply exports sufficient information for an external scheduler to work
14:34:08 <sstriker> Let's avoid the term scheduler for a minute
14:34:37 <sstriker> Maybe have the execution of the pipeline support remote execution?
14:34:42 <sstriker> Is that a clearer description?
14:35:08 <juergbi> to me this sounds like buildstream would control remote execution
14:35:27 <juergbi> however, my understanding was that something external would control execution based on information provided by buildstream
14:37:01 <sstriker> Right now we execute locally directly, right?  Instead of doing that there would be a "plugin API" that we would call instead.  The standard implementation would do the local building as today.  Another plugin could remote execute, queue, etc, and signal on completion.
14:37:35 <juergbi> right now we execute locally directly
14:37:42 <juergbi> however, the plugin approach is not how i remember it
14:37:52 <juergbi> ssam2, any input here?
14:38:12 <ssam2> i'm a bit confused
14:38:16 <angelos> Like @sstriker I was also thinking that BuildStream would control remote execution; the idea being that we could work towards accellerating 'bst build' invocations when connected to the network, and fall back to local when offline.
14:38:36 <ssam2> ok
14:39:07 <ssam2> i feel like we might need to discuss this in more detail to come up with a better plan
14:39:27 <ssam2> not that this is a bad plan, just that it's not what i understood the plan to be :-)
14:39:36 <juergbi> yes, probably best with tristan
14:39:45 <ltu> yeah, we should perhaps take this outside of the meeting, to the mailing list
14:40:18 <sstriker> Instead of plugin, I think we used terms like events and potentially callbacks.  In any case, it's probably best for another forum at this stage
14:41:15 <juergbi> event plugins are in progress, however, i wasn't expecting them to be used for remote execution of builds
14:41:26 <juergbi> yes, let's discuss this outside this meeting
14:41:41 <sstriker> event plugins in the current form are just notifications
14:42:01 <juergbi> exactly
14:42:41 <ltu> #info for distributed builds, a design / implementation plan should be discussed on the mailing list
14:42:55 <ltu> again, it's something to land in the next cycle
14:43:17 <ltu> does anyone have a strong atachment to the topic, and want to take an action to begin the discussion on the ML?
14:44:32 <sstriker> I'll volunteer, but don't promise a long post outlining all views at this point
14:45:51 * paulsherwood hates distributed builds... we did several implementations, it wasn't worth the complexity... i hope this is a different topic :)
14:48:26 <ssam2> i think there may be a Codethink IRC glitch occurring
14:52:03 <ssam2> it seems just to just be affecting laurence now
14:52:20 <ssam2> ok, well
14:52:22 * tlater was kicked off his bouncer
14:52:33 <ssam2> #action sstriker Send outline mail on distributed builds to buildstream-list
14:52:43 <ssam2> i think that was the last thing on the agenda
14:52:52 <ssam2> does anyone have other stuff to raise now ?
14:53:10 <sstriker> I'm good
14:53:34 <angelos> Minor thing
14:53:51 <angelos> I have an MR I'm considering openning, which would stay open for quite a while
14:54:23 <angelos> It's to make debugging plugins easier - having in-process jobs
14:54:29 <angelos> so it's easy to use pdb etc.
14:54:51 <angelos> Should I raise a WIP MR, or keep it to myself? :)
14:55:11 <jonathanmaw> angelos: I think a WIP MR would be helpful
14:55:14 <ssam2> it's definitely a pain that normal debugging tools don't work
14:55:17 <juergbi> yes, i don't see any issues with long-standing WIP MR
14:55:42 <angelos> great, will do, thanks. It's super-hacky, maybe someone can finish it off properly.
14:56:17 <ssam2> ok, anything else ?
14:56:28 <angelos> Nothing from me
14:56:43 <juergbi> nothing from me either
14:58:14 <ssam2> laurence is about to rejoin, seems to have had some big connectivity issue
15:02:20 <ssam2> ok, next meeting is Tuesday 9th January, 14:00 UTC
15:02:25 <ssam2> we will reconvene then :-)
15:02:30 <ssam2> #endmeeting
15:02:40 <ssam2> (probably need laurence back to actually do that)
15:03:10 <juergbi> yes, i think we do
15:21:56 <ltu> hi all, sorry about that, i lost connection just after discussing the write up to the mailing list
15:22:05 <ltu> #action sstriker to post an initial write up of how to approach distributed builds to the mailing list
15:22:22 <ltu> I'll get the scrollback I've missed and add it to the meeting summary which I'll email out
15:23:00 <ltu> Assuming there is nothing left to discuss for today, I'll end the meeting
15:23:40 <ltu> #endmeeting