MVP

[saintsal]

@aliyajypsy @vivek @cryptowanderer

MVPs

An MVP succeeds when:

1. it informs us about unknown unknowns, and changes our concept based on what we discover
2. we come out of it having had fun, and feeling momentum (usually with deeper relationships with the people we’re trying to serve)

It doesn’t succeed if doesn’t reveal anything we need to change. It doesn’t succeed if it’s designed as a stepping stone to prove the concept is correct, before scaling it.

So to make sure MVPs are designed for learning, a good practice is to make them to be disposable. We design them not to be built upon, but to be discarded once they’ve taught us what they need to. We’ll need to be careful about this, to catch ourselves when we’re over-building or over-automating. Good MVPs are often “Wizard Of Oz” prototypes, where we’ll be behind the curtain pulling levers.

---

Mempool risks

Here are the risks everyone shared and voted on, and they boil down to these questions:

• How will random applicants affect Kernel culture in the mempool?
• Will #defence work?
• Will Buddying Up work?
• Will people engage on the forum?
• Will Guild Leaders shift to new roles?
• Will fellows form lasting groups once they join the Fellows track?*

The MVP

Ideally, we’d run invidual experments or MVPs for each of these questions, but they’re pretty context-dependent on each other.

So instead, I think we should go with a simmered down version of the Mempool program, removing all the parts we’ve run and have confidence in.

That reduces to a 4-week program, with the main event being a weekend hack-a-thon (make-a-thing?) preceded by an unconference for workshops. Applications to this is via our Mempool system and a mini Mempool track.

We’ll aim for ~50 participants total. That’s small enough to be manageable if things go off-track (which we should expect since this is experimental) and large enough that it hits the 50-person group size for the Welcome Squad.

In more detail it looks like this (note the top 2 layers are weekly loops, and the bottom is a 4-week schedule)

* This MVP design covers all but the last of our main questions from above.

It’s easy to look at this, see its similarity to the Mempool Shape, and see how we expect it to work. But since it’s an MVP, we need to look at it with different eyes and ask, what should I pay attention to to learn where my underlying assumptions are wrong? To do that, what will you observe during this?

Observables

We’ll want to be explicit about what we’re observing in advance, so we can plan the MVP accordingly.

Here’s a starting point for things to observe, to get you thinking:

Measurables are marked with a #
Qualitative or subjective observations marked with a ~

Note: Ideally, MVPs are small enough that there acan be 2-3 measureables and 2-3 qualitative obsevables to focus on. Here there are a lot more, so to make them more manageable, we’ll break them down into groups for each week.

Will Buddying Up work?

• # team formations / week
• # of fellows attending learn track / week
• ~ catching up with fellows to see if anything interesting happened during Learn Track for them

Will #defense work?

• # fellows ing / week
• # signed up to review
• ~ checking in with reviewers who didn’t review this week

Will Guild Leaders shift to new roles?

• # Guild Leaders signing up (esp early signups and ideas they want to build)
• # Guild Leaders participating to the end
• # Guild Leaders running unconf sessions
• # Invite acceptance rate
• ~ asking them about their current goals and how their contributions to kernel fit in

Will people engage on the forum?

• # Active fellows / week
• # % of participants on forum / week
• # difference between active forum and active slack users

How will random applicants affect junto culture in the mempool?

• # fellows returning to juntos
• # joiners asking questions to stewards
• ~ visiting juntos and observing
• ~ asking fellows for feedback on new joiners
• ~ noting which junto sessions had disruptive behaviours

Will the unconf work?

• # Attendance rate
• # % of participants achieving learning goals

Will fellows from across blocks participate?

• # Attendance rate from fellows
• ~ look at learning goals & outcomes for fellows

Will welcome squads work?

• ~ follow up after each onboard
• ~ follow up on relationships 1 week after hackathon

Will Matrix work?

• # sign up rate
• # join rate
• # week-on-week active users
• # week-on-week active users in #mempool rooms

Make-a-things

Our hackathon concept was pretty light going into this, and we wanted to do a few things to set Kernel’s apart from typical hackathons:

• It’s collaborative more than competitive
• It’s more about shipping, less about pitching
• It’s about shipping anything useful, not just software

So that’s where “Make-A-Thing” comes in, instead of hack-a-thon. What’s a thon anyway and why would anyone want to hack one?

There are a few other disinctions that cascade from this:

• There are no judges or cash prizes. There are NFT prizes (glory!) How do we attract people who want to learn and connect?

• There are no sponsors. Hacakthons typically offer ways to learn about sponsors technology, so instead we’ll promote the chance to learn from particular fellows, and their accessibility in such a small event.

• Everybody makes/hacks, everybody helps. Instead of a mentor/hacker division, everybody is expected to work on a project, and everyone is expected to be available to help others. (Everyone “hacks” and everyone “mentors”.)

• The in-program Make-A-Thing is intended to be open to Fellows only, but Fellows can invite anyone from the mempool onto their teams. In the MVP version, everybody can form teams, but fellows get 1 invite to next block’s learn track.

Next steps

1. Give some thought to things we might observe to inform us (respond here today or tomorrow with your thoughts)
2. Finesse or change the MVP design accordingly. Let’s talk about this async first ideally (then in a group call later this week if needed) to make sure we’re asking and answering the right questions.
3. Operationalise this as an event: comms, event planning, wiring (just keep in mind disposability over repeatability to optimise this for learning and adaptation)

[cryptowanderer]

@saintsal really neat, thank you.

1. When you say 50 people, do you mean 50 new applicants? Are we making a distinction between “new” applicants and current fellows?
2. In both #defense and Guild Leaders you mention “signed up”. Will we not, rather, invite specific people for this MVP and so should the metric not be “accept invitation”? Likely just a semantic diff, but worth asking for clarity on ops level.
3. Who will the Welcome squad welcome (esp if the 50 above are already mostly fellows), and to what will they be welcomed (esp if we throw away what doesn’t work)? I see the whole flow fairly clearly except for this part… (I think this is also illustrated by the lack of #quant metrics for welcome squads)

I also think we can add “~ relevant and engaging topics” to the Will the uncof work metrics. Something qualitative there seems necessary, and basing it around the relevance to current research and questions that people brought in and how it then informs what they build would be good to get some grip on.

[saintsal]

@cryptowanderer said in MVP:

When you say 50 people, do you mean 50 new applicants? Are we making a distinction between “new” applicants and current fellows?

I was thinking 50 total, since that’s easily manageable. That said, we might want to aim for 100 total, or 33 applicants per week, hoping to get 50 for the hackathon part itself. I wonder if we can get a benchmark on show-up rate from similar hackathons.
Here I wasn’t making a distinction between fellows and non-fellows, but it might be worth thinking of a target ratio.

What do y’all think of these numbers?

• 10-15 teams of 3-5 seems easy for the make-a-thing part.
• 50+ is necessary to try the Welcome Squad and Unconference. The unconf could even benefit going up to 70-80.
• 10 people per week for the Research Clinic is a good target, but it could handle more people (who’d just observe)
• 10-20 / week is pretty healthy for the Intention Circle.
• 10-15 juntos per week (~half to be run as Paper Party sessions & half scheduled on convo)
• If we want to run dedicated Learn Juntos each week, 30-40 targetted but increasing as the weeks progress

@cryptowanderer said in MVP:

In both #defense and Guild Leaders you mention “signed up”. Will we not, rather, invite specific people for this MVP and so should the metric not be “accept invitation”?

Yeah, I like that better. The important thing here is just setting ourselves up to be able to measure these things - and to measure a participation rate for reviewers or any sub-group, we’ll need a denominator.

@cryptowanderer said in MVP:

Who will the Welcome squad welcome (esp if the 50 above are already mostly fellows)

Overall, I also want to get a clearer idea of what the Welcome Squad entails. @vivek Here, I just had in mind a place to run it with a similar group size to see how it goes. As for welcoming the welcome squad, we could do that as part of the town hall for fellows when we explain the MVP and its purpose, or could brief the welcomers a half-hour before the event itself. Still up in the air, and I think that’ll be clearer once we have an ops plan for the Welcome Squad event itself.

add “~ relevant and engaging topics” to the Will the uncof work metrics.

A graceful addition . I plan on “popping into” all the sessions for a few minutes, and just seeing how people engage. Also at the end, I like to do a plenary where everyone has the chance to share 1 thing they learned they feel others would also benefit from in case they missed it. The feedback form I rely on also asked what they wanted to learn, if they learned those things. Those indicate which sessions people found relevant, and by subtraction, we can see which sessions maybe weren’t so engaging. So, where we missed on curation both on missed shots (irrelevant sessions) and unseen targets (goals unaddressed).

Looking at ways to spot assumptions, I wonder how to spot if the unconf itself (not just some sessions) misses the mark somehow… any ideas?

[vivek]

Very good stuff.

@saintsal said in MVP:

Overall, I also want to get a clearer idea of what the Welcome Squad entails. @vivek Here, I just had in mind a place to run it with a similar group size to see how it goes.

I like 100 total, with a 1/n buddying up ratio, which means the Welcoming Squad will be 50 new people, 50 existing fellows.

Teams & Buddies --> Welcome Squads?

Regarding teams & team formation, where do we see that happening?

Are these ‘teams’ groups that are explicitly making something together, or are they intended to be support structures to one another as they answer questions? A bit of both?

While a buddy brings you into Kernel, need they be on your team?

Answering my own question: Buddying will get us groups of two, we’ll need something else to get to teams, perhaps in the… Welcoming Squad!

Right after you meet your buddy, you meet all 100 people in the group, too (50 newcomers to Kernel, 50 Kernel Fellows).

In this context, the 50 newcomers are being welcomed by 50 handpicked Kernel people. Both are participating in Kernel Recreation.

Before the Welcoming Squad
IMO, before the Welcoming Squad event, we should share:

• The bio’s of the participants in the event
• Their open questions
• The chat for the event

@saintsal said in MVP:

What do y’all think of these numbers?

10-15 teams of 3-5 seems easy for the make-a-thing part.
50+ is necessary to try the Welcome Squad and Unconference. The unconf could even benefit going up to 70-80.
10 people per week for the Research Clinic is a good target, but it could handle more people (who’d just observe)
10-20 / week is pretty healthy for the Intention Circle.
10-15 juntos per week (~half to be run as Paper Party sessions & half scheduled on convo)
If we want to run dedicated Learn Juntos each week, 30-40 targetted but increasing as the weeks progress

These all look good to me. My head goes towards communication between Application & Learn Track, to make sure everyone knows their options well during that time (Paper Party, Intention Circles, Learn Juntos, Vibe Hours). @aliyajypsy maybe we should play with this soon and share back with the group.

Thank you for kicking us off, Sal.

[aliyajypsy]

Thank you for painting this MVP picture, Salim. It is exciting to imagine experimenting and playing with these ideas we have been talking about!

A major #~ measurable & subjective observation for me is whether our written communication will achieve the desired outcomes, that is, will the audience understand the MVP from the blogpost, the tweet ? Will candidates and current fellows understand the participation we need for the MVP to function? How do we overcommunicate to deliver our space’s requirements?

My assumption is that written communication is not sufficient for our audience to 100% understand the MVP, nor to convert enough current fellows to participate in make-a-thing, buddying up & welcoming. I suggest that we host at least two town halls after the blogpost is on the website & the tweet announcement made, to provide as much clarity in advance as possible. @saintsal , I know you were already planning a town hall for the kernel community. I want to emphasize the importance of multidimensional communication (more than just written messaging) that is intended for multiple stakeholders:

• A town hall for the public via a Twitter space unless we prefer more moderation via pre-registration for a Zoom/Jitsi call. Here we will describe the MVP & have a Q&A, & compile a FAQ that we’ll post to Twitter or website or both. We could also consider using the ‘what if’ Twitter space with Fang & Nirmala if timing & purpose feel right - it is currently set for March 23, a week from tomorrow. Post recording.

• A town hall for the Kernel community internally via Convo (maybe two for multiple time zones). Here we will describe the MVP, new programming offerings,& expound on the role of defending fellows, buddying up fellows/welcoming squad, & garner interest. Important call to action: airtable forms ready to share & start recruiting defenders, buddying up fellows/welcoming squad & record interest if current fellows imagine supporting in other ways. Post recording.

• Can discuss if a town hall for mentors and/or guild leaders makes sense here too.

I also have an assumption about the number of new participants needed in the mempool MVP to materialize substantive weekly engagement. I estimate it would be more in the lines of 75-100 new people. I think there might be a wow factor once we start the MVP - that the entrance into the mempool has a lower bar to entry compared to the old block and the buzz this could create on Twitter - ‘being accepted into Kernel’ & adding it to a new fellow’s profile.

A concern I have is that we will accept people based on their written application, they enter & then realize “Oh this actually requires lot of agency & self-starter energy to attain any gains & make it into the ‘real Kernel.’ My job search is more important right now or my life is just so busy I can’t give this program what it needs - can I defer my mempool spot to the next one?”

Predicting human behavior is difficult. I could be very wrong, & I hope I am. Maybe our #mempool-defending protocol will sufficiently be good at eliminating such people. I just feel compelled to share my gut feeling though. I guess the beauty in the design is we can observe how the caliber and engagement are developing once we have 50 new people & can adjust if the participation doesn’t seem to be enough. (& those accepted into the mempool, are we calling them fellows, or applicants? Or are they just people?). PS - still throwing my hat in the ring to make the two tracks verbs - ‘Learning Track’ & ‘(Re)Creating Track’ for mempool track & fellows track.

A question i’d like for numbers in the make-a-thing:
What if only 10 current fellows sign up to participate in make-a-thing? How will that serve ~50-100 mempool people in the make-a-thing? Why are we limiting it to current fellows, why can it not be more open? Or is this a part of buddying up that I’m missing?

I also ask @cryptowanderer 's question:

to what will they be welcomed (esp if we throw away what doesn’t work)?

After we run the MVP for 4 weeks, do we stop, assess, discard, and build anew? Begin quicker convergence process round 2? Does this MVP not include attempting to run ‘fellows track’? Or what happens to the fellows who were selected for buddying up & were preparing to be welcomed by the squad? Or is this buddying up specific to the make-a-thing & I am confusing different programming pieces?

@vivek , can you expound / be more specific?

My head goes towards communication between Application & Learn Track, to make sure everyone knows their options well during that time (Paper Party, Intention Circles, Learn Juntos, Vibe Hours). @aliyajypsy maybe we should play with this soon and share back with the group.

[saintsal]

Thanks for these thoughts. They’re a good start, but kind of veer off the mark, so let’s retrace our steps.

I see a lot of thought has gone into scale and promotion, which is getting ahead of ourselves a bit. The main question right now is what assumptions do you have that you’ll want to learn about, and what will you observe?

So the decisions on size and other parameters need to flow from that. The second we involve another goal, it stops working as an MVP. It’s a very steep drop off. (The question of numbers is simply to learn if we’re either able to hit targets, or if the relevant context of our observations require a certain minimum of participants. For example, do we learn if Welcome Squads work with 50 people if our MVP puts 10 people or 100 people in a Welcome Squad? Not really, so this is where we non-learning goals overriding the learning ones.)

Note: Minimal and product are usually confused when people meet the term MVP. “Product” is not a product as a thing a customer uses, but the result of multiplying two factors. Those factors: our effort and time. An MVP is something that maximises our learning with minimal effort and time, and usually doesn’t look like a minimised end-user product.

@vivek I didn’t pick up any reflection on what assumptions you want to learn about – did I miss them?

@aliyajypsy I see some of that starting. noting your concerns is useful – just shift your response from solutionising to open-ended questions, like: what number of Learn Track participants is sufficient or too many? Then, what do we need to observe to learn that? Then I can design the MVP accordingly.

Things like public promotion are not part of this MVP. We don’t have that as a risk to evaluate. Trying to cater to stuff like that is where the cliff drop-off happens. As soon as we’re trying to design for things like public perception, as soon as we’re trying to include more people than we can easily swing to new design on-the-fly, or more people included than we can each individually talk to, we no longer have something designed for our learning. MVP game over. The MVP can’t be precious like that, which is why have to make it disposable. (This partially is why I want to do a private town hall for the fellows that will help, and promote publicly without calling it an MVP.)

So think about the Mempool Shape that we’re evaluating (the actual program, not the MVP), reflect on assumptions and learning goals, then think about the observations that would inform that. I will then design the MVP, which will be the minimum possible so we can make those observations.

[vivek]

Things I want to learn about are mostly encompassed in Sal’s metrics, but I’ll center my interests here.

Are participant questions clear, centered, and do they make progress towards them via the interdependence of Kernel?

• number (#) of clear project questions which make it from mempool to fellows track
• number (#) of made things and quality of progress (~) during the Fellows Track (Make-A-Thing)

Does the event lead to meaningful relationships?

• number (#) of ‘teams’ / ‘peer groups’ (?) formed
• number (~) of meaningful relationships fostered
• number (~) of altruistic interactions between peers

I leaned upon our game designer friends for the key characteristics I was searching for here.

1. Friendship: The formation and maintenance of healthy, meaningful friendship networks.
2. Thriving individuals: The facilitation of eudaimonic happiness. Players feel competence, volition, and relatedness, both for themselves and for their friends.
3. Altruism: The promotion of activities that involve intrinsically motivated cooperation.
4. Positive group norms: The spread and enforcement of shared altruistic social norms within and across groups.
5. Shared goals: Players work towards those goals via mutual interdependence, and achieve feelings of purpose and meaningfulness.

[vivek]

@saintsal said in MVP:

Next Steps

1. Give some thought to things we might observe to inform us (respond here today or tomorrow with your thoughts)
2. Finesse or change the MVP design accordingly. Let’s talk about this async first ideally (then in a group call later this week if needed) to make sure we’re asking and answering the right questions.

Somewhere between Step 1&2, here are some unanswered MVP design questions I see noted on the forum.

(There were other clarifying questions posted, but the ones I pulled were explicitly related to clarifying the design, suggesting potential finesses and explorations). Please add if I missed any.

• Vivek: Where does team formation happen? Past buddying up, are you moving into teams to work on the same things? Into ‘peer groups’ with others, working on your own questions? A mix of the two?
• Sal: What will the welcome squad entail?
• Sal: How to spot if the unconf misses the mark?
• Aliya: What if only 10 current fellows sign up to participate in make-a-thing? How will that serve ~50-100 mempool people in the make-a-thing?

Note that Andy’s 3 questions were answered by Sal, but also are informative.

[aliyajypsy]

My assumption: we will need a larger group of mempoolers to see substantive engagement.

My question: what number of Learn Track participants is sufficient or too many? How do we measure substantive engagement?

• '# of people selected for buddying up/total # of mempoolers - what percentage of mempool is selected for buddying up?
• will the weekly juntos & workshops feel empty or crowded, not enough participation or not enough time for all to participate?
• '# of alumni fellows available for buddying up, welcoming, participating in make-a-thing

–

My assumption: people will drop out of mempool.

My questions: at what point in the 4 week mempool loop will they drop out? When do they make the decision that the mempool requires too much activation energy, too many hours a week for their current life circumstances? How will we know when they make the decision, do we pre-emptively announce, ‘hey in case you’re thinking the mempool isn’t for you, please talk to a steward first. if youre sure you’d like to bow out, here is a survey we’d appreciate you fill out so we can improve.’

• '# of juntos & workshops attended
• '# if & ~ of contacts with other fellows
• ~ what questions are they asking, are they authentically exploring them in paper parties, research clinics, WIP wednesdays
• ~ ability to reach out to a steward/anyone they trust when uncertain/need help
• ~ do they reach out to others to help them?
• ~ learning path created? what is the quality?

–

My assumption: it will be difficult to maintain diversity metrics. Even if we successfully admit the target numbers via #mempool-defending…
My question: will we be able to maintain diversity metrics in mempool while dropouts occur? How do we ensure that dropouts are not disproportionately from underrepresented communities?

• '# of highly participating mempoolers & their demographics
• '# of non-participating mempoolers & their demographics
• ~ Steward outreach to non-participating mempoolers to attempt to troubleshoot & guide

Reminder of Diversity Policy for next Block:
min 2% non-binary
min 49% women
min 30% coders
max 40% US-based

–

My assumption: the forum is where mempoolers will connect with each other, share readings & learning paths, finding overlapping bodies of literature in their libraries & form friendships.

My question: what will the technical capacity of the forum be? How sophisticated will it be? What features are crucial for the development of a learning path & to stimulate discussion & cooperation among each other? How can we create it to seed friendships, especially if Slack channels will be limited?

–

A bit unrelated, but still want to raise at some point…

My assumption: we’ll receive a wide range of applications from various types of people (at least if we open MVP applications to the public).

My question: do we want to set a filter in the application, require a 1 DAI contribution with every application submitted so that at least everyone applying proves they know how to create a wallet and transfer cryptocurrency? This reminds me of a question within Kernel services - what is the minimum knowledge we expect from a person interested in joining our web3 community?

[aliyajypsy]

I agree with this @saintsal :

too early for scale and promotion

(This partially is why I want to do a private town hall for the fellows that will help, and promote publicly without calling it an MVP.)

I was confused - how will we advertise/attract new candidates to apply to the mempool? Must we promote publicly? We currently have 201 candidates whom we deferred to interview for KB8 & about 20 fellows who deferred their acceptance to KB8. That’s already ~220 people with high interest in the program (still a question if deferred accepted fellows go straight to fellows track or to mempool). This pool of folks could serve a funnel for the MVP, as well as asking specific current fellows to invite 1-2 wholesome friends to apply, in case we think it advantageous not to open the floodgates & invite the public via Twitter. Maybe better to be quieter as we work out the MVP?

[saintsal]

These are awesome - thank you!

I will re-assemble all these and adapt the MVP accordingly (and answer your questions as I go). I’ll try to break these into smaller, separate MVPs too, that’d make things easier for us to run.

[saintsal]

@vivek said in MVP:

Are these ‘teams’ groups that are explicitly making something together, or are they intended to be support structures to one another as they answer questions? A bit of both?

Making something together. The emphasis is on shipping something of value to others, doesn’t need to be software.

Regarding teams & team formation, where do we see that happening?

The standard way for hackathons is a find-a-team chat channel and some kind of listing, like in Airtable. EthGlobal also does a live event for team leads to shill. Those are all mediocre.

Meowy shared that Web3con did way better than other hackathons, with 2 f/t facilitators manually connecting people to each other (between 400 hackers!) They have a channel for each sponsor (so you can pop in and ask for help with their tech or get feedback on your idea) and a channel for each team (so people can pop into team channels to offer help) The facilitators basically respond to people looking for teams, connect them to others, and offer to create a team channel as a semi-official way to launch the team. It’s good, old-fashioned match-making.

This makes sense for our hackathon, though doesn’t directly inform the Buddying Up hypothesis for the Kernel program.

While a buddy brings you into Kernel, need they be on your team?

No. The Buddying Up hypothesis will be tested separately from the team formation. Fellows may invite non-fellows to the next block. We’ll give them each one invite, and not mention Buddying Up. Then we’ll see how many naturally invite others, and can follow up to ask why or why not.

This is quite different from doing the buddying up in team formation (which isn’t how the program is designed either)

[saintsal]

@vivek said in MVP:

Sal: What will the welcome squad entail?

This was a question for you and @aliyajypsy . Can you outline a ritual or do a prototype? That’ll give me a sense of what to test and where’s a good place for in the MVP

[saintsal]

@aliyajypsy to your question:

What if only 10 current fellows sign up to participate in make-a-thing? How will that serve ~50-100 mempool people in the make-a-thing?

Okay if this is a question about the program, then there will be max 10 teams in the Make-A-Thing.

If it’s about the MVP, then we learn that only 10 fellows signed up, and find out why.

If there are 100 people in the mempool in the MVP and only 10 fellows, they can form their own teams anyway.

[saintsal]

@aliyajypsy said in MVP:

How will we know when they make the decision, do we pre-emptively announce, ‘hey in case you’re thinking the mempool isn’t for you,

For the MVP, yes I think we plan on reaching out to everyone when we notice they didn’t return. Ideally, we’d have them all in DMs somehow (on their platform of choice, not on our servers)

@aliyajypsy said in MVP:

My question: what will the technical capacity of the forum be?

It’ll be like this one or Discourse. Standard stuff. Using it to seed discussions will be on us, and our hypothesis is it’ll stem from threads we start in the Research Library / Clinic. We also expect the juntos and in-person events to be meaningful contact surfaces for people to meet each other.

@aliyajypsy said in MVP:

require a 1 DAI contribution with every application submitted

Since we want Kernel to be open to non-web3-initiates, I’d say no.

@aliyajypsy said in MVP:

We currently have 201 candidates whom we deferred to interview for KB8

That sounds good to me. They’ve already been attracted through Kernel channels, so represent the population we’re trying to learn about.

[saintsal]

Keeping a list of questions here:

https://hackmd.io/b1fg2tJwQcehC0W1B0nIBA

Nothing new, just added what you’ve shared and reorged it a bit.