11/30/2022

One year at Kumo

I often get asked by aspiring founders to share lessons I’ve learned in the one year since the starting of Kumo. At Kumo I wear two hats – one as a co-founder and the second as a head of engineering. This article will focus on the second aspect of my role: we will talk about what we learned in the first year of building the Kumo product and the engineering organization.

The formation of a company and building of a product is perhaps quite like what the early explorers may have felt setting sail on the oceans with rudimentary equipment. There was some idea of a destination but the path and the challenges we would face were unknown. The destination in mind was a hypothesis of the problem we wanted to solve:

We can make predictions about the future as easy as querying the past (SQL).

The hypothesis was based on experience and some early conversations among potential early users. However the ergonomics of our product were unclear. We knew how to build Machine Learning tools for Advanced Users but we needed to understand what interfaces/APIs would work for Analysts. Those early months where we interviewed dozens of ML Users every day would sometimes feel like navigating the aforementioned ship through fog (a feeling familiar to phD students trying to land on a problem formulation). With each user conversation came greater clarity – little glimpses of land out in the yonder till we arrived at our first milestone on the product interface – the “Predictive Query”. At Kumo the Predictive Query is a SQL like interface designed to make forward looking predictions. More on that here.

In those first 3 months we realized that many of our big company processes would just add baggage. In larger companies senior leadership insulates engineers from ambiguity for good reason as it becomes harder to manage that at scale. Engineers at Kumo are not only part of problem formulation but we openly discussed uncertainty of outcomes (eg., it is possible that a user does not use this interface as expected). Embracing the uncertainty and being transparent that many decisions are hypotheses helped us create a culture where we could say that we made a wrong move and course correct. It pushed us to a mindset of failing fast rather than being stuck in analysis paralysis. The picture below captures a diagram a few engineers drew at the 6 month mark to capture that philosophy.

In order to thrive in ambiguity, every engineer who came on board needed to be just as passionate about the overall mission as the founding team. With individuals and company goals aligned, transparency in communication and emphasis on teamwork, we found it much easier to steer the Kumo ship when we hit moments of fog. With the fog out of the way, the wind blows right for a while and you can sail at speed. But new moments of fog come and recede – for example a subsequent one for us was when we had to understand the security and privacy constraints of customers and having one system that satisfied all of them. In each of these cases the following 3 principles helped us sprint forward each time:

Listen to the User
Stay Agile
Hire and Coach for startup DNA

Listen to the user

A culture of customer empathy

Over the last year we have met a few 100 Analysts, Data Scientists, Analytics Engineers, Data Engineers and the leaders of data organizations in doing our research. Customer and user research calls have been always open to all at Kumo. Not all engineers join the calls but by making it available on a tool like Gong, we could create a culture where we openly shared excerpts of calls on the right slack channels or company meetings. This meant that we could openly discuss calls that went well but also post mortem calls that did not. By having engineers be part of the journey and hiring engineers who had the resilience to listen to candid feedback about the product meant that everyone had the opportunity to understand the reasons behind decisions. For example, in our early product we did not automate training dataset creation, but it became increasingly apparent that to cater to the analyst audience we had to automate this part of the workflow. This decision resulted in a rewrite of the code base. Aligning the team on an “all hands on deck” effort towards this goal was easy because we had reached the decision together as a team.

What features to build?

We started our journey with a small sample size of users who were willing and excited to build the product with us. While it was tempting to listen to every feature request in order to convert them to paying customers, we needed to focus and align our scarce resources to build the right product for the right problems. At each opinionated request we needed to ask:

Does it makes sense for Kumo to solve this specific customer pain point i.e., is it in line with the original hypothesis we started out with?

Many times the idea would go into a backlog, but unlike the typical engineering backlog that would continue to expand indefinitely and remain ungroomed, each request needed to be quickly researched. We would quickly probe other customers to see if this was a must have feature. As our active customer base increased, it became easier to discover common patterns, or even schedule a quick call with a champion/advocate to understand their organization’s needs. Once we understood the broader impact of solving a specific problem, we needed to implement and validate the solution with great agility.

Agility

Agility for an engineering team comes from both organizational structure and process as well as code structure and tools.

Operational complexity

If a company has a well-defined product roadmap, it makes sense to align each team in a designated swim lane and specialize in a particular area, maintaining a rigid structure of APIs to interface with one another. However, if a company wants to maintain agility and speed over a dynamic roadmap, a more flexible structure is needed. For us, it made the most sense to run Kumo in pods, which form and persist to achieve a concrete goal. Pods can disband or downsize as the goal is complete, or completely pivot if the priorities change. Pods are composed of a mix of skill sets needed to achieve the goal

We also rally around release versions. Release captain duty is one that rotates among the engineering team and they ensure that last mile integrations are well tested before they land in customer environments. Most of our engineers enjoy their release captaincy rotations as it creates an environment of trust and responsibility where customer impact feels real and immediate. A release captain is also forced to understand what went into a release and thereby has to extend their knowledge beyond their comfort zone in the code base.

Code structure and Tools

At Kumo our starting point was our Open Source library PyG. People love PyG because it is easy to onboard, use and the documentation quality is high. Given this kernel and the core initial team, unlike many other companies, Kumo started at a high bar for code quality. I was pleasantly surprised to see the onboarding time for new engineers drastically lower than my previous jobs (oftentimes it took 1 week from joining to get to the first commit) due to code quality. Having a central set of principles that govern readability and code organization has made it much easier to have engineers learn and pick up new areas of the code base and run the pods successfully.

We have also found that investing time in building tooling for testing is important. In particular testing a machine learning package for regressions in a SaaS environment poses unique challenges. However these investments were necessary to drastically reduce the release cycle, and reduce maintenance overhead. A good test framework ensures we can upgrade all customers in each release and thus have fewer versions to maintain in production across our customers.

Hire for startup DNA

From the above it is evident that startups need individuals that are adaptable and resilient. We need folks who are willing to wear any hat. For example our infrastructure pod pivoted from working on optimizations in memory and throughput to working on tools and testing after they achieved the scale outlined in this blog post simply because the current level of scale satisfies a pretty broad customer base and there were other important and urgent problems to solve.

In a startup where everyone is firing on all cylinders, there is little room for ego or territorialism. While in a larger company there was room to carve out work for individuals who were brilliant but somewhat lone rangers, collaboration is key to startup success. Given the unknown unknowns when innovating, we thrive in a culture where engineers openly say “I’ve underestimated my LOE for this Sprint, I could use help” and others say “I’m actually done with mine, let me figure out how to help you”. Within the team/company we keep re-emphasizing our values: that we are customer centric, optimistic about innovation and better together.

Acknowledge:
Many thanks to Vanja and Jure and the entire team for all the discussions that framed our thinking. In addition, I would like to thank Bill Coughran and the Seqouia founders community for sharing their learnings with us. Thanks Ivaylo for the editorial help in this article!