A Build vs. Buy Decision for External Data Solutions is a Balancing Act

Time, money, resources, and scalability can influence when, not if, your business needs to buy an external data solution.

Build or DIY? We face this question on a regular basis in both our personal and professional lives. Do I remodel my kitchen, or hire a company to do it all? Do I want to buy an application for my team, or build one from scratch? For any of these decisions, most people would make a pros and cons list, or weigh both options and consider factors like the time, cost, what else I could be doing instead, and anything else needed to accomplish these to-dos. 

Choosing whether to build or buy an external data solution is no different. To begin evaluating which option is best for your company, of course you’ll want to consider four factors: time, money, resources, and scalability. The quantity of each of these can tip the scales towards build or buy, and help make the path forward clearer. 

These factors can also color the main question you’ll want to ask yourself when determining to build or buy for external data specifically: “Are we focused on investing in and seeing a return on our core business focus, or on shifting to building and supporting an application, software, or product for external data?” 

Balancing Critical Decision Factors 

The amount of time, money, and resources available for your business can determine how quickly you can also scale when it comes to external data. A common mistake made by those who choose to build solutions–often smaller startups who don’t have the capital to invest in enterprise solutions, or larger companies whose DataOps teams are at capacity–don’t factor in scale to the solutions they create. So while having a few data engineers work out of a homegrown data management system may work for now, you must also consider who is maintaining that same pipeline for its entire lifecycle and the time to build it initially and the impact that has on your roadmap. And that comes after the solution has been designed and scoped–a process that can easily take months to get through. So while at a glance building is a better financial solution, the financial impact any of these delays may have on your business is critical to consider long-term. 

Resources are critical to consider as well. Specifically, your employees. The number of external data sources available on the market today is infinite and continues to grow, meaning that businesses are expected to keep up by ingesting more external data constantly to keep up with competitors. Your data engineering team will quickly reach its capacity and its wit’s end as it’s expected to maintain more and more data sources, which is a tedious, time-consuming process with no immediate value, as the value comes from the data sources after all the heavy lifting is done. And data engineers are expensive, often hovering around $350,000 annually, and that’s if and when you can find them, as their skillset is in high demand and short supply. Additionally, other organizations are out there collecting data on your organization, and that includes employee satisfaction. 

Choosing to build a data product is a huge monetary investment, but without a plan to productize whatever solution your team develops, there is no ROI associated with it. Investing your time, money, and resources to build a solution that already exists on the market is great if you have an infinite amount of time and money, and need a completely customized solution. 

But it’s more likely that your data engineering teams will quickly hit a breaking point as you scale your data sources, and continuing to throw bodies at the problem is an expensive solution with no maximum capacity if your pockets are deep, creating a never-ending cycle. 

Making a Decision 

It’s obvious that buying is the better decision when it comes to external data solutions. At Crux, we’ve invested $100M+ into our external data solution. Could your organization afford to invest that much in building a solution, or accomplish what we’ve already done? It makes financial sense to invest in something that has been built specifically to solve your problem. 

But it is possible that buying doesn’t make sense in your current organization. It may be too small or not have wiggle room in the budget for an up-front investment. And that’s okay, but the key here is to plan for that in the future. If your business uses two data sources right now and is comfortable with having their one data engineer maintain them, that’s great. But that doesn’t scale, so the critical piece of building is knowing when to invest in a market solution, and financially planning for that. 

In the long run, it doesn’t make sense to shift your company’s core competency to external data solutions, when the money, time, and resources needed to do so are much better spent on value-add work. 

Whether you’re building and want assistance planning for the future or are ready to invest in the best external data solution the market has to offer, we would love to hear from you.

Reach out to Crux for more information on how to successfully onboard external data with a purchased solution, no matter what stage of business you’re in. Or if you're ready to discuss when is the best time to buy for your organization, email me at charles.ashwanden@cruxdata.com.