3 min read

February Community Newsletter – Data Engineering: Get to the Crux

Picture of Crux Crux Feb 14, 2018

Blog

Hello from the CEO

Philip Brittan, Crux CEO

Delightful data is useful and useable. Data Scientists make data  useful through analysis that extracts valuable insights from the data. But first, Data Engineers make that data  usable by whipping it into shape: loading it, cleaning it, normalizing it, mapping it, joining it, and other transformations that get the data ready for Data Scientists to wring value out of it.

While Data Science gets the headlines, Data Engineering is working hard behind the scenes to make the Data Science magic possible. And by working hard, I mean that Data Engineering typically accounts for 70-80% of the total effort a firm spends on making use of data. Data Science and the unique insights it delivers are business differentiators, but most firms spend a minority of the time on them.

That’s why forward-looking companies increasingly turn to a partner like Crux. By offloading their Data Engineering work, these companies give more time and energy to Data Science and move much more quickly to produce valuable new insights that power their businesses.

Crux brings laser focus, deep expertise, operational oversight, and a valuable network of data suppliers to help you orchestrate, implement, and operate your information supply chain.

At Crux, we make data delightful.

 

Crux Insights Blog

How can you keep the right data flowing into your business? It is simple: Orchestrate, Implement and Operate. Read about Crux’s three step process in our  last blog post. 

 

 

Five in 5 with Head of Data Engineering Andrew Clark

Andrew Clark is Crux’s head of data engineering with a tall ask. At 6’6” he sees the full spectrum of data needs for Crux clients. With deep experience in managing unstructured data, he’s a master of data transportation, storage and repackaging.  Here are five questions  in 5 minutes with Andrew:

 

What does a data engineer do?
At Crux, being a data engineer means handling the tough work that makes data more actionable for our clients, and designing the tools that make our clients’ lives easier over time. Data engineers sit on the “data wrangling” side of the pipeline, meaning we are the folks who handle the hard work of figuring out where certain elements of the dataset live, slicing and dicing data, and repackaging it for distribution.

 

How has the data engineering landscape changed in the last 5 years?
Today, the folks managing information supply chains are embracing the fact that the whole process does not need to exist on-premises anymore. While firms used to believe their data engineering was their “secret sauce”, today they realize it’s the insights they can glean that are more important. Using experts like Crux to remove as much of the tedious, upfront work as possible is now the preferred model.

 

What are you most excited about?
At Crux, we’re illustrating the art of the possible for our clients. What was once difficult has now become easy. Helping clients realize the full potential of their data is truly exciting.

 

What do you do when you are not engineering data?
I am a big outdoorsman, so my favorite activities tend to be outside. I am an avid cyclist, I have a motorcycle and am currently building an airplane.

 

What would geolocation data tell us about you?
If you were to assess my geolocation data, you’d probably find that when I am not working, I like to go to places where the population density is low. This means you’ll probably find me on my bicycle, hiking, or somewhere outdoors and away from the city.

 

Crux Community

Is it difficult to get access to useable data? Let Crux experts engineer your data to make it ready to use. Our data engineers take on your data challenges so that you can spend your time finding signals. Click HERE  to chat with our team of experts.

Have data to share? Our data supplier community is growing by leaps and bounds.  Our diverse datasets range from stock quotes to corporate trends to transportation data and more.  No data is irrelevant.  Create a Crux login HERE to browse our network and become a supplier.

Out and About

We’ve been building our community. In the past month, we’ve met with hundreds of suppliers and buyers of alternative data.

 

Quandl Alternative Data Conference | January 18, 2018
New York, NY

 

Battlefin Discovery Day Miami | January 30-31, 2018
Miami, FL

 

Outsell Data Money | February 1, 2018
New York, NY

 

AI in Fintech Forum | February 8, 2018
Stanford University, Stanford, CA
What Cloud Marketplaces Do and Don’t Do

What Cloud Marketplaces Do and Don’t Do

Picture of Crux Crux: Jan 2, 2024

Not long ago, we observed here in our blog that the critical insights that drive business value come from data that is both (1) fast and (2) reliable.

Read More
The 3 Dimensions of AI Data Preparedness

The 3 Dimensions of AI Data Preparedness

Picture of Crux Crux: Oct 31, 2023

This past year has been exciting, representing the dawning of a new age for artificial intelligence (AI) and machine learning (ML)—with large...

Insider Blog
Read More
How Do Small Hedge Funds Solve the Big Problem of External-Data Integration?

How Do Small Hedge Funds Solve the Big Problem of External-Data Integration?

Picture of Crux Crux: Oct 24, 2023

How do you get white-glove customer service from a major data supplier?

Insider Blog
Read More