<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=293842391037690&amp;ev=PageView&amp;noscript=1">

Astronomer Blog

Culture / Data Science / Dev / Growth

Why Every Data Scientist Needs A Data Engineer

*Image cred: Eunice Lituanas; estimated reading time: 3 minutes, 32 seconds

The data scientist was deemed the “sexiest job of the 21st century.” The Harvard Business Review reasons that this “hybrid of data hacker, analyst, communicator and trusted adviser” is a rare combination of skills, worth a high paycheck.

Too good to be true? Yes, according to Forbes. Turns out, data scientists spend most of their time (up to 79%!) on the part of their job they hate most.

Topics: big data data science data engineering

Why Are Data Scientists Frustrated?


In this video, Aaron Brongersma (data engineer) asks Viraj Parekh (data analyst/scientist) about the most frustrating part of his job and talks about a how the Astronomer platform has changed the game for one customer's data scientist.  

Topics: big data data science data engineering

Data Engineers Talk Data Engineering: A Webinar

Topics: big data data science dev data engineering

Normalizing Data for Warehouse Centralization

*Image thanks to William Bout

*Estimated reading time: 8 minutes and 46 seconds

A very common initiative these days is data warehouse centralization across an organization. DWaaS (data warehouse as a service) has become commoditized to the point that organizations of every size can begin setting up a reporting infrastructure starting at only a couple hundred dollars a month. This is really exciting and, when copying data from other structured databases, a relatively simple process.

Topics: big data data science

Data Wrangling 101: Using Python to Fetch, Manipulate & Visualize NBA Data


Last week, Viraj's article by this name was featured on the Yhat blog... Yhat makes data science [app]plicable for developers by allowing data scientists to programmatically employ R and Python models into production applications via REST API endpoints. The Yhat blog focuses on data science, machine learning and engineering. Check out the intro, which links to the full post on Yhat!

Topics: data science

Six Principles for Sending Surveys

Estimated reading time: 6 minutes, 11 seconds

One of the most common methods an organization can use to collect data is through sending out surveys. At face value, this seems perfectly simple. After all, you just want a few simple answers to a few simple questions.

It’s only when you want to take these responses and try to derive insight out of them that things become very difficult.

Topics: data science

Why Is My Data Playing Hard to Get?

Estimated reading time: 8 minutes, 3 seconds

If you’ve read our blog before, you know that we talk a lot about everything industries (from healthcare to the NBA) can do with data properly structured and stored. But what kind of data are we talking about and why exactly is it so hard to access, organize and store?

Topics: data science

Astronomer Takes PLOTCON: Day 3

Day 3 of PLOTCON rolled out the red carpet for some data analytics superstars including the creators of Jupyter and Anaconda. Viraj and Ben started to write their daily wrap-up (check out Day 1, Day 2 and why we're even here, if you missed it), but it quickly turned from a wrap-up to an open fanboy letter to the analytics community. We had to edit it a bit, but enjoy!

Topics: data science

Astronomer Takes PLOTCON: Day 2

Day 2 of PLOTCON has wrapped, so it’s time for the Astronomer team to debrief from the day. If you missed it, first check out why we’re here and what we learned on Day 1.  

Topics: data science

Astronomer Takes PLOTCON: Day 1

Ben and Viraj just got done with day 1 of PLOTCON 2016. (If you want to know why they're excited to be there, check out Viraj's post about it.) After all the presentations and exhibitions, they sat down to go over their thoughts and shared their conversation with us. Here’s what they had to say: 

Topics: data science