Data Engineering Blog

All about the Plumbing of Data Science

10 Steps to get a Data Engineering job

Jobs in the field of Data Engineering are in great demand. But how do you get a job like this? 1) Set your personal Goals The first thing you have to do is set personal goals. What do you actually want to achieve? Where do you see yourself in the next few years? What exactly do you want to...


How To Best Start Your First Data Engineering Project!

You want to become a Data Engineer, but don't know how to set up a data engineering project? I will show you! Do not make this mistake! First of all you should not make the mistake that unfortunately many people make! Often people want to build the whole thing from the beginning. They say:...


Data Engineering vs Data Science

What is the difference between data science and data engineering? Data Science Data Scientists work with the data and are basically doing the analytics part. They take the data and create new insights. They do the analytics for instance by using Machine Learning techniques or do just simple...


Where to validate incoming data?

When you watch the blueprint I also use in my cookbook you see the different phases: Connect, Processing Framework, Store and Buffer. At the beginning you think about where you validate the data? To make sure that the data is okay and makes sense. Do you validate the data directly on the API -...


AWS, Azure or GCP?

I get often asked: What is the best or easiest cloud platform to start with for data engineering? Here are my thoughts on this. A general overview. Market shares Amazon Web Services (AWS) is globally the biggest cloud provider (approx. 50% of the market). This goes for the US and the EU...