Big Data Engineer vs Machine Learning Engineer vs Data Scientist- Questions that helped me choose…

Well, this is actually something quite interesting. And I am sure a lot has been said about these roles on the web and this post is mostly for folks who just started out or are starting, and generally, there are three sorts of people:

Geetha: Geetha is Someone who is already in the field looking to further upskill herself. Her question would be something like “Hi, I just started working as a data engineer, what do I do or what do I learn to become better at my job”.

Ramesh: Ramesh is someone who is already set on switching to a role. His question would be “How do I become a data scientist?”

Suresh: Suresh is someone who is not yet sure as to where she wants to switch to and he needs help. “His question is more like, hey I am working as a Java programmer and I think I like to do A, B and C, and I want to work in the Data/AI domain and what would be the right role for me?”

So the thing I have realized there is a lot of help on the web available to Geeta(as in content helping her with the roles and responsibilities and what she can study or a career path for her).

https://365datascience.com/wp-content/uploads/2018/05/365-Data-Science-Infographic.png

Ramesh and Suresh are the ones that require help. A lot of us have been here. I was here too. With the way everything is being marketed of late it is easy to be swayed to pick up a course for a particular role like a data scientist to only later realize you dont like maths and statistics. Well, I tried at hands a few years back on a side project, and I realized this isn’t my cup of tea. And I am a firm believer in Peter Drucker’s philosophy that It is easier to become great at something that you are good at and like than to become even average at something that you are bad at.

Eg. A person who is good with statistics might naturally like working and developing data science models as compared to someone who actually doesn’t.

So over time, I discovered a few questions to ask yourself if you fit in either Suresh or Ramesh’s profile using the grid below:

  1. Does the idea of programing excite you?
  2. Does building scalable software systems excite you?
  3. Are you good at SQL and understand databases well?
  4. If you like building systems, do you also enjoy trying to explore different tools, set them up from the ground up?
  5. Are you someone who likes maybe not to build tools to do any analysis, but rather look at algorithms that can be used to solve a problem?
  6. Do you like working on the deployment and infrastructure end as well?
  7. And do you want someone who likes core data structures and algorithms?
  8. Are you someone who likes maths and statistics? and likes working on mathematical models?
  9. Are you someone who likes looking at data, trying to find out insights and patterns within it?
  10. If you like looking at data, do you also want to clearly understand the domain and solve a business problem?
  11. When it comes to work, do you enjoy working alone?
  12. Do you want to be an awesome race car driver?
  13. Do you want to be an awesome race car mechanic who builds cars instead?
  14. Are you okay with a job where you need to speak to multiple others to do your job?

Using these questions I sort of build a grid for myself to see which role might suit me better.

As you can see there is are overlaps between roles, and your interests might not fit into one single box. And each organization has a slightly different role definition. The idea is to use this as a skeleton to explore what you want to do. I’d like to quote Peter Drucker here “A person can perform only from strength. One cannot build performance on weakness, let alone on something one cannot do at all.”. So choose your path wisely and not because someone is doing something. Remember at the end of the day every single person is different.

Leave a comment