You will find references to data often in this blog. Perhaps I’ve even been guilty of a phrase, “It’s all about the data.” Back in 2016, I wrote a post where the title included both “data” and “engineering.”
Marketing managers have been pinging me this year evidently after doing web searches for key words. They get a match on one of my blog posts and write trying to get a link added or an article published. They usually don’t know my focus or even what type of media this is. Many think I’m traditional traded press.
It must have been in such a manner that the marketing manager for Jelvix, which looks to be a Ukrainian software development and IT services company, wrote to me referencing this post I did in 2016. She referenced an article on the company web site by Python developer Vitaliy Naumenko regarding whether or when do you need a Data Scientist or a Data Engineer.
That is an interesting question–one which I have not run across in either my IT or my OT travels.
According to IBM’s CTO report, 87% of data science projects are never really executed. 80% of all data science projects end up failing. Mainly, this happens due to the market’s inability to distinguish data scientists and engineers.
Even now, it’s surprisingly common to find articles online about data scientists’ responsibilities when some of them belong to the data engineer job description. A lack of understanding of what data scientists can and cannot do leads to a high failure percentage and common burn-out.
The thing is, neither data scientists nor engineers can act on their own. Scientists hugely depend on engineers to provide infrastructure. If it’s not set up correctly, even the most skilled scientists with excellent knowledge of complex computational formulas will not execute the project properly.
The data development and management field include many specialties. Data engineers and scientists are only some of the roles necessary in the field. These positions, however, are intertwined – team members can step in and perform tasks that technically belong to another role.
Check out this image, for example. I like the addition of business as well as technology.
Check out his entire article if you are involved with doing something with all the data you are collecting. He suggests organizations for small, medium, and larger organizations. Unfortunately for me, industrial or manufacturing markets are not listed as specialties of the company. But the company has some good ideas to share.
