Datathon cases

4 Common Issues Data Scientists Deal With

0
votes

Data scientists possess a unique skill set that can drastically increase efficiency within a company or organization in a very short amount of time. As unique as the skillset is, the language with which data scientists use to convey their findings to the rest of the team, and with the uniqueness comes some good and some bad.

The good is certainly the high demand for the jobs and the competitive pay, but often data scientists run into some fairly unique issues as well. Because the relative novelty of the big data boom, many of these issues come in the form of communication, but the vastness of the internet, where data analysts conduct their business, also brings many challenges. Here are four common issues that data scientists  face.

Poor Communication with PM Team

As the nature of the work conducted by data scientists and developers is far from common knowledge, project managers often struggle to properly convey what they need from the data analysis team. Overcommunication between these two teams is paramount, and no questions should be left unsaid, or a lot of wasted time and effort can be a result. Data scientists should encourage their PM counterparts to ask things like “Is your team able to do (X) for us?” rather than “We need (X),” as some tasks that a layperson would think would be easy for data analysis teams may, in fact, be extremely difficult. If a scenario results in the latter, data science teams can discuss their relative capabilities and form efficient solutions.

“Wasted” Data/Time

Sometimes unused analysis can result from miscommunication, as mentioned above, and almost 1 in 5 data scientists report this lack of clear direction as their most common issue, but data still goes unused very frequently, even if the analysis delivered was exactly what the PM team ordered up, and miscommunication is still the source of the problem.

When data scientists report their findings to the executive decision makers, they often struggle to clearly convey their findings in a way that a non-data scientists could fully comprehend. This issue is so common, however, that many data analysis software offerings now include less-technical readout options for easier mass consumption, generally in the form of graphs and charts. If this is an issue your team faces, finding one of these programs that uses the same programming language that your team is currently using, it can be huge help with the communication barriers.

Bad Data

Bad data, or data that scientists deem unreliable, can stem from a number of occurrences, but one of the most common is called imbalanced data. This happens when a data set comparing two things and the likeliness of these two things to perform one way or another has a lot more information on one of those theoretical “things” than the other.

Another reason data can become unusable, is when upgrades to software do not allow old data to be read or analyzed. There is a fine line to toe between upgrading for efficiency and remaining with the known to maintain normalcy, and this in and of itself is an issue, though one normally discussed by stakeholders outside of the data science team.

Privacy Infractions

One of the key reasons for the boom in data was the evolution of the internet and what the public uses it for. Global ecommerce experienced a whopping 504% increase during the decade that was the 2010s, and with the COVID-19 pandemic forcing many companies and organizations to conduct their business online, it is growing even faster over the last two years. With this growth comes threats, however, and cybercrime is a major issue faced by data scientists, but also a reason there is a lot of money to be made in data security, especially in sensitive fields like finance data and health data.

Solutions

Generally, the communication issues are best solved with training, but more and more technology-related solutions are coming to the surface, which does make the most sense. A combination of technology and effort can help increase the communications, and ultimately the efficiency of the team as a whole.

Share this

One thought on “4 Common Issues Data Scientists Deal With

Leave a Reply