Datathon Sofia Air Solution – Air station measurement bias correction using Pearson correlation coefficient

Posted 5 CommentsPosted in Datathons Solutions

This article aims to improve the estimation of the measured PM10 pollutants. In Sofia, there are several air pollution measurement stations. They measure PM10 particles, which are particles found in the air with a diameter between 2.5 and 10 micrometers.

The measurement stations fall into two categories, official stations and citizen stations. The official stations provide reliable measurements, they are better monitored and documented. The down-side is that they are only 5 and they are all concentrated in a single region. The citizen stations represent devices mounted on people homes or properties which measure PM10 particles. There is a whole network of such devices. They are many in number and provide a good coverage of the city. The problem with those measurements is that they are biased because of many local factors. Therefore the measurements form the citizen stations are not as reliable as those from the official stations, but on the up-side they are many in numbers.

In this article we define a method to reduce the bias of the measurements from the citizen stations.

Monthly Challenge – Sofia Air – Solution – Jacob Avila

Posted Leave a commentPosted in Prediction systems

Preliminary Analisys Due to the objective focused on predicting air quality forecast for the next 24 hours per station, first step should be data understanding for citizen science air quality measurements to group it by station and summarize them by day. To complete this task for inspection and pre-processing in order to find missing data, outliers and […]

20.10 – Open seminar “Spectral theory of generalized Laguerre semigroups.” at IMI – Bulgarian Academy of Sciences

Posted Leave a commentPosted in News

The day 20/10/2010 was the first to be officially announced by the UN and UNESCO as the world day of statistics. It was then decided that the role of statistics will be celebrated worldwide quinquennially. For the forthcoming 20/10/2015 the department of probability and statistics at the Institute of Mathematics and Informatics (IMI)  at the […]

Searching for our Junior Community Leader

Posted Leave a commentPosted in News

Today we live in an information enabled, connected and data-driven world. Forward-looking companies and individuals are harnessing the power of information and proactively taking data-driven decisions. The Data Science field is on the rise and rightfully so, as it is instrumental in driving research and business forward. Data Science Society develops a platform where data […]

Monthly Challenge – Learning-by-doing in Data Science

Posted Leave a commentPosted in News

The theory is not enough Academic education is indeed the best long-term investment in our professional and personal achievements. However, nowadays it becomes crucial for universities to include different practical seminars in their educational programs with the aim of preparing students for the real problems which they will be solving as professionals in a given […]

October Data Science Monthly Challenge

Posted Leave a commentPosted in Prediction systems

Why you should join the Data Science Monthly Challenge and what you can expect?

The Data Science Monthly Challenge provides an exceptional opportunity for participants to be involved in finding a solution to a real data science problem [] step by step. The proposed gradual approach towards advanced business problems will give participants a chance to familiarize themselves in depth with each of the important steps which should be considered during the development of an effective and high-quality data science projects.

And last but not least the monthly challenge is an excellent opportunity for data enthusiasts to prepare themselves for participation in the Global Datathon organized by the Data Science Society during which the time is constrained and there is a much higher level of competition. The acquired skills and deeper understanding during the monthly challenges will play a key role and serve as a competitive advantage of the teams in such large-scale events such as the Global Datathons. Nevertheless, the monthly challenge can also be inspiring for those with more competitive attitude because there will be voting for each article and peer-to-peer reviews and each week the best-voted articles in progress will be uploaded on the News section of the site. 

So, what are you waiting for? 🙂
Register now for the learning challenge before 15th of Oct at


Posted 3 CommentsPosted in Prediction systems

Cell phones have become a necessity for many people throughout the world. The ability to keep in touch with family, business associates, and access to email are only a few of the reasons for the increasing importance of cell phones. Today’s technically advanced cell phones are capable of not only receiving and placing phone calls, but storing data, taking pictures, and can even be used as walkie talkies, to name just a few of the available options.
Dataset, The Telenor Case – What do Game of Thrones and Telecoms Have in Common? contains the data of delays in networks (RAVENS). The delays of RAVENS are ranging from 26/07/2018 – 05/08/2018. Each RAVEN_NAME represents the Tower. There are 7847 unique RAVEN_NAMES for different networks like 2G/3G/4G. There are 5 unique families.
To provide optimum solution to business problems we are solving the problem in two steps (i) Data Analysis and coding in PYTHON and (ii) Time Series model building in R Studio.
In data analysis we have found the solutions for the problems and found the number of delays (failures) of RAVENS. We also found the Top_10 RAVENS with and without fails. We also detected the Family names and Member names with most and least fails in networks (failures).
The methods of prediction & forecasting of the problem is done by using Time Series model building. As the name suggests that it involves working on time (years, days, hours, minutes) based on data, to derive the hidden insights to make informed decision making. Time series models are very useful models when it is serially correlated data. Based on mobile data, to predict the four days we have divided the data into train and test .We have done Time series analysis by using Arima, Simple exponential analysis and Recurrent Neural networks (RNN).
Finally we conclude that by considering the Root mean square error for these algorithms, we got RNN (Recurrent Neural Networks) as the best algorithm to predict the future for days. Based on the RNN algorithm the prediction of delays for the next four days were analyzed. We have plotted the graphs based on the Time series model for all the algorithms.

Datathon Telenor Solution – Game of Prediction (GoP)

Posted 3 CommentsPosted in Datathons Solutions

The objective of this analysis is to find out the ravens that are not reaching the destination on time. This kind of analysis would help us to scrutinize and understand the towers(ravens) who would require our utmost attention, in order to improve the reasons which are playing a major role in the delays.
The data-set talks about the networks between the towers (ravens). The land based communication happens with the help of signals.
A cellular network or mobile network is a communication network where the last link is wireless. This wireless transmission is done by a tower which comprises of a transmitter and a receiver (for the wireless transmission). The channel provides transmission for both the data as well as Voice transmission.
Every cellular network has different set of frequencies, to avoid any kind of overlapping and interference. Despite of many precautions for maintaining the setup, there are few parameters that are still impacting the transmission. Few parameters can be classified as:
 Infrastructure
 Interference between the frequencies
 Climatic conditions
 External Factors (Predators etc.)
For this our first approach is to create a “Decision Model” which can help us to give value to our business and help in improving the communication.
****** The tools that we using in order to predict is ******
1. Visual Analysis using different plots
2. Usage of ARMA (Auto-regressive- Moving- Average- Model)
The usage of this Decision Model will help us in forecasting the failure rate for next 4-7 days in regards to the Ravens.