Datathon Sofia Air Solution – Air station measurement bias correction using Pearson correlation coefficient

Posted 5 CommentsPosted in Datathons Solutions

This article aims to improve the estimation of the measured PM10 pollutants. In Sofia, there are several air pollution measurement stations. They measure PM10 particles, which are particles found in the air with a diameter between 2.5 and 10 micrometers.

The measurement stations fall into two categories, official stations and citizen stations. The official stations provide reliable measurements, they are better monitored and documented. The down-side is that they are only 5 and they are all concentrated in a single region. The citizen stations represent devices mounted on people homes or properties which measure PM10 particles. There is a whole network of such devices. They are many in number and provide a good coverage of the city. The problem with those measurements is that they are biased because of many local factors. Therefore the measurements form the citizen stations are not as reliable as those from the official stations, but on the up-side they are many in numbers.

In this article we define a method to reduce the bias of the measurements from the citizen stations.

Datathon Telenor Solution – WILDLINGS ANALYSIS ON TELENOR – GAME OF THRONES

Posted 3 CommentsPosted in Prediction systems

Cell phones have become a necessity for many people throughout the world. The ability to keep in touch with family, business associates, and access to email are only a few of the reasons for the increasing importance of cell phones. Today’s technically advanced cell phones are capable of not only receiving and placing phone calls, but storing data, taking pictures, and can even be used as walkie talkies, to name just a few of the available options.
Dataset, The Telenor Case – What do Game of Thrones and Telecoms Have in Common? contains the data of delays in networks (RAVENS). The delays of RAVENS are ranging from 26/07/2018 – 05/08/2018. Each RAVEN_NAME represents the Tower. There are 7847 unique RAVEN_NAMES for different networks like 2G/3G/4G. There are 5 unique families.
To provide optimum solution to business problems we are solving the problem in two steps (i) Data Analysis and coding in PYTHON and (ii) Time Series model building in R Studio.
In data analysis we have found the solutions for the problems and found the number of delays (failures) of RAVENS. We also found the Top_10 RAVENS with and without fails. We also detected the Family names and Member names with most and least fails in networks (failures).
The methods of prediction & forecasting of the problem is done by using Time Series model building. As the name suggests that it involves working on time (years, days, hours, minutes) based on data, to derive the hidden insights to make informed decision making. Time series models are very useful models when it is serially correlated data. Based on mobile data, to predict the four days we have divided the data into train and test .We have done Time series analysis by using Arima, Simple exponential analysis and Recurrent Neural networks (RNN).
Finally we conclude that by considering the Root mean square error for these algorithms, we got RNN (Recurrent Neural Networks) as the best algorithm to predict the future for days. Based on the RNN algorithm the prediction of delays for the next four days were analyzed. We have plotted the graphs based on the Time series model for all the algorithms.

Datathon Telenor Solution – Game of Prediction (GoP)

Posted 3 CommentsPosted in Datathons Solutions

The objective of this analysis is to find out the ravens that are not reaching the destination on time. This kind of analysis would help us to scrutinize and understand the towers(ravens) who would require our utmost attention, in order to improve the reasons which are playing a major role in the delays.
The data-set talks about the networks between the towers (ravens). The land based communication happens with the help of signals.
A cellular network or mobile network is a communication network where the last link is wireless. This wireless transmission is done by a tower which comprises of a transmitter and a receiver (for the wireless transmission). The channel provides transmission for both the data as well as Voice transmission.
Every cellular network has different set of frequencies, to avoid any kind of overlapping and interference. Despite of many precautions for maintaining the setup, there are few parameters that are still impacting the transmission. Few parameters can be classified as:
 Infrastructure
 Interference between the frequencies
 Climatic conditions
 External Factors (Predators etc.)
For this our first approach is to create a “Decision Model” which can help us to give value to our business and help in improving the communication.
****** The tools that we using in order to predict is ******
1. Visual Analysis using different plots
2. Usage of ARMA (Auto-regressive- Moving- Average- Model)
The usage of this Decision Model will help us in forecasting the failure rate for next 4-7 days in regards to the Ravens.

Datathon Telenor Solution – Exploratory Data & Predictive Analytics -Analogy of Game of Thrones With Telenor Telecommunications

Posted 1 CommentPosted in Datathons Solutions

->This datasets is regarding the time series analysis on the failure rate of RAVENS sending the messages from kings landing to the north.
It depicts the analogy of Telenor communication  and Game of Thrones.
-> Sending ravens is one of the most fundamental parameters in mobile communications engineering.
For land-based mobile communications, the received raven variation is primarily the result of multipath fading caused by obstacles such as buildings (or clutter) or terrain irregularities; the distance between link end points; predatory animals, and interference among multiple transmissions, for example wars.
This inevitable raven variation is the cause of communication dropping, one of the most significant quality of service measure in operative communication. For this reason, various techniques and schemes are employed in the planning, design and optimization of raven networks to combat these propagation effects.
This normally covers the network physical configuration which include all aspects of network infrastructure deployment such as locations of base nests; additional food; sometimes guards, etc.
A typical example of these schemes and techniques is the use of models for flight prediction based on measured data.
Based on one month data with flight fails, the participants have to make time-series analysis and predict the future amount of fails.

Datathon Air Sofia Solution – Team Teljapenosss

Posted 3 CommentsPosted in Prediction systems

— Team Teljapenosss Team Members — Jalapeno (Nasiba Zokirova) Team Mentor: petya-par   Business Understanding The levels of air pollution allegedly caused by solid fuel heating and motor vehicle traffic are ever growing in the City of Sofia. The primary economical impact for the City of Sofia was a ruling by the European Court of […]

Datathon NSI Solution – Team Lemurs

Posted 1 CommentPosted in Prediction systems

The NSI case – Predicting Household Budgets by team Lemurs   Team Tsvetan (cecopld) Radoslav (rdimitrov) (mr-reliable) (khadeer) Business Understanding The survey’s data on expenditure of household according to COICOP (quarterly and annually) are used for the purposes of producing macroeconomic statistics – National Accounts and Consumer Price Index. In order to optimize the cost […]

Datathon NSI Mentors’ Guidelines – Economic Time Series Prediction

Posted 1 CommentPosted in GD2018 Mentors, Mentors

In this article the mentors give some preliminary guidelines, advice and suggestions to the participants for the case. Every mentor should write their name and chat name in the beginning of their texts, so that there are no mix-ups with the other menthors. By rules it is essential to follow CRISP-DM methodology (http://www.sv-europe.com/crisp-dm-methodology/). The DSS […]

Datathon Telenor Mentors’ Guidelines – On TelCo predictions

Posted Leave a commentPosted in GD2018 Mentors, Mentors

In this article the mentors give some preliminary guidelines, advice and suggestions to the participants for the case. Every mentor should write their name and chat name in the beginning of their texts, so that there are no mix-ups with the other menthors. By rules it is essential to follow CRISP-DM methodology (http://www.sv-europe.com/crisp-dm-methodology/). The DSS […]

Datathon Sofia Air Mentors’ Guidelines – On IOT Prediction

Posted Leave a commentPosted in GD2018 Mentors, Mentors

In this article the mentors give some preliminary guidelines, advice and suggestions to the participants for the case. Every mentor should write their name and chat name in the beginning of their texts, so that there are no mix-ups with the other menthors. By rules it is essential to follow CRISP-DM methodology (http://www.sv-europe.com/crisp-dm-methodology/). The DSS […]

Datathon Kaufland Mentors’ Guidelines – On Predictive Maintenance

Posted Leave a commentPosted in GD2018 Mentors, Mentors

In this article, the mentors give some preliminary guidelines, advice, and suggestions to the participants for the case. Every mentor should write their name and chat name at the beginning of their texts so that there are no mix-ups with the other mentors. By rules, it is essential to follow CRISP-DM methodology (http://www.sv-europe.com/crisp-dm-methodology/). The DSS […]

Datathon Kaufland Solution – Kaufland case – Team3

Posted 1 CommentPosted in Datathons Solutions

In [1]: import s3fs import pandas as pd import matplotlib.pyplot as plt import matplotlib.dates as mdates import seaborn as sns import numpy as np import pywt In [2]: fs = s3fs.S3FileSystem(anon=True) fs.ls(‘datacases/datathon-2018-2/’) Out[2]: [‘datacases/datathon-2018-2/kaufland’, ‘datacases/datathon-2018-2/nsi’, ‘datacases/datathon-2018-2/ontotext’, ‘datacases/datathon-2018-2/telelink’, ‘datacases/datathon-2018-2/telenor’] In [3]: fs.ls(‘datacases/datathon-2018-2/kaufland’) Out[3]: [‘datacases/datathon-2018-2/kaufland/20180820_Kaufland_case_IoT_and_predictive_maintenance_events.xlsx’, ‘datacases/datathon-2018-2/kaufland/20180920_Kaufland_case_IoT_and_predictive_maintenance.csv’, ‘datacases/datathon-2018-2/kaufland/sample_Kaufland_case_IoT_and_predictive_maintenance.csv’] Events¶ In [4]: with fs.open(‘datacases/datathon-2018-2/kaufland/20180820_Kaufland_case_IoT_and_predictive_maintenance_events.xlsx’, ‘rb’) as f: df_events = pd.read_excel(f) In [5]: df_events Out[5]: […]