1
votes
Team intro:
We are team of experts from ShopUp (AI in Retail and eCommerce) Sergi Sergiev and Desislava Nikolova . Our goal was to experiment, play with data and share our experience. We hope you are going to enjoy the reading and please vote + .
Introduction
The main objective of the Article recommender case, is to optimize the suggestions to the readers of articles online. The case has the final goal of engaging the user with topics which are the closest to his points of interest.
Case Summary
The main idea of the case is to predict the next best article for the visitor.
The evaluation of the model will be for the same users using the data for the next time period. The training dataset is for the 30 days. The articles are almost 60 000 and the visitors are over 2 300 000. Total datapoints: 2 350,470,700. Evaluation dataset is going to be for the next 1 day.
Figure: The chart of the tasks
Net Info has provided data with historical visits of articles per user. The data consists of user ID, URL, timestamp, article title and article views.
Features:
- User id
- Time
- URL
- Page Title
- Page Views
Output format: VisitorID, first_best_article, second_best_article for the next 1 day/hour/minute
Research
We looked at different models and made a short list of them:
- Sentence transformer with BERT
- it seems to be on sentence level, can be trained with other languages
- there is russian model
- seems simple but training is the challenge we may have and on sentence range
- Doc2vec on gensim
- Doc2Vec & Logistic Regression with code
- Top2Vec – doc2vec, UMAP, HDBSCAN, centroid computation, n-closest word vectors
- Document Clustering – TFIDF, pretrained word embeddings and text hashing.
- Various word embeddings
- TF-IDF, word2vec, ELMO, Universal Sentence Encoder, Flair embeddings, Spacy embeddings, WMD (Word Movers Distance)
- https://github.com/swapnilg915/cosine_similarity_using_embeddings
- Feed Forward Neural Network Text Classifier
- DocMap
- List of recommenders
- Others
Solution and approach
We decided to create a hybrid recommender focusing on content and user preferences.
About the content we wanted on use Deep Learning in order to provide unsupervised embedings based on the text of the article.
The first step is to gather some additional data, so we decided to scrape some information from the provided articles. The scraping is done with selenium and it creates an additional file, in which we store some important things from the article: website_link, title, subtitle, text, date_of_posting and hashtags. It is not clear which one will describe the article the best. We can compare them and choose one.
Models
- For BERT model we use Russian model because of the time limitations
BERT is designed to help computers understand the meaning of ambiguous language in text by using surrounding text to establish context. https://searchenterpriseai.techtarget.com/definition/BERT-language-model
- Recurrent neural network (RNN)
A recurrent neural network (RNN) is a type of artificial neural network commonly used in natural language processing (NLP). RNNs are designed to recognize a data’s sequential characteristics and use patterns to predict the next likely scenario. We can use this method to predict the next cluster given the previous history we have for each user, in that case we can predict what is the future cluster of interest for that person. We used old code from article which we create in 2019 on summer school https://shopup.me/blog/recommenders_systems/
Figure: Basic RNN
The clusters are representation of the different topics and the interest of the users. For the final decisions, we will use the idea that similar people with similar interests might like the articles which others like. With those clusters will be possible to group people, use some of them for verification and then improve the suggestions for the others.
Evaluation
In order to evaluate the performance of a model and thus be able to choose the highest performing model, a testing functionality is available.
The function makes use of the split into training data and test data.We suggest to convert the data to sequence of chosen articles and to take the first 20 per user and to predict the next 10. For ranking methods we decided to use three metrics which are relevant for measuring ranking:
- Mean Reciprocal Rank (MRR)
- mean Average Precision (MAP) – is supposed to be a classic and a ‘go-to’ metric for measuring the order
- Distribution coef which we defined as how many correct clusters are selected. For example 6 out of all (10) = 60%
The three functions are in the code mean_av_pres(),distribution_coef() and compute_mrr().
Final solution:
The following is the diagram of what out model looks like. The blue parts are ready and the red ones not yet.
Figure: Block scheme of the final solution
The code can be viewed and examined here:
1. Load data and packages¶
In [0]:
!pip install -U sentence-transformers
!pip install git+https://github.com/UKPLab/sentence-transformers#pretrained-models
Requirement already up-to-date: sentence-transformers in /usr/local/lib/python3.6/dist-packages (0.2.6.1) Requirement already satisfied, skipping upgrade: scipy in /usr/local/lib/python3.6/dist-packages (from sentence-transformers) (1.4.1) Requirement already satisfied, skipping upgrade: torch>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from sentence-transformers) (1.5.0+cu101) Requirement already satisfied, skipping upgrade: scikit-learn in /usr/local/lib/python3.6/dist-packages (from sentence-transformers) (0.22.2.post1) Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from sentence-transformers) (1.18.4) Requirement already satisfied, skipping upgrade: tqdm in /usr/local/lib/python3.6/dist-packages (from sentence-transformers) (4.41.1) Requirement already satisfied, skipping upgrade: transformers>=2.8.0 in /usr/local/lib/python3.6/dist-packages (from sentence-transformers) (2.9.1) Requirement already satisfied, skipping upgrade: nltk in /usr/local/lib/python3.6/dist-packages (from sentence-transformers) (3.2.5) Requirement already satisfied, skipping upgrade: future in /usr/local/lib/python3.6/dist-packages (from torch>=1.0.1->sentence-transformers) (0.16.0) Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->sentence-transformers) (0.14.1) Requirement already satisfied, skipping upgrade: regex!=2019.12.17 in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers) (2019.12.20) Requirement already satisfied, skipping upgrade: tokenizers==0.7.0 in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers) (0.7.0) Requirement already satisfied, skipping upgrade: sentencepiece in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers) (0.1.90) Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers) (2.23.0) Requirement already satisfied, skipping upgrade: sacremoses in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers) (0.0.43) Requirement already satisfied, skipping upgrade: filelock in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers) (3.0.12) Requirement already satisfied, skipping upgrade: dataclasses; python_version < "3.7" in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers) (0.7) Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from nltk->sentence-transformers) (1.12.0) Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers) (2.9) Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers) (3.0.4) Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers) (2020.4.5.1) Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers) (1.24.3) Requirement already satisfied, skipping upgrade: click in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers>=2.8.0->sentence-transformers) (7.1.2) Collecting git+https://github.com/UKPLab/sentence-transformers#pretrained-models Cloning https://github.com/UKPLab/sentence-transformers to /tmp/pip-req-build-8axyk952 Running command git clone -q https://github.com/UKPLab/sentence-transformers /tmp/pip-req-build-8axyk952 Requirement already satisfied (use --upgrade to upgrade): sentence-transformers==0.2.6.1 from git+https://github.com/UKPLab/sentence-transformers#pretrained-models in /usr/local/lib/python3.6/dist-packages Requirement already satisfied: transformers>=2.8.0 in /usr/local/lib/python3.6/dist-packages (from sentence-transformers==0.2.6.1) (2.9.1) Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from sentence-transformers==0.2.6.1) (4.41.1) Requirement already satisfied: torch>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from sentence-transformers==0.2.6.1) (1.5.0+cu101) Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from sentence-transformers==0.2.6.1) (1.18.4) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.6/dist-packages (from sentence-transformers==0.2.6.1) (0.22.2.post1) Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from sentence-transformers==0.2.6.1) (1.4.1) Requirement already satisfied: nltk in /usr/local/lib/python3.6/dist-packages (from sentence-transformers==0.2.6.1) (3.2.5) Requirement already satisfied: tokenizers==0.7.0 in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers==0.2.6.1) (0.7.0) Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers==0.2.6.1) (2.23.0) Requirement already satisfied: filelock in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers==0.2.6.1) (3.0.12) Requirement already satisfied: sacremoses in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers==0.2.6.1) (0.0.43) Requirement already satisfied: sentencepiece in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers==0.2.6.1) (0.1.90) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers==0.2.6.1) (2019.12.20) Requirement already satisfied: dataclasses; python_version < "3.7" in /usr/local/lib/python3.6/dist-packages (from transformers>=2.8.0->sentence-transformers==0.2.6.1) (0.7) Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch>=1.0.1->sentence-transformers==0.2.6.1) (0.16.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->sentence-transformers==0.2.6.1) (0.14.1) Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from nltk->sentence-transformers==0.2.6.1) (1.12.0) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers==0.2.6.1) (2.9) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers==0.2.6.1) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers==0.2.6.1) (2020.4.5.1) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->transformers>=2.8.0->sentence-transformers==0.2.6.1) (1.24.3) Requirement already satisfied: click in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers>=2.8.0->sentence-transformers==0.2.6.1) (7.1.2) Building wheels for collected packages: sentence-transformers Building wheel for sentence-transformers (setup.py) ... done Created wheel for sentence-transformers: filename=sentence_transformers-0.2.6.1-cp36-none-any.whl size=80985 sha256=e055d9a2c5d6e2141c636f5c135348ceb2c2cdafd20b7cbe68bac5c310623d3e Stored in directory: /tmp/pip-ephem-wheel-cache-kw3r0m9e/wheels/97/ec/44/a63b7b633eae01893eea846a08788d4f8f921286b3f66dcd4e Successfully built sentence-transformers
In [0]:
from fastai.collab import *
from fastai.tabular import *
from fastai.text import *
import numpy as np
import os
import pandas as pd
from spacy.lang.bg.examples import sentences
from google.colab import drive
drive.mount('/content/gdrive', force_remount=True)
root_dir = "/content/gdrive/My Drive/Colab Notebooks/datathon2020/"
base_dir = root_dir + 'data/'
Mounted at /content/gdrive
In [0]:
2.1 Loading scraped data¶
We scrape data from provided links and capture info about Title, subtitle, date of creation, image links, hashtags and text
In [0]:
df = pd.DataFrame()
for filename in os.listdir(base_dir):
if filename.startswith("scraped_pages"):
path = os.path.join(base_dir, filename)
df_ = pd.read_csv(path)
df = df.append(df_)
#print(os.path.join(base_dir, filename))
#continue
df = df.reset_index()
df["date"] = pd.DatetimeIndex(df["date"]).date
df.head()
Out[0]:
| index | link | title | subtitle | date | image | hashtags | text | |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | Тежка катастрофа в София, кола се обърна по таван | За щастие пътният инцидент се е разминал без с... | 2020-05-16 | https://m.netinfo.bg/media/images/43102/431022... | ['катастрофа', 'София'] | ['Тежка катастрофа между два леки автомобила е... |
| 1 | 1 | www.vesti.bg/bulgaria/katastrofata-razminavane... | Катастрофата, разминаване между думите на Пенч... | Инцидентът е станал при заход към приземяване,... | 2020-05-16 | https://www.vbox7.com/play:09b91178e5 | ['катастрофа'] | ['Наземният контрол е дал указание на Йвайло П... |
| 2 | 2 | www.vesti.bg/bulgaria/vasil-ivanov-s-novi-razk... | Васил Иванов с нови разкрития за партия "Възра... | След изборите, от панелния си апартамент в раб... | 2020-05-16 | https://www.vbox7.com/play:757ff3d1cb | [] | ['Партия "Възраждане" не само е получила парти... |
| 3 | 3 | www.vesti.bg/tehnologii/tajniiat-kosmicheski-s... | Тайният космически самолет X-37B се готви за н... | Той ще се завърне в ниска орбита на 16-ти май | 2020-05-16 | https://m4.netinfo.bg/media/images/40971/40971... | ['самолет', 'космос', 'военен проект', 'военни... | ['Тайният космически самолет X-37B на американ... |
| 4 | 4 | www.vesti.bg/sviat/dw-dokazatelstva-za-zverstv... | DW: Доказателства за зверствата на Асад и Руси... | Очевидци разказват за жестокостите | 2020-05-16 | https://m4.netinfo.bg/media/images/43047/43047... | ['сирия', 'асад', 'русия'] | ['Сирийските правителствени войски и техните р... |
2.2 Can load content data provided by Netinfo¶
In [0]:
path = os.path.join(base_dir+ 'datathon2020vesti.csv')
df1 = pd.read_csv(path, sep=";")
In [0]:
2.3. Sentences Embedding with a Pretrained Model¶
This example shows you how to use an already trained Sentence Transformer model to embed sentences for another task using BERT / RoBERTa / XLM-RoBERTa & Co. with PyTorch¶
First download a pretrained model with Russian language which is supposed to be the closest to the Bulgarian. Training option was not consider having in mind the short time.
In [0]:
In [0]:
from sentence_transformers import SentenceTransformer
#model = SentenceTransformer('bert-base-nli-mean-tokens')
model = SentenceTransformer('distiluse-base-multilingual-cased')
In [0]:
df[0:3]
Out[0]:
| index | link | title | subtitle | date | image | hashtags | text | |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | Тежка катастрофа в София, кола се обърна по таван | За щастие пътният инцидент се е разминал без с... | 2020-05-16 | https://m.netinfo.bg/media/images/43102/431022... | ['катастрофа', 'София'] | ['Тежка катастрофа между два леки автомобила е... |
| 1 | 1 | www.vesti.bg/bulgaria/katastrofata-razminavane... | Катастрофата, разминаване между думите на Пенч... | Инцидентът е станал при заход към приземяване,... | 2020-05-16 | https://www.vbox7.com/play:09b91178e5 | ['катастрофа'] | ['Наземният контрол е дал указание на Йвайло П... |
| 2 | 2 | www.vesti.bg/bulgaria/vasil-ivanov-s-novi-razk... | Васил Иванов с нови разкрития за партия "Възра... | След изборите, от панелния си апартамент в раб... | 2020-05-16 | https://www.vbox7.com/play:757ff3d1cb | [] | ['Партия "Възраждане" не само е получила парти... |
In [0]:
len(df)
Out[0]:
1662
In [0]:
# run the model and generate embeddings
sentences = df.text
sentences
sentence_embeddings = model.encode(sentences)
print(sentence_embeddings[3])
[ 0.019099 0.000158 0.03033 -0.012973 ... 0.021687 0.051281 0.015552 0.041067]
And that's it already. We now have a list of numpy arrays with the embeddings.
In [0]:
2.4 Add clusters plus visualization¶
In [0]:
from sklearn.cluster import KMeans
num_clusters = 30
clustering_model = KMeans(n_clusters=num_clusters)
clustering_model.fit(sentence_embeddings)
cluster_assignment = clustering_model.labels_
In [0]:
df["clusters"] = cluster_assignment
df[0:3]
Out[0]:
| index | link | title | subtitle | date | image | hashtags | text | clusters | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | Тежка катастрофа в София, кола се обърна по таван | За щастие пътният инцидент се е разминал без с... | 2020-05-16 | https://m.netinfo.bg/media/images/43102/431022... | ['катастрофа', 'София'] | ['Тежка катастрофа между два леки автомобила е... | 3 |
| 1 | 1 | www.vesti.bg/bulgaria/katastrofata-razminavane... | Катастрофата, разминаване между думите на Пенч... | Инцидентът е станал при заход към приземяване,... | 2020-05-16 | https://www.vbox7.com/play:09b91178e5 | ['катастрофа'] | ['Наземният контрол е дал указание на Йвайло П... | 17 |
| 2 | 2 | www.vesti.bg/bulgaria/vasil-ivanov-s-novi-razk... | Васил Иванов с нови разкрития за партия "Възра... | След изборите, от панелния си апартамент в раб... | 2020-05-16 | https://www.vbox7.com/play:757ff3d1cb | [] | ['Партия "Възраждане" не само е получила парти... | 16 |
In [0]:
# fit a 2d PCA model to the vectors
from sklearn.decomposition import PCA
#X = model[sentence]
pca = PCA(n_components=2)
result = pca.fit_transform(sentence_embeddings)
In [0]:
#Plot helpers
import matplotlib
import matplotlib.pyplot as plt
# create a plot of the projection
fig, ax = plt.subplots(figsize=(24,18))
ax.scatter(result[:, 0], result[:, 1], c = df.clusters, s=40)
ax.set_title('Tweets')
#words = df.title
words = df.hashtags
for i, word in enumerate(words):
plt.annotate(word, xy=(result[i, 0], result[i, 1]))
plt.show()
In [0]:
# visualize a random cluster
df.hashtags[df.clusters == 12]
Out[0]:
0 ['катастрофа', 'София']
1 ['катастрофа']
12 ['спират', 'движение', 'Подуяне', 'разтопен би...
18 ['носене', 'маска', 'задължително', 'става']
22 ['Милен Цветков', 'катастрофа', 'арест']
...
1524 ['турист', 'лифт', 'пампорово']
1600 ['Бремен', 'Германия', 'пожар', 'мигранти', 'б...
1644 ['стрелба', 'смърт', 'Москва']
1650 ['катастрофа']
1655 ['Италия', 'Генуа']
Name: hashtags, Length: 79, dtype: object
2.5 Finally we have a cluster for each Article¶
In [0]:
df.head()
Out[0]:
| index | link | title | subtitle | date | image | hashtags | text | clusters | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | Тежка катастрофа в София, кола се обърна по таван | За щастие пътният инцидент се е разминал без с... | 2020-05-16 | https://m.netinfo.bg/media/images/43102/431022... | ['катастрофа', 'София'] | ['Тежка катастрофа между два леки автомобила е... | 1 |
| 1 | 1 | www.vesti.bg/bulgaria/katastrofata-razminavane... | Катастрофата, разминаване между думите на Пенч... | Инцидентът е станал при заход към приземяване,... | 2020-05-16 | https://www.vbox7.com/play:09b91178e5 | ['катастрофа'] | ['Наземният контрол е дал указание на Йвайло П... | 1 |
| 2 | 2 | www.vesti.bg/bulgaria/vasil-ivanov-s-novi-razk... | Васил Иванов с нови разкрития за партия "Възра... | След изборите, от панелния си апартамент в раб... | 2020-05-16 | https://www.vbox7.com/play:757ff3d1cb | [] | ['Партия "Възраждане" не само е получила парти... | 3 |
| 3 | 3 | www.vesti.bg/tehnologii/tajniiat-kosmicheski-s... | Тайният космически самолет X-37B се готви за н... | Той ще се завърне в ниска орбита на 16-ти май | 2020-05-16 | https://m4.netinfo.bg/media/images/40971/40971... | ['самолет', 'космос', 'военен проект', 'военни... | ['Тайният космически самолет X-37B на американ... | 29 |
| 4 | 4 | www.vesti.bg/sviat/dw-dokazatelstva-za-zverstv... | DW: Доказателства за зверствата на Асад и Руси... | Очевидци разказват за жестокостите | 2020-05-16 | https://m4.netinfo.bg/media/images/43047/43047... | ['сирия', 'асад', 'русия'] | ['Сирийските правителствени войски и техните р... | 16 |
In [0]:
3. ShopUp sequence data model¶
We use RNN to create predictions of next cluster of interest
In [0]:
df1 = pd.DataFrame()
for filename in os.listdir(base_dir + "/behaviour"):
if filename.startswith("vest000000000000"):
path = os.path.join(base_dir+ "/behaviour", filename)
df_ = pd.read_csv(path)
df1 = df1.append(df_)
#print(os.path.join(base_dir, filename))
#continue
df1 = df1.reset_index()
#df["date"] = pd.DatetimeIndex(df["date"]).date
df1.head()
Out[0]:
| index | pagePath | pageTitle | time | visitor | |
|---|---|---|---|---|---|
| 0 | 0 | www.vesti.bg/temi-v-razvitie/tema-koronavirus/... | Болницата в Сливен спира приема на пациенти с ... | 2020-05-04 09:26:59.361 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 1 | 1 | www.vesti.bg/temi-v-razvitie/tema-koronavirus/... | Болницата в Сливен спира приема на пациенти с ... | 2020-05-04 09:27:24.113 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 2 | 2 | www.vesti.bg/bulgaria/energijnoto-bydeshte-na-... | Енергийното бъдеще на България след коронавиру... | 2020-05-04 09:27:30 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 3 | 3 | www.vesti.bg/sviat/iazovirna-stena-se-skysa-hi... | Язовирна стена се скъса, хиляди домове в Узбек... | 2020-05-04 09:31:06 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 4 | 4 | www.vesti.bg/sviat/iazovirna-stena-se-skysa-hi... | Язовирна стена се скъса, хиляди домове в Узбек... | 2020-05-04 09:31:38.622 UTC | EulCYIFawDfXSLSTgg51+Q== |
In [0]:
del df1["pageTitle"]
df1.head()
Out[0]:
| index | pagePath | time | visitor | |
|---|---|---|---|---|
| 0 | 0 | www.vesti.bg/temi-v-razvitie/tema-koronavirus/... | 2020-05-04 09:26:59.361 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 1 | 1 | www.vesti.bg/temi-v-razvitie/tema-koronavirus/... | 2020-05-04 09:27:24.113 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 2 | 2 | www.vesti.bg/bulgaria/energijnoto-bydeshte-na-... | 2020-05-04 09:27:30 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 3 | 3 | www.vesti.bg/sviat/iazovirna-stena-se-skysa-hi... | 2020-05-04 09:31:06 UTC | EulCYIFawDfXSLSTgg51+Q== |
| 4 | 4 | www.vesti.bg/sviat/iazovirna-stena-se-skysa-hi... | 2020-05-04 09:31:38.622 UTC | EulCYIFawDfXSLSTgg51+Q== |
In [0]:
df1.rename(columns={"pagePath":"link"}, inplace=True)
df1.columns
Out[0]:
Index(['index', 'link', 'time', 'visitor'], dtype='object')
In [0]:
result = pd.merge(df1, df, on=['link'])
result[0:5]
Out[0]:
| index_x | link | time | visitor | index_y | title | subtitle | date | image | hashtags | text | clusters | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 331 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | 2020-05-10 04:56:03.349 UTC | EunYfMugi4/XLC0Mbt3ekg== | 0 | Тежка катастрофа в София, кола се обърна по таван | За щастие пътният инцидент се е разминал без с... | 2020-05-16 | https://m.netinfo.bg/media/images/43102/431022... | ['катастрофа', 'София'] | ['Тежка катастрофа между два леки автомобила е... | 3 |
| 1 | 331 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | 2020-05-10 04:56:03.349 UTC | EunYfMugi4/XLC0Mbt3ekg== | 0 | Валери Симеонов: Божков ще се върне с белезници | Кризата ще подейства на политическия ни живот ... | 2020-05-16 | https://m.netinfo.bg/media/images/41865/418654... | ['Валери Симеонов'] | ['Държавата е длъжна да форсира връщането към ... | 16 |
| 2 | 1533 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | 2020-05-09 14:03:12.326 UTC | EurS/Lg8IJTCcnFSpNd6oA== | 0 | Тежка катастрофа в София, кола се обърна по таван | За щастие пътният инцидент се е разминал без с... | 2020-05-16 | https://m.netinfo.bg/media/images/43102/431022... | ['катастрофа', 'София'] | ['Тежка катастрофа между два леки автомобила е... | 3 |
| 3 | 1533 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | 2020-05-09 14:03:12.326 UTC | EurS/Lg8IJTCcnFSpNd6oA== | 0 | Валери Симеонов: Божков ще се върне с белезници | Кризата ще подейства на политическия ни живот ... | 2020-05-16 | https://m.netinfo.bg/media/images/41865/418654... | ['Валери Симеонов'] | ['Държавата е длъжна да форсира връщането към ... | 16 |
| 4 | 1564 | www.vesti.bg/bulgaria/valeri-simeonov-bozhkov-... | 2020-05-10 05:21:39.385 UTC | EurS/Lg8IJTCcnFSpNd6oA== | 0 | Тежка катастрофа в София, кола се обърна по таван | За щастие пътният инцидент се е разминал без с... | 2020-05-16 | https://m.netinfo.bg/media/images/43102/431022... | ['катастрофа', 'София'] | ['Тежка катастрофа между два леки автомобила е... | 3 |
In [0]:
unique_links = pd.unique(df.pagePath)
unique_links[0:10]
Out[0]:
array(['www.vesti.bg/temi-v-razvitie/tema-koronavirus/le-temps-bylgarski-general-e-geroj-ot-vojnata-sreshtu-covid-19-6109053',
'www.vesti.bg/bulgaria/tri-zheni-zaginaha-pri-tezhka-katastrofa-kraj-silistra-6109203',
'www.vesti.bg/bulgaria/zheni-kalkandzhieva-kriza-e-i-niamam-i-pari-6109228',
'www.vesti.bg/temi-v-razvitie/tema-koronavirus/pada-zabranata-za-turizym-shte-otvoriat-hoteli-6109261',
'www.vesti.bg/temi-v-razvitie/tema-koronavirus/novi-18-sluchaia-na-covid-19-u-nas-obshto-1829-6109259',
'www.vesti.bg/bulgaria/borisov-goranov-nikoga-ne-e-tvyrdial-che-ne-poznava-vasil-bozhkov-6109299',
'www.vesti.bg/bulgaria/kakvi-sa-versiite-za-katastrofata-koiato-otne-zhivota-na-tri-zheni-6109281',
'www.vesti.bg/temi-v-razvitie/tema-koronavirus/vypreki-koronavirusa-te-go-napraviha-pylen-voenen-parad-6109345',
'www.vesti.bg/sviat/es-prizova-skopie-da-prilaga-dogovorite-s-atina-i-sofiia-6109225',
'www.vesti.bg/temi-v-razvitie/tema-koronavirus/koronavirusyt-v-rusiia-polovinata-zarazeni-sa-v-moskva-6109359'],
dtype=object)
In [0]:
%%time
# sort the dateframe
result = result.sort_values('visitor')
vector = []
clusters = []
customer = ""
for _, row in result[0:10000].iterrows():
values = row["clusters"]
if customer == row["visitor"]:
clusters.append(values)
else:
vector.append([customer, clusters])
customer = row["visitor"]
clusters = []
clusters.append(values)
print(vector)
[['', []], ['E++3MFP7eV7wR3rVM6N7sw==', [27, 27]], ['E++RIWdYSkwflDB68hnIbw==', [26]], ['E++SgtIno5VhvxxK9ti3LQ==', [7, 7, 7, 7]], ['E++nkNdjZAMzS2gpqnVokQ==', [24, 24]], ['E++vJVhZZTJzZj753MVLHg==', [28, 28]], ['E+/0s9SEEa4AkIzaWWO1YQ==', [23, 29, 29, 3, 16, 16, 16, 26, 16, 23, 3, 7, 16, 3, 16]], ['E+/TlPtV5TIn30lAB+uSZA==', [27, 27, 27, 27]], ['E+/TvgEEeRJ0Nm+ClX+fLg==', [8, 23, 23, 23, 23, 23, 8, 23, 23, 23]], ['E+01KNFOGkZDagjKuzWT7A==', [7, 7]], ['E+0heC5aVKdIjvrbKlWKJw==', [16, 16, 16, 3]], ['E+0iDTvMhJHE8xHW1vQvzw==', [19, 19, 19, 15, 7, 15, 16, 19, 19, 15, 19, 15, 23, 7, 7, 7, 29, 23, 23, 29, 19, 7, 29, 15, 15, 15, 7, 7, 16, 29, 7, 3, 19, 25, 3, 3, 19, 3, 3, 3, 3, 3, 19, 3, 19, 19, 25, 25, 25, 25, 25, 19, 19, 7, 3, 3, 19, 19, 7, 15, 19, 3, 19]], ['E+0pUKXzmkMw6kr2wA6c4A==', [6, 6, 6, 6, 6, 6, 6, 6, 6, 6]], ['E+1ERRhj3CuecpXFEvQdBg==', [7, 7, 7]], ['E+1f4j39DVfYxMA8RrdOqQ==', [26, 26]], ['E+1sh6+u16Q322deMHh1oA==', [11, 24, 24, 11, 11, 11, 11, 11]], ['E+1v34SYC3znV/jgJ5kxTw==', [7]], ['E+2F5PCjygF+m07mi0zE4A==', [11, 11, 21, 21, 8, 8, 26, 26, 26, 26, 8, 8]], ['E+3+7AE7Wh1fahFLjqG5ag==', [7, 23, 23, 7]], ['E+3Kx54y+JomN71M1IJ65Q==', [28, 28]], ['E+3h0E8OEWBPRFwdIVLjWA==', [26, 26]], ['E+4QW+RFEVbcXF+80JI1Vw==', [26, 26, 26, 26]], ['E+4ffmQYG8BiHnq/hCfdVQ==', [3, 3, 8, 3, 3, 3]], ['E+5MOYw8RCT5juihMjjr5w==', [11, 11, 11, 11, 11, 11, 11, 11, 11, 11]], ['E+5ix8YjCHSAh4Lji45J3g==', [19, 3, 7, 16, 3, 7, 7, 3, 3, 3, 3, 5, 5, 18, 18, 19, 3, 19, 19, 7]], ['E+5yphSATfnoLMNqmVdLyQ==', [7, 23, 26, 7, 7, 26, 7, 7, 27, 27, 26, 8, 26, 8]], ['E+6+8GAoZQE0POkTm7DtNQ==', [11, 28, 11, 11, 11, 11, 11, 8, 11]], ['E+6KBQCuYhfzDxbr651azg==', [26, 24, 24, 24, 24, 26, 16, 11, 8, 25, 26, 26, 11, 16, 26, 8, 26]], ['E+6iNz17N6g4f4u/bCqJtA==', [24, 24]], ['E+78LR1nSb/TIFTFa3w8Aw==', [8, 1, 1, 8, 1, 1]], ['E+7AVDPuEjampmIm1cJZZw==', [15, 15, 7, 7, 15, 15, 8, 8]], ['E+7bAZQMVDO86QyzxjslBw==', [8, 8, 25, 25]], ['E+7fx+BWzoJ3Jz6isPvLqA==', [9, 8, 9, 6, 6, 8]], ['E+7sNgqojZI4dWLAL2KzfQ==', [17, 17]], ['E+8/uNewOyMTyGaEiBFpkg==', [26, 26, 26, 26, 7, 7, 26]], ['E+80bU58nK7KwLFer9kOiw==', [7, 7]], ['E+8cZIkO1vDl73R9ZG0fnQ==', [26, 8, 8, 8, 8, 8, 11, 11, 8, 25, 25, 25, 25, 11, 17, 25, 25, 25, 25, 25, 25, 17, 25, 25, 25, 25, 7, 8, 8, 25, 26, 26, 8, 7, 25, 11, 8, 7, 7, 26, 8, 26, 25, 8, 26, 7, 8, 8, 8, 8, 8, 8, 7, 8, 8, 8, 11, 11, 26, 26, 8, 25, 11, 25, 11, 8, 8, 11, 7, 8, 8, 8, 8, 8]], ['E+8fiXiDe3pzjsPI0YGuhw==', [3, 3]], ['E+8yAHLk3QhtTUvbXGvS0Q==', [19, 1, 24, 23, 24, 24, 24, 24, 19, 23, 7, 19, 23, 19, 8, 23, 18, 18, 8, 8, 3, 19, 7, 19, 24, 26, 1, 29, 19, 19, 27, 7, 27, 29, 29, 25, 25, 29, 26, 8, 25, 25, 6, 7, 19, 25, 7, 7, 6, 19, 25]], ['E+91LjJM0iRBhn1Q+21ZJw==', [7, 7]], ['E+9Je4NdbFLAtGL3BEuGlw==', [7, 7, 7, 8, 7, 7, 3, 7, 8]], ['E+9XHYqOl35JWZOBczAWWg==', [7, 7, 16, 16]], ['E+9h07lT7IiliidCV+z3UQ==', [8, 11, 11, 23, 24, 24, 8, 23, 8]], ['E+9mr0o80ol5PQe9fYNZOw==', [11, 11, 11, 11]], ['E+AYt66mEUU7DdAYDJgWgg==', [7, 7, 7, 7]], ['E+AiPB7BtpZUlHgc9RtV6g==', [8, 8]], ['E+BOVAJjOwgQfsEjvIaBug==', [7, 7]], ['E+BWP2F4+SzmoyyoJDdAqw==', [26, 8]], ['E+CCyOGFpRke/K3nDqW0yg==', [25, 25, 25, 25, 26, 26]], ['E+CsbFv4srgxlUYqHGoNuA==', [8, 8]], ['E+CsdpOfYkSS3JVR8upR4g==', [29, 29, 7, 7, 7, 26, 26, 7, 19, 19]], ['E+CuWgaYZJlF/brImQYlqA==', [8, 27, 26, 11, 11, 27, 26, 8]], ['E+D9KuTSmvBtE+3N74F+IA==', [19, 26, 26, 7, 26, 26, 7, 7, 19, 19, 19, 7]], ['E+DMBP9AYJA9WyY+krMViw==', [3]], ['E+Dyd5DMV1uGQZFi/z7y5w==', [24, 24, 24]], ['E+ELLJB7pzQFAx4SGrQl2Q==', [23, 19, 19, 19, 19, 23, 7, 7, 23, 23, 23, 7, 7, 23]], ['E+FDsgHbs573FHQIc8So7g==', [8]], ['E+FT9b597JkIPTMt75+E4Q==', [11, 11]], ['E+Fj2oxhxOvZYS9PYJ462g==', [8]], ['E+FjCHm9z+6e5uInXkFbwA==', [16, 7, 7, 16, 7, 7, 16, 7, 16, 7]], ['E+Fl5Vf1F7A67mq8Hhe+lw==', [16, 16]], ['E+G3bRMY6yS0a67ncaAguw==', [14, 29, 16, 11, 16, 16, 11, 23, 17, 17, 14, 16]], ['E+H6XOXyOze7w0Pe0H9KCA==', [28, 28]], ['E+H7ekYS0u6wfbC2N4tCRw==', [7, 7, 7, 7]], ['E+H8mmuZ+JH83raNPs9EWw==', [7, 7, 8, 8, 8, 8]], ['E+HV1KXDHNdVawHO0eUIoQ==', [9, 9, 11, 9, 9, 9, 9, 9, 9, 9, 9, 11, 11, 11, 11, 11, 9, 9]], ['E+HarRBgjnskpJUkIxvVQA==', [27, 27]], ['E+HvNVGSGFCB7dAYDNV9Ug==', [7, 7, 7, 7, 8, 7, 7, 7, 7, 7, 7, 7, 26, 26, 7, 7, 7, 8]], ['E+IfH/hbGoP2xAaX6b/ocQ==', [11, 7, 8, 11, 7, 8, 8, 8, 8, 8]], ['E+JS2fuAqPmm2UqmHFdAPw==', [26, 26, 7, 7]], ['E+Jk8bOx104Z8ETtznxH+w==', [8, 8]], ['E+LLJMHDH3le/FcvXjzdQQ==', [29, 29]], ['E+LcSjjoaoxoaT29BNkosg==', [7, 11, 24, 7, 23, 7, 7, 7, 7, 27, 7, 7, 7, 27, 8, 8, 26, 8, 8, 8, 11, 8, 11, 11, 11, 11, 7, 8, 8, 11]], ['E+LgSlmvqR1Bbe1anxlKbw==', [8, 8]], ['E+M+vbYukhV8AMu2JqQJ3A==', [28, 8, 8]], ['E+M2ofcsZC94b7lSozBqJg==', [27, 27, 27, 27]], ['E+MNfmy7YnDK2LIDjGRScw==', [15, 15]], ['E+Mq/GGRBSyPXAplBYtdOA==', [26]], ['E+N226tfMeJ/C3NJpW/PGA==', [8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 26, 25, 26]], ['E+N6ffWTgHuGyO7HhMc0IA==', [26, 26, 26, 26, 26, 25, 25, 26, 26, 26, 19, 26, 19, 26, 26, 26]], ['E+NDnqfc4ckXckNP+OgKyA==', [25, 17, 17, 9, 14, 9, 14, 2, 9, 9]], ['E+NH0VHaciFEWO+IfIyj5Q==', [16, 16, 16, 16, 16, 16, 16, 16]], ['E+NSvJUPUra+BAihxnsBgA==', [7, 7]], ['E+NpFsVSHjV4rLThIuPkTA==', [7, 7]], ['E+Nxt2UoNcuAR6cAGc2a/w==', [7, 19, 7, 19, 26, 3, 26, 26, 3, 26, 29, 25, 29, 29, 29, 26, 26, 26, 29, 17, 18, 26, 26, 26, 26, 26, 2, 8, 18, 8]], ['E+O3oI7yTSZ2fcvUNRixwg==', [17, 29, 29, 17, 16, 16]], ['E+OD/Fo0qX9PlG/M6kXyVQ==', [7, 7]], ['E+OoMoeevsl6QhhArbT2Sw==', [7, 7, 7, 7]], ['E+OubeXhVWF0ARl3YyV68A==', [24, 24, 24, 24]], ['E+PCqCFdzzDF1jdQ3wdxeQ==', [9, 8, 8, 9, 8]], ['E+PiEaXR1/FhC/+C8bh2ng==', [26, 26, 18, 18, 19, 19, 29, 29]], ['E+Pnf3FJzTwh6Sr7flo+gA==', [29, 29, 25, 25]], ['E+QOtZLTN5OW1rU7ft0/IA==', [11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 27, 27, 11]], ['E+QuFRwX1fSmmN/41KLGgw==', [27, 27, 29, 29]], ['E+SN7yLSZs4qe4IVQ3d24w==', [26, 8, 3, 3, 15, 15, 17, 17, 16, 16, 26, 7, 8, 7, 7, 7, 3, 3, 7, 7, 7, 7, 7, 7, 7, 7, 8, 7, 8, 7, 21, 21]], ['E+SqO+EX3r1cRyp4xIlFag==', [28, 28]], ['E+TB0XfLHfm8UDZn9ZjN8g==', [26, 26, 26]], ['E+TNnmNTskIlGKMl2rpC1g==', [3, 3]], ['E+TqbAFV4p2O8rSaJpMegA==', [18, 18, 18, 18, 18, 18, 18, 18]], ['E+USXA1H7MmnvhJvCQ/DhA==', [24, 8, 8, 24, 8, 8, 8]], ['E+V6C6W6bZVCotc4IAIpVA==', [25]], ['E+V7hPX/grFupek05E+y6g==', [21, 21]], ['E+VOwnQhHXYuuVyUdHnRsg==', [8, 8, 7, 3, 8, 18, 7, 18]], ['E+VPsfur/G4CJlTiK/pCzg==', [7, 7, 11, 11, 27, 23, 27]], ['E+W6nBjHq0axF1nSfLtrCg==', [9, 9]], ['E+WFQxKFU7t+c6iuBWSrEA==', [15, 19, 19, 7, 7, 26, 26, 11, 24, 16, 16, 16, 15, 16, 16, 8, 8, 24, 16, 11]], ['E+WiHTLYKylHcSX/i6ABlw==', [7, 7, 8, 7, 7, 7, 7, 8, 8, 8]], ['E+X4mBa4vbA0JFsaSFFbJg==', [7, 7, 16, 26, 7, 7, 16, 7, 7, 7, 7, 16, 16, 16]], ['E+X4shKp5R+pMVTtH8Wsdg==', [16, 7, 16, 16, 7, 16]], ['E+XmeiaSeS7mTOD4ElMDjg==', [7, 7]], ['E+XwZmAdH0SqCvMU1RAxxQ==', [28, 28, 28, 28, 28, 28]], ['E+YB/pmE4Lp/TLAqqzH1PA==', [16, 16]], ['E+YBjKxo2XwdNaMNkZda/w==', [25, 25, 8]], ['E+YNbOZdY1oqJzCYx4E2HQ==', [9, 9]], ['E+YfuyDC31t2j0E8p1mLSQ==', [7, 24, 7, 7, 7, 7, 7]], ['E+YlsucSL0GLr2VxtokB7Q==', [27, 8, 8, 27, 8, 8, 11]], ['E+Ynu7n42rRwd7R2hDExDQ==', [25, 25, 7, 7]], ['E+YuhPehrAaMB0V/GbR95w==', [5, 5]], ['E+Z5AwFaZ7t6LBMgfDiu7g==', [25]], ['E+ZCbMz+xENboHdbImbIvQ==', [25, 25, 26, 7, 7, 26]], ['E+Zxdzg9Y91sHX2s8GroNQ==', [23, 23]], ['E+b30X8c81Q5siHuQL+mvA==', [7, 25, 23, 8, 7, 8, 7, 26, 25, 26, 26, 7, 26, 7, 7, 8, 8, 7, 7, 7, 8, 11, 11, 7]], ['E+b7jGom/XOPzJZI+7rcMA==', [8, 8]], ['E+bdKFqWzbvhRTOYQwOhJA==', [23, 23]], ['E+c8Ug0OkEENOnHit9BX5g==', [16, 16, 16, 16]], ['E+dIhXI1yUkAUDSAEzBU8g==', [8, 8]], ['E+dRfVH+0FeTXaYx3TZDUA==', [7, 7, 7, 7, 25]], ['E+ebczjILA8REfSGpSVR0A==', [26, 3, 3, 25, 11, 19, 7, 7, 11, 26]], ['E+ezc3BDp1fsCk9eaR6q/Q==', [26, 26, 26, 23, 3, 3, 28, 28, 26, 27, 23, 11, 11, 9, 11, 11, 11, 26, 26, 26, 11, 27, 9]], ['E+f24NBBPmc6EVum3z117g==', [8, 8, 8, 8, 8, 8]], ['E+fBakU7sJIaoHOlWw9wjg==', [7, 7]], ['E+fXpIA/+ZDmOwIkopRiQA==', [6, 8, 6, 9, 9, 8]], ['E+gVK0wHAOdqRes0YqxdMA==', [19, 19, 7, 19, 7, 7, 7, 26, 16, 16, 16, 16, 16, 16, 16, 16, 7, 7, 7, 19, 19, 3, 3, 8, 8, 8, 8, 7, 8, 26, 26, 19, 8, 8, 8, 27, 27, 26, 26, 26, 26, 19, 19, 8, 8, 11, 11, 11, 11, 11, 11]], ['E+glfi3KDFuh/uVzooL2iQ==', [26, 26]], ['E+goqGA4+2LZ7I7WJERyWA==', [17, 17]], ['E+h/3sWSWiUgxg2pW1BwUg==', [16, 16]], ['E+hBFng1HfUQIjpucNvsvQ==', [7, 7]], ['E+hHSWo8zrGFABBzB/ypiQ==', [7, 7, 7, 25, 7, 16, 16, 25, 7, 7]], ['E+hOpebBy5CZOFOYjmx1ew==', [24, 8, 8, 7, 7, 26, 26, 24, 8, 27, 26, 24, 24, 3, 27, 16, 11, 19, 8, 19, 11, 11, 16, 11, 16, 23, 11, 23]], ['E+hZYkYqhFNJGxerRt1fwA==', [7, 7]], ['E+hbPaPTXaSmjgvAvVlXJw==', [19, 7, 8, 19, 7, 7, 8, 8, 8]], ['E+hppMj1WvYAB/DxDSyBfg==', [11, 26, 11]], ['E+ivzmZTgnPv0c8mwTj8Bg==', [28, 28]], ['E+j++X798MbNVLeR0Gd/jA==', [26, 7, 26, 7]], ['E+jbg31lf3U0kA+iBA7OvQ==', [8, 26, 26, 8, 25, 25, 11, 25, 26, 26, 25, 11]], ['E+jcEV43k7FGW0uOOeQ7Yw==', [26, 26]], ['E+jy1vvZPDlfADzVkwbGKQ==', [23, 23, 23, 23]], ['E+jz85Rv7zFYjI/qLfaT2Q==', [11, 11, 11, 11]], ['E+kCfl44LKTAL9F9DBFMoQ==', [28, 28]], ['E+kR7GFMrTZ7MgRU1b+taA==', [11, 11]], ['E+kSC6diFhbOcY6EaSlzcw==', [24, 24]], ['E+l6Xc72TNulJU1Kvd2HVw==', [7, 7]], ['E+lApJCI4Ncm0iIjyFaXQg==', [11, 8, 26, 11, 7, 7, 19, 11, 18, 7, 7, 18, 7, 7, 7, 7, 3, 3, 7, 7, 7, 25, 25, 19, 24, 25, 24, 24, 27, 7, 27, 23, 24, 19, 19, 26, 26, 16, 16, 19, 3, 24, 24, 24, 25, 25, 11, 23, 3, 26, 23, 16, 16, 3, 26, 24]], ['E+ldV5tyvf8876JGAr5l7w==', [3, 3]], ['E+n4wcnkQb+WsMxxKSX1bQ==', [6, 6]], ['E+nUYS19fKY+/NHYjciy1Q==', [7, 23, 23, 7, 7, 7]], ['E+nnPX/EkBCmTyS0x3/VTg==', [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5]], ['E+oDvSgjo+mwxcz1kvg0zg==', [16, 16]], ['E+p7e4Kg+bc0ix+iEoxrAg==', [7, 7, 7, 7]], ['E+pZlPK+DqJ4OY1XFfkj9w==', [9, 9]], ['E+q89tCP1zUchnZI1r9Www==', [24, 24, 24, 24]], ['E+qEVCkfVg5JCB/a9dwCew==', [3, 3]], ['E+qdA7a+qmQvPEkZKi9rjA==', [26, 7, 23, 23, 26, 7, 7, 17, 26, 26, 17, 1, 25, 25, 7]], ['E+rj38UergkQ65Uxi03h8g==', [7, 7, 7, 7, 27, 7, 27, 27, 27, 7]], ['E+sA1nvoaLO1IMmV8Gcecg==', [8, 8, 8]], ['E+sbQ5a3UF0P/s2vyh42gg==', [8, 8]], ['E+suCnW1ggdjk0r27z+LKQ==', [7, 7, 7, 7]], ['E+t1w7ta7moF2uyofp2p0w==', [3]], ['E+tJ4YjlM0nwa6sKcuAhjw==', [16, 16]], ['E+ta41Dna8WeMT0JuGXrDg==', [8, 8, 18, 18, 26, 26, 26, 26, 21, 8, 21, 8, 8, 8, 25, 18, 21, 8, 25, 25, 18, 21, 8, 25]], ['E+tiINcqv6nTPvxR8ocmTw==', [28, 28]], ['E+tkIipVZjwiIUw6TCKQow==', [26, 26, 8, 29, 29, 7, 7, 26, 19, 8, 8, 19, 29, 7, 26, 7, 7, 3, 7, 7, 7, 25, 16, 7, 16, 25, 11, 11, 25, 26, 16, 16, 26, 7, 29, 26, 7]], ['E+uGlIBXQ0ISD6WqvXkVCA==', [7, 27, 7, 27]], ['E+v0CTAgvnHC4VUh+toRJA==', [28, 28]], ['E+vRmztwT98EynbeRyfdpw==', [7, 7]], ['E+vbaUEwV/DeT5UxIdqnIg==', [26]], ['E+vgGPntB7eeRknQS/f7PQ==', [11, 7, 28, 7, 19, 28, 28, 27, 27, 28, 7, 19, 7, 7, 8, 11, 11, 11, 26, 7, 26, 8, 8, 15, 15]], ['E+vq3EmfgrNs+lJL1EdgXQ==', [3, 3]], ['E+wF3Z6jxMyYoCZM7u7iwA==', [9, 9, 9, 9]], ['E+wXtlM/Hc4QbbaWebKLqA==', [3, 25, 3]], ['E+woSdWxvofcdchP5Qn8bQ==', [15, 15, 3, 3]], ['E+wx9gEUhnYBSNRXDG0bYQ==', [27, 27]], ['E+x2bsiWCDyyBOZ8fgsl3Q==', [23, 23]], ['E+x6e4DOrmA8/g/QmCdTpw==', [7, 7, 26, 26, 7, 7, 26, 26, 21, 3, 8, 8, 26, 12, 12, 21, 3, 26, 3, 16, 16, 7, 7, 7, 7, 7, 7]], ['E+xDfjwEJ61312vV/YYhig==', [25, 25, 25, 25]], ['E+xPxoASNET++Si/Frn4pw==', [3, 3]], ['E+xsFaMoEcqXsY+YSeurKw==', [24, 11, 11, 11, 17, 17, 11, 11, 26, 26, 26, 26, 26, 11, 24, 7, 24, 7, 7, 7, 7, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 8, 7, 7, 7, 7, 7, 26, 24, 16, 8, 24, 26, 24, 3, 24, 24, 24, 26, 24, 26, 16, 26, 3]], ['E+yuv4u5tCS4x1kZKuEFRA==', [26, 26]], ['E+zh5SPcliHk4MVIoj37Ng==', [25]], ['E+zkCIhapBgWKLf6zViQoQ==', [3, 16, 16, 3]], ['E/+5rBp+W1plMKaFsUZVTA==', [16, 11, 11, 16, 16, 16]], ['E/+Pf1eDcZMImskp9yN9MQ==', [8, 8]], ['E/+WabnsiRjlbDTBGLdjCw==', [7, 7, 3, 23, 23, 19, 17, 24, 26, 24, 24, 19, 17]], ['E/+XQEfeXCt84ioo7ZfVDQ==', [16, 16, 16, 16, 19, 19]], ['E/+eoO+/TcwjVAQUYfbn2w==', [25]], ['E/+oISY92mzKgDWJk1mHVg==', [8, 8]], ['E/+udNcp78avrNkbMVChhw==', [15, 15, 24, 24, 3, 3, 7, 11, 7, 7, 11]], ['E//K1TnPcmJjAY0ggaoNtg==', [11, 19, 11, 11, 11, 17, 7, 19, 3, 7, 3, 11, 11, 3, 17, 16]], ['E//Pf8uinUlBJCiCEJ/W0A==', [7, 26, 26, 7]], ['E/0/eau44dxdKP5Ypnejkw==', [25, 25]], ['E/0IKRd4qL4+AS1UfJc9KA==', [18, 18]], ['E/0MES/0FR4032MmHWwaxw==', [1, 1]], ['E/0aAH/5XEFNpBTXq2wlUw==', [7, 7, 24, 7, 7, 7, 7, 19, 24, 7, 7, 3, 29, 29]], ['E/1V/X5/IXGY+OteVFScrg==', [21, 7, 7, 21]], ['E/1m/PDhp3MeldY1mEVt9A==', [7, 7]], ['E/1wpth/9g3FfXbju4kJcQ==', [3, 29, 15, 1, 1, 1, 1, 15, 3, 18, 29, 29, 18, 18, 18, 3, 3, 1, 1, 29]], ['E/2Adhsaj7e1C/LxD6q6GQ==', [3, 3]], ['E/2KLmN2t32Zcr2jIZNBPg==', [7, 7]], ['E/2oG5Z3sjeX0UYVlDFxQg==', [8]], ['E/33+brXiY3WgqXE7f+nfQ==', [3, 3, 7, 29, 3, 3, 29, 29, 29, 11, 11, 8, 7, 8, 7, 7, 11]], ['E/3Eg2zPqUgoi92MNqNZlQ==', [7, 7, 7, 7, 7, 25, 26, 7, 8, 8, 8, 8, 7, 7, 11, 21, 8, 26, 3, 3, 8, 8, 8, 21, 21, 26, 26, 21, 26, 21, 21, 11, 18, 18, 21, 24, 24, 21, 24, 24, 8, 8, 3, 8, 8, 7, 11, 7, 14, 14, 24, 7, 29, 29, 3, 11, 25, 25, 8, 8, 8, 8, 8, 8, 25, 25, 11, 8, 8, 8, 8, 8, 24, 7, 7, 8, 7, 9, 9, 9, 9, 9, 8, 7, 26, 19, 19, 9, 8, 8, 2, 24, 26, 8, 27, 2, 27, 8, 7, 24, 7]], ['E/4+uTUqRpIsejR6MekvLg==', [24, 24]], ['E/4izfwbSU+E8iuP4d9g6g==', [7, 19, 7, 19]], ['E/4zsgN/yda0XKjP7P/G1g==', [7, 7, 16, 16]], ['E/5HkZwsyLbUUwZu7D3abA==', [8]], ['E/5g6sHkXcrLciI02KjFXg==', [14, 14, 25, 25, 25, 25, 25, 25, 25, 25, 14, 14, 14, 14, 14, 7, 14, 7]], ['E/6/C7LAdsQ+xFgfX1Cz1g==', [29, 29, 29, 29, 29, 29, 29, 29]], ['E/6B3WnWgYL+CoIvR1cMhQ==', [19, 19, 19, 19]], ['E/7FrHeDd7QBd6HSB0AkSg==', [19, 19]], ['E/7g7k6SkVl26ge2ogu0Zg==', [27, 27]], ['E/7yng54sqLQP0Ifj9Ejzw==', [9, 17, 3, 9, 9, 9, 17, 19, 19, 24, 9, 9, 19, 19, 3, 24]], ['E/87vI4I0SQ3FEN1OXlSxw==', [8, 8]], ['E/8E4Lwqv8tk5V7HvrRjLA==', [3, 6, 6, 6, 6, 3]], ['E/9BUN1NCAHYzf4CBNB4SQ==', [8]], ['E/9Q0TnsqifR87qgHyEz+w==', [29, 26, 7, 7, 29]], ['E/9QMLgvc0uYfOoU9aSxAQ==', [3, 7, 7, 7, 7, 11, 8, 8, 7, 7, 27, 11, 3, 11, 25, 25, 7, 24, 11, 24, 24, 8, 27, 24, 7, 3, 8, 11]], ['E/AVhRxYjW2JmYn7EO6HmA==', [27, 27]], ['E/As6rCcOlK8l/1X2SEVsg==', [3]], ['E/BTZelaYit1+m6yBBOi/g==', [27, 27]], ['E/Cl+MICYfNYpXXfwiVS2g==', [26, 19, 19, 19, 26, 19]], ['E/D0A9L1hsbLeXyl0/vIuA==', [25]], ['E/DVXWcvf1SNOrntxfvBpA==', [17, 17, 8, 7, 7, 7, 7, 17, 17]], ['E/DWajtoirtKEosBF4V4Cg==', [23, 23]], ['E/DlDzJDo0HiaYJluojwGQ==', [26, 26, 8, 26, 8, 8, 26, 7, 16, 16, 7, 26, 26, 21, 7, 21, 25, 25, 25, 25, 26, 3, 7, 3, 8, 8, 8, 26, 16, 7, 7, 3, 8, 16, 26, 24, 24, 24, 24, 24, 16, 16, 16, 16, 24, 25, 26, 25, 25, 26, 17, 17, 7, 24, 24, 26, 7]], ['E/EP1OOH7BK0aSm+EtJzKA==', [3, 3]], ['E/EewF+5liXAQH7LPlbmMQ==', [9, 9]], ['E/EqW3UwMGyK/iGv8pfaIQ==', [7, 7]], ['E/F32JCn6gIfepi2cHpzSA==', [9, 8, 8, 8, 8, 8, 8, 9, 9, 21, 21, 9, 9, 9, 8, 8, 9, 8, 8, 9, 9, 8, 9, 17, 17, 8]], ['E/FRau+vb5QrSlfHzd01ZA==', [3, 3]], ['E/FvIPaWYWpUMUDvMHHCew==', [8, 8, 8, 8]], ['E/G8EIvgltk5hXRwAXDHoQ==', [19, 19, 19, 19, 27, 27]], ['E/GCZsEEwdy1oIzBv11PkQ==', [8, 8]], ['E/GmBMoFfdTyHDfBc2yvgg==', [8, 8]], ['E/GnxI2q9F2rZTMams7+nw==', [7, 16, 26, 9, 26, 26, 9, 26, 25, 25, 25, 25, 8, 16, 7, 8, 24, 24, 24, 24]], ['E/GvEqhPFEy4rtXYRNUZRw==', [8, 8, 8, 8, 8, 8, 16, 8, 8, 8, 8, 8, 8, 16, 8, 8, 8, 7, 19, 7, 7, 8, 8, 8, 8, 3, 3, 3, 3, 3, 3, 3, 26, 26, 24, 24, 19, 7, 19, 19, 19, 19, 19, 19, 24, 24, 19, 19, 7, 7, 7, 7]], ['E/H4ttZjQh/q4BRMk36dmg==', [8, 8, 8, 26, 16, 16]], ['E/Hc294ZrXq4ca69ukmBoA==', [23, 25, 25, 9, 16, 26, 27, 27, 8, 16, 8, 25, 9, 26, 8, 25, 11, 7, 23, 23, 23, 7, 7, 8, 7, 23, 7, 7, 7, 7, 8, 8, 23, 16, 8, 11, 23, 16, 29, 29, 26, 26, 3, 3, 8, 8, 23, 23, 8, 8, 8, 8, 8, 26, 8, 8, 16]], ['E/JO4Fm4l/yxYAb4N7Nfaw==', [7, 7]], ['E/JWDrFNJ+vCih81tY7g4Q==', [3, 3, 3, 3, 24, 8, 8, 8, 8, 8, 8, 8, 8, 8, 24, 8, 8]], ['E/JbJPntjZcA/ubpNaqXAA==', [8, 8, 8, 8, 8, 25, 26, 8, 25]], ['E/K+SYoGzlXDzGgM3nmToQ==', [7, 7]], ['E/K3BYifeZyxE9fUbOlFgA==', [29, 29]], ['E/KB34wR8UbrutKhanACyw==', [11, 27, 27, 27, 27, 27, 11, 11, 11, 27, 27, 27, 27, 11, 27, 11, 27, 27, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 24, 24, 24, 7, 7, 7, 7, 7, 7, 8, 24, 18, 8, 8, 11, 25, 8, 11, 8, 11, 7, 11, 11, 11, 7]], ['E/KWcSDKwqsJ9jwnuLiFTA==', [8, 23, 23]], ['E/LCostKwg5fXtAexllAsg==', [3, 17, 3]], ['E/LNvhKtpVEGWgy6ZbUdYw==', [16, 16, 8, 8]], ['E/M/Zbvw7vMGUG84HaYxUA==', [26, 26, 26, 19, 19, 26, 26, 26, 19, 26, 25, 8, 26, 26, 26, 25, 26, 26, 19, 26, 24, 15, 3, 24, 24, 15, 26, 24, 25, 25, 25, 25, 25, 25, 16, 16, 16, 16, 16, 16, 16, 16, 19, 19, 19, 11, 26, 15, 24, 19, 9, 9, 11, 15, 24, 19, 24, 8, 8, 1, 7, 7, 24, 8, 11]], ['E/M3nuQ5byp6c//CJ4ymOg==', [3]], ['E/MClKB16z6DHlqS1IjwzA==', [11, 18, 18, 18, 7, 7, 26, 26, 26, 26, 26, 11, 25, 3, 3, 25, 7, 11, 11, 25, 25, 26, 7, 7, 7, 7, 7, 7, 7]], ['E/MWl/njJs58VRoNYg+s1g==', [15, 19, 21, 19, 26, 24, 24, 24, 19, 8, 8, 3, 15, 19, 19, 19, 24, 25, 19, 21, 26, 26, 16, 16, 15, 15, 3]], ['E/MdK6qyr3ukXbcUohmQpw==', [7, 7, 7, 7]], ['E/MfNeicGP07bvfJDotTig==', [24, 24, 7, 7, 7]], ['E/NRDlN/gmsBMmsBy4/D7g==', [8, 3, 3, 3, 3, 3, 11, 11, 16, 17, 11, 11, 11, 17, 17, 7, 7, 16, 21, 26, 26, 8, 26, 21, 26, 16, 16, 3, 1, 1, 1, 1, 3, 3, 11, 26, 26, 3, 3]], ['E/NfpDt7CXJRJO+IapZFmQ==', [26, 26]], ['E/Nmxu79EQnteqACTVI3mg==', [8, 8, 8, 8, 8]], ['E/NzVTUAz7e5KDgScUxzcg==', [7, 3, 3, 7]], ['E/O3NrBr6UQ9HvxaEvCOqA==', [7, 7]], ['E/Odvv+iGd6pHOXONcRK8w==', [24, 7, 24, 25, 26, 3, 26, 26, 26]], ['E/Ox8cEYied+aVjjiLUC3g==', [7, 7]], ['E/PYrY6ZLeUh2IevPml/LQ==', [8, 8, 7, 21, 8, 7, 8, 21]], ['E/PiSzvYd/vRr8sbhwuG8A==', [7, 16, 16, 26, 7, 26, 16, 16]], ['E/PtX7/MZp4Zas6d9m+3gQ==', [9, 8, 8, 9, 16, 16]], ['E/Q1EV19xABZ89clenSpSA==', [11, 11, 11, 11, 11, 11, 11, 11]], ['E/Q8qeTs4/McPIJSiu5dDA==', [24, 24]], ['E/QAMte5YlFVfoZG2TrBzg==', [3, 3]], ['E/QwL24Tr+rKR1vRL3eA0g==', [11, 11]], ['E/RZoK/3/tsdYZ9rDqAtQA==', [11, 18, 18, 11, 7, 11, 15, 7, 7, 4, 7, 7, 7, 7, 7, 7, 15, 7, 4, 8, 7]], ['E/RafobvLclN8kLapL+Xlg==', [8, 8, 8, 8, 6, 6, 8, 8, 8]], ['E/S04M4K2lclS23hKhWChg==', [25, 25, 8, 8, 3, 8]], ['E/S9rHdzPtNXCAg0PhR+0g==', [15, 19, 19, 3, 3, 19, 15, 15, 19, 11, 11, 11, 11, 19, 15, 15, 19, 15]], ['E/SDbWjaeEW8Pz/KsHQ7ig==', [8, 17, 7, 7, 17, 8]], ['E/SG3F8mz3cWBGL4tkgQ+g==', [28, 28]], ['E/STplX833r+nZSXKqWMOA==', [21, 21, 21, 23, 11, 14, 9, 9, 25, 3, 9, 25, 25, 25, 9, 17, 17, 17, 19, 19, 25, 25, 11, 11, 17, 14, 8, 11, 14, 14, 21, 8, 11, 8, 11, 26, 11, 26, 8, 8, 8, 8, 8, 8, 11, 11, 26, 21, 26, 21, 24, 21, 29, 8, 24, 16, 24, 16, 8, 8, 8, 8, 8, 25, 8, 24, 19, 19, 29, 29, 2, 26, 26, 26, 26, 7, 7, 7, 7, 2, 8, 27, 27, 24, 24, 29, 25, 21, 26]], ['E/SX2939wppWaH88ZD0JCA==', [19, 19]], ['E/SalBR7dUfAV3xJzFfEtA==', [7, 7]], ['E/Sb4iywR/vqPdipZil+4w==', [7, 7]], ['E/ScQsKOK2fLb4UWSAEh7g==', [7, 7]], ['E/SpqKXBKeEb0D5Va1TTmQ==', [8, 8]], ['E/T1c/ZpL8NlpQScnEkzhA==', [8, 7, 8, 7, 7, 7, 7, 7, 25, 24, 24, 29, 29, 16, 8, 11, 11, 8, 8, 8, 7, 15, 7, 7, 7, 15, 7, 24, 7, 7, 7, 24, 7, 26, 7, 26, 26, 4, 16, 16, 8, 8, 7, 8, 8, 7, 8, 25, 8, 7, 25, 11, 8, 11, 8, 11, 7, 8, 8, 8, 7, 3, 26, 25, 8, 7, 21, 8, 7, 12, 24, 24, 12, 8, 16, 11, 16, 8, 8, 8, 3, 3, 3, 3, 8, 3, 15, 7, 7, 8, 8, 8, 8, 8, 8, 8]], ['E/TGJqRAw7EHbtVZ0CWuPg==', [26, 8, 8, 8, 8, 11, 8, 8, 26, 8, 26, 11, 11, 8, 8, 8, 8, 8, 8]], ['E/TJm7AFnfhrhppN8R+18A==', [6, 6]], ['E/TQbibgBu8tgouKO4II9g==', [3, 19, 19, 3]], ['E/UpEivuioGv7sOtZK98Jw==', [11, 8, 8, 8, 11, 11, 8, 11]], ['E/Uu2+SJTshiHxu4ZMU9bw==', [7, 7, 7, 7]], ['E/UyfyUZk63ACfyXixayIw==', [25, 25, 25, 24, 25, 9, 7, 24, 24, 24, 25, 26, 24, 15, 15, 24, 9, 26, 7, 24, 24, 15, 15, 15, 15, 15, 15, 24, 24, 15, 15, 15, 15, 25, 16, 25, 7, 29, 26, 19, 19, 15, 15, 15, 15, 15, 15, 16, 16, 26, 3, 3, 3, 23, 23, 11, 11, 29, 1, 25, 25, 25, 25, 16, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 1, 1, 1, 25, 26, 7, 7, 26, 26, 29, 29, 26, 26, 7, 7, 7, 7, 11, 11, 16, 16, 7, 7, 16, 27, 7, 7, 7, 7, 7, 7, 7, 7, 7, 27, 7, 7, 7]], ['E/V1Pf1nHeVBW/tMWk/FXw==', [3]], ['E/W++BC1kTjypYGW2kUxSg==', [8, 8]], ['E/WM7AvC3+e2n+XuPAkw7A==', [21, 21]], ['E/XL+W+mXt29To2kC29AzQ==', [8, 8, 8, 8, 8, 8]], ['E/XiUfMXpt3DXJZDZrPRpA==', [7, 7]], ['E/YJgP5t4wyTNrRbY3DrPA==', [11, 11, 11, 11]], ['E/YdXdiGNgNQpWptpk/J+w==', [26, 26, 26, 26, 7, 8, 8, 8, 8, 7, 7, 11, 7, 7, 11, 8, 8, 8, 7, 7, 8, 7, 7, 26, 26, 26, 26, 3, 8, 3]], ['E/Z9Ej973klDTmt4ls5a2Q==', [19, 19, 19, 26, 3, 3, 3, 3, 24, 26, 24, 2, 2, 19, 3, 3, 25, 1, 1, 24, 26, 25, 3, 26, 24, 24, 26, 24]], ['E/ZCatYNzPT0Vx1INUjxgw==', [8, 27, 27, 27, 8, 16, 8, 7, 7, 7, 8, 7, 27, 8, 8, 8, 8, 8, 8, 8, 8, 8, 16, 8, 8, 8, 8, 8, 8, 16, 8, 8, 16, 8, 8, 8]], ['E/ZeKHuh0iJFVMTc4WuEpw==', [8, 8, 8]], ['E/ZyNPzd/8M/qFb7OnjZVQ==', [25, 16, 16, 25]], ['E/a/X4sdxaQ0CxICD8zsMw==', [26, 11, 19, 19, 23, 7, 7, 28, 28, 8, 8, 7, 25, 25, 23, 8, 11, 11, 7, 7, 7, 23, 23, 16, 7, 7, 3, 16, 9, 9, 11, 25, 7, 11, 26, 25, 17, 8, 26, 26, 26, 1, 1, 1, 2, 1, 8, 8, 8, 8, 26, 25, 11, 3, 29, 29, 8, 8, 18, 8, 8, 8, 11, 8, 8, 19, 26, 19, 24, 24, 7, 7, 24, 24, 1, 1, 19, 7, 7, 7, 8, 16, 16, 26, 26, 8, 17, 26, 26]], ['E/a72hR6jAwkE0BdEWOFiw==', [8]], ['E/aDTFzTkpbZ5uJIuI9Y6Q==', [9, 8, 8, 9, 9, 9]], ['E/aNeDp0hOAZPhcasbVSLw==', [26, 7, 7, 7, 26]], ['E/am+BwEnwYKKTAhw2JU+g==', [25, 25, 25, 25, 26, 7, 7, 7, 7]], ['E/am+bi6KaPD5RCkZB3e8Q==', [7, 7, 7, 7, 7, 7]], ['E/bMFwLehCOrXRF0L6mVWg==', [26, 25, 25, 18, 26, 25, 25, 26, 18, 18, 18, 26]], ['E/cEY3DzLcF1daHnTNAmvg==', [29, 29]], ['E/cQx9b7CqAnXruy3rOSew==', [7, 7, 8, 8, 11, 11, 7, 7, 7, 7, 7, 8, 7, 19, 8, 27, 19, 7, 27, 8, 7, 7, 8, 8, 8, 8, 8, 7, 7]], ['E/d3t1eFraYmsBmSzrKpiA==', [8, 26, 11, 7, 26, 8, 23, 7, 27, 27, 7, 24, 24, 7, 16, 24, 23, 7, 24, 26, 7, 7, 11, 26, 16]], ['E/dHAVb13M83V9WRgBPh7Q==', [6, 6]], ['E/dbfFnKHtVy6OjpSyZ3iw==', [15, 15, 15]], ['E/dzfUZLM5wxZ8AHMh/H8Q==', [8, 8, 8, 8, 8, 8, 8, 8]], ['E/e2Hws9yubdeF9tQpRePQ==', [3]], ['E/eLkXPMAQz50aSB4ROvqg==', [16, 15, 16, 15]], ['E/f3Y7x6H7SoV4p3iywrpA==', [16, 16, 16, 16, 15, 19, 9, 29, 29, 15, 3, 19, 19, 3, 19, 9, 9, 9]], ['E/fSWaU2qwWXSiC3fPrrNg==', [25, 25]], ['E/fSdj8QsvMgC6XqnpaHlQ==', [24, 16, 16, 24, 24, 15, 16, 24, 26, 26, 24, 16, 24, 16, 16, 16, 24, 16, 24, 16, 16, 24, 16, 16, 15, 24]], ['E/ggrHPIgtNPdfh9jM6A7A==', [8, 7, 7, 8]], ['E/gtXIdg9X83ofT917NQoA==', [8, 8]], ['E/gudo7w81vPD8goY60lAA==', [8, 8]], ['E/hOhv/VQeLAcVXKrUpKzQ==', [26, 7, 7, 16, 26, 26, 26, 26, 25, 16, 25, 26, 16, 16]], ['E/hSbs/GGL01zqPaiUC+Dg==', [26, 29, 29, 29, 29, 26, 26, 24, 24, 26, 8, 8, 8, 8]], ['E/i7jkoUgvD+sYJH9gzd1g==', [26, 16, 16, 26, 26, 26]], ['E/iA6JCqJtkwvZ7WkoD03Q==', [27, 27]], ['E/iH9DGIrLfaHg+eqSa74w==', [8, 26, 26, 26, 8]], ['E/iKQYv7l5oAnd6JXUb8nQ==', [25]], ['E/iagMfHcCHk7wy3mcKFlw==', [24, 7, 15, 15, 26, 24, 15, 24, 15, 7, 26, 7, 7, 1, 3, 1, 3, 7, 24, 7]], ['E/idxSUE6TiMGMSz0SeAwA==', [19, 19]], ['E/jEuh8wmbtjsoS7pluAPA==', [3, 26, 17, 24, 26, 24, 24, 24, 7, 7]], ['E/jWzfsyF64K1V4NwMhvbw==', [7, 7]], ['E/jgKOgNgZ4pkXcmkcYc3A==', [8, 7, 7, 8, 21, 21, 8, 8]], ['E/kEWxvEYloJjrRqRUdR8w==', [7, 7]], ['E/kUmMv67BzSze+dnDzgTQ==', [19, 24, 18, 19, 19, 19, 19, 19, 19, 19, 26, 19, 26, 26, 19, 19, 19, 24, 19, 16, 18, 24, 24, 25, 19, 16, 7, 7, 7, 7, 7, 7, 26, 26, 19, 26, 26, 19, 19, 15, 8, 8, 3, 3, 3, 7, 7, 7, 19, 19, 26, 8, 19]], ['E/khjs8wHqU/3feqVVP8iA==', [11, 11]], ['E/lCQuRvwsqYi4n1iOO0Sw==', [9, 9]], ['E/lG7GkfSdkb+jZoI/EQZw==', [7, 7]], ['E/mPH2EcX/NQJqE4QIC5mg==', [19, 8, 19, 3, 19, 11, 8, 19, 19, 19, 19, 8, 19, 26, 26, 19, 19, 3, 19, 3, 3, 24, 3, 24, 3, 19, 24, 19, 29, 24, 29, 8, 8, 3]], ['E/mTy9QBUx+upNAfB76zsw==', [25, 25, 25, 25, 8, 8, 7, 7, 7, 7, 8, 14, 8, 14]], ['E/myR+RlsaMPvQmRDxyRdA==', [25, 11, 11, 25, 25, 25]], ['E/mzt5qFtv51Sivm4eh+2w==', [9, 8, 8, 9]], ['E/nCFu5F2VGXImdKWYkttg==', [16, 16, 15, 26, 16, 15, 8, 24, 16, 24, 24, 16, 16, 24, 24, 24, 8, 7, 7, 7, 24, 24, 24, 24, 8, 24, 8, 26, 24, 25, 7, 7]], ['E/nYcQAv2swCrm9Fl3AGdg==', [21, 7, 7, 21, 8, 26, 15, 26, 15, 15, 14, 14]], ['E/nsVjXNTRhg/My0vWDpBA==', [25, 25, 25, 25, 25, 25]], ['E/o3Jxvn8QZCgJ0zylTLjg==', [7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 11, 11, 7]], ['E/oBiro3Vh/Xole2rWQ9UQ==', [8, 8]], ['E/oz1/RaWHiAlIfR5/OEbA==', [8]], ['E/pDmhp8Qj87t2nTbOWTIw==', [7, 7, 7]], ['E/pGAvwXinE/kJKjHZXZpQ==', [8, 8, 26, 7, 11, 7, 26, 11, 11]], ['E/pLAy4rSYb6mDUFRFr6qw==', [17, 17, 11, 11, 17, 24, 11, 11, 11, 17, 24, 11]], ['E/pPldcDChulfxhBjFQvAg==', [8, 8, 8, 25, 8, 26, 26, 8, 8, 25, 8, 26, 7, 7, 8, 8]], ['E/pQ85swcF5f3MePRGGr7g==', [26, 21, 26, 8, 26, 24, 21, 26, 1, 8, 1, 24, 25, 8, 26, 19, 24, 25, 7, 19, 7, 11, 11]], ['E/qH58wi1j6p36M4KvuZJA==', [3]], ['E/qWN8ScpmC1Z8TSj8WHNA==', [8, 8]], ['E/qhGzV6Gi3CvvLyOc/Q1A==', [16, 7, 8, 16, 8, 16, 8, 16, 7, 16]], ['E/qkfgSjhL0tZ0MFDV/OmQ==', [7, 7]], ['E/r6gsiPSObizdeVxfsbcw==', [11, 11, 26, 26, 26, 11, 26, 15, 26, 26, 15, 26, 11, 7, 26, 7, 7, 26, 7]], ['E/rG0x3z4lEpyLE5GMscRw==', [19, 19]], ['E/rVEIa/sXmzBlDzffvRwg==', [8, 8, 7, 8, 7, 19, 7, 7, 26, 8, 19, 9, 8, 9, 3, 17]], ['E/sE8Xrvkg9MLMamHxfVfQ==', [9, 9]], ['E/son7HEIOB7sxASFAa9uw==', [15, 15, 15, 15, 8, 7, 3, 26, 8, 11, 26, 3, 7, 7, 7, 7, 7, 8, 8, 26, 7, 25, 8, 8, 8, 25, 8, 26, 25, 8, 8, 7, 25, 7, 8]], ['E/t2RxUkA7BtKHSm5t1byg==', [3, 23, 23, 24, 8, 23, 24, 24, 8, 24]], ['E/twElSPMa1reZIhRDsVYg==', [21, 21]], ['E/vKDeXbhaBHT9CQoXxKWg==', [8, 8]], ['E/vekodkeKeX8oFP2c/RSA==', [3]], ['E/w8HzwIuHoTI1KHNvjTYw==', [8, 8, 7, 7, 7, 7, 16, 16, 8, 27, 27, 21, 27, 21, 21, 11, 19, 19, 19, 19, 19, 26, 26, 21, 8, 8, 18, 8, 8, 27, 27, 27, 27, 19, 19, 3, 11, 18, 8, 19, 24, 15, 15, 15, 27, 15, 24, 17, 17]], ['E/wTYDQT6ELd5PHogAuv9Q==', [7, 7]], ['E/x41Vn94ctzf1oCcyzbxQ==', [24, 24, 24, 24, 24, 24, 24]], ['E/y4vk4vn6Od7nlaGrlfnQ==', [28, 28]], ['E/zOMexDXesSpfKLwctCXQ==', [6, 6]], ['E/zQBLdcg0NdlWixybGZ8g==', [19, 27, 16, 11, 11, 7, 25, 19, 11, 26, 7, 27, 26, 8, 8, 26, 7, 26, 26, 26, 7, 7, 17, 19, 25, 26, 26, 27, 27, 26, 25, 7, 19, 25]], ['E/zYRJy6qcpUO+RCKDn2/w==', [3, 24, 24, 24, 3, 24, 25, 24, 26, 26, 1, 1, 24, 26, 26, 26, 26, 26, 26]], ['E/zYRNvqInFPqkdkVzNIcA==', [26, 26, 26, 26]], ['E/zYiTOLVEaTPnx1wuc7nA==', [28, 28, 28, 28, 7, 11, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 11, 11, 11, 11, 25, 7, 7, 7, 7, 7, 11, 7, 7]], ['E/znseEegTZlX1IHx3ONqA==', [29]], ['E/zs7aJeW8LFggqKV1Ehbg==', [15, 15, 8]], ['E/zsnjydnu21Ivw3ZCfPwA==', [7, 7, 7, 7]], ['E/zwotWrFH+Qxibeg5qmwQ==', [1, 1, 25, 25, 1, 1]], ['E01/xsUz6q9zI+ed0J33vQ==', [7, 8, 7, 8]], ['E011ewWlZE00L75n2d+j7w==', [19, 19]], ['E015C6tV36dgyaT7u3cvdQ==', [7, 25, 7, 7, 7, 17, 25, 7, 17, 17, 17, 7]], ['E01qoj6iKC1FLldyT8KTHQ==', [7, 7, 7, 8, 8, 7, 7, 8, 8, 7, 25, 25, 25, 25]], ['E02Ng/0f0uBAgkIN1oiNbw==', [8, 23, 8, 23, 19, 26, 19, 3]], ['E03fD9NKq8HYdZLX8B+9Ng==', [28, 12, 12, 17, 26, 26, 26, 26, 26, 26, 26, 26, 12, 26, 26, 17, 17, 26, 26, 26, 21, 21, 28, 28, 11, 11, 11, 28, 11, 11, 21, 12, 17, 11, 21, 26, 11, 11]], ['E04LLdzO36HPT08pClkFFw==', [8, 8]], ['E04PLbIbU+MdYJgSWqV7cQ==', [28, 28]], ['E04X10eICJjGBGA9zFYkBw==', [22, 22]], ['E053aX1h7KCqqhb2mMvtrQ==', [29, 29, 29, 26, 29, 25, 8, 26, 25]], ['E0565AYCqi5nfOSlWrZlMA==', [1, 1]], ['E05AeU7+x+m17NQLNMdYtg==', [3]], ['E05aIZOo4bruk7aAAHj5iQ==', [18, 18]], ['E05zwQ4N1K6AUBY+sDZ9TQ==', [25, 25]], ['E07HLPUJzeBsu5o1UGgP3g==', [7, 7]], ['E07U+fgkBiPPYArwN+m+9Q==', [6, 6]], ['E07ukZrSaAWNhQvZa2jSjg==', [6, 6]], ['E08+k2n+rxK9EEnncgG/Fg==', [8, 8, 3, 8, 8, 8, 8, 3, 8, 8, 8, 8]], ['E08E1g98KxGjn95/a79w2w==', [6, 6]], ['E08KUAbYnhUuqhEPYJoAeQ==', [8, 8, 8, 8]], ['E08UOd2UFyJFj62X4IqWtQ==', [8, 25, 25, 19, 7, 8, 29, 29, 19, 19, 3, 3, 19, 7]], ['E08VZZlRjezz1Rn7Cu5zNw==', [3]], ['E08vkMEuXmXxAZ9Alq8y+A==', [3, 3]], ['E09HxOvta9R5Vudhw6z4EA==', [7, 7, 7, 7, 7, 17, 16, 16, 7]], ['E09TXVANIuZJTuvXiRhjEQ==', [8, 8, 7]], ['E0AjWyMCc3CmuxTjY6j7Pw==', [18, 18, 18, 18, 18, 18]], ['E0BFKOBzshDS4aNmZlbaYA==', [26, 24, 24, 26]], ['E0BWqdbADLKaHNmDE+t9lQ==', [25]], ['E0BcLJBgffqBAC228cHa1g==', [8, 8]], ['E0Bx1Gf9npFiRMuvmavpow==', [28, 28, 28, 28]], ['E0BzTly2qyG78xWa6umqzw==', [9, 9]], ['E0CUVgoy310lRRDh9ENMYg==', [8, 8]], ['E0CV3pyH1Db+NQ7Gcu9KVQ==', [21, 21]], ['E0CZoFnEVk538Av7XFe2vQ==', [29, 29, 29, 29, 29, 29, 8, 8]], ['E0CfKVVC6TfsLLeHvwbhKA==', [8, 8, 8]], ['E0D0pBhfrMEPGJ+GjVgFTQ==', [8, 25, 8, 7, 7, 26, 26, 7, 8, 25, 8, 7, 8, 7, 8, 5, 5, 7]], ['E0DCy21yp+ZjzdVV6Tihyg==', [25, 25]], ['E0E0ZYYbADHIvNv1VwNj2w==', [3, 3, 23, 19]], ['E0E2vKHEY4m/18dU2HcRyw==', [25, 25]], ['E0E9JfY7GI3agh3lnFc1FQ==', [11, 11]], ['E0EEOiFc8NI/3poC5i2nMw==', [28, 28]], ['E0EK5aBHXQ7WE6wri+U1Aw==', [16, 7, 7, 8, 7, 8, 3, 8, 8, 7, 16, 19, 7, 19, 19, 3, 7, 7, 7, 19, 8, 7, 7, 8]], ['E0ESw9TmqqG1+d0ASZra0g==', [11, 7, 11, 11, 7, 11, 26, 18, 11, 11, 3, 3, 18, 26, 8, 9, 7, 7, 16, 16, 9, 7, 9, 9, 9, 9, 17, 9, 17, 8, 9, 7]], ['E0EV3/zA7aSiXr3G6aHlAg==', [26, 26]], ['E0Ea/abAiyFCXnSDcMyM6Q==', [3, 3, 3]], ['E0Ee/26r7nz8LAuN5xhxaA==', [27, 8, 17, 17, 24, 27, 27, 27, 27, 27]], ['E0EhN3y2Cx8dC90sM2N1ZQ==', [27, 11, 11, 17, 11, 17, 27]], ['E0F1FEzfVSgG5NaXzy52WA==', [24, 25, 24, 24, 7, 26, 26, 8, 7, 8, 8, 8, 8, 8, 3]], ['E0FGcjfZOmK+0WZKcD3h1g==', [7, 8, 3, 8, 8, 8, 8, 7, 1, 1, 3, 3, 1, 7, 1, 3, 8, 7]], ['E0FUuspeT0p+B3Pq2u0kSQ==', [26, 26, 8, 21, 21, 8, 8, 8]], ['E0Fi024lNlENpm4lJ/HmQg==', [8, 8, 8]], ['E0G3rjSg84FIyCDl4JvpJA==', [11, 11, 11, 11]], ['E0G8RPcAJahBQ3BZVP7IGA==', [22, 22]], ['E0GSgplyVhAKtC3U228z0Q==', [26, 26]], ['E0GSooUBm62V3dQ25ITBsw==', [26, 26, 26, 26, 26, 26, 11, 11, 8, 8]], ['E0Gcoq5OTbBZTRyZsf85xg==', [21, 21]], ['E0HNsJk5a5Y9sqJgHKJ1vg==', [25]], ['E0I/tjaCLR+3OdJoYwo9DQ==', [7, 7, 7, 7]], ['E0I29j4I2PimFl9HHkVvaw==', [3, 26, 16, 16, 26, 26, 3, 7, 19, 7, 3, 3, 3]], ['E0I5Ck4hxuxhKaVZvhK40w==', [25, 25, 25, 8, 8, 8, 8, 25]], ['E0IP8r8xAgdzwNteoWYY2Q==', [8, 7, 7, 8, 8, 8, 7, 8, 25, 25, 7, 7, 7, 7, 25, 25, 7, 8, 7]], ['E0JLZRnSB4UHSYq/vPYqxg==', [8, 8, 8, 8, 11, 8]], ['E0JXMIuKZwTBlvDqzb5QgA==', [29, 29, 29, 29]], ['E0JZlcDWlvO1G6kjFjxePA==', [3, 7, 3, 7, 7, 7, 7, 7]], ['E0Jd3Z2tjdBK9Woy+lL7Yg==', [3, 3, 19, 19, 19, 19, 8, 3, 8, 19, 7, 7, 7, 7, 8, 7, 8, 8, 9, 9, 9, 9, 24, 8, 8, 8, 8, 8, 19, 24, 23, 23, 23, 8, 7, 7, 7, 3, 3, 19, 3, 3, 1, 8, 3, 24, 3, 8, 3, 21, 3, 26, 24, 25, 8, 25, 21, 21, 21, 7, 25, 25, 19, 3, 19, 25, 25, 19, 24, 19, 26, 8, 26, 19, 3, 26, 3, 3, 3, 3, 3, 7, 11, 7, 7, 7, 7, 24, 3, 3, 26, 26, 26, 26, 11, 3, 19, 3, 3, 3, 3, 3, 3, 3, 26, 26, 3, 26, 26, 26, 25, 3, 25, 19, 26, 26, 24, 26, 25, 25, 19, 26, 26, 26, 3, 3, 3, 3, 19, 19, 19, 19, 16, 7, 7, 3, 24, 25, 19, 19, 11, 26, 16, 16, 16, 11, 16, 11, 16, 16, 16, 16, 16]], ['E0JoQUg6bdT/upYRr2MGaQ==', [7, 7]], ['E0JpKVQOZUxJwH8p9W1z5g==', [26, 26, 26, 26]], ['E0KSsV4rOeB/B+MKGCTmxQ==', [29, 29]], ['E0KUpm3mwHX0seG9dAp7RQ==', [8, 8]], ['E0KvM83Yx4SYBv7YPNYTww==', [17, 17]], ['E0KyWhf3HHS1ddArpVM1lw==', [29]], ['E0LT1ZOX1oHHyATszU2IpQ==', [4, 24, 4, 4, 3, 4, 3, 24, 24, 26, 3, 8, 8, 3, 26, 26, 26, 26, 3, 26, 3, 8, 26, 26, 26, 8, 8, 3, 3, 24, 17, 3, 17, 7, 7, 7, 7, 8, 26, 8, 7, 19, 8, 8, 8, 7, 19, 7, 19, 19, 19, 19, 7]], ['E0LVJePtvY+Zwxrn/SPkzQ==', [26, 7, 7]], ['E0MjmOANIsW2NqpaJEPXvw==', [27, 27, 7, 7, 7, 3, 3, 3, 3]], ['E0MokptkkRVZxAPJ83upNA==', [26, 26, 1]], ['E0N18i5qNi2OJlkFnDRJTQ==', [7, 7]], ['E0N9V97h9ME6mtv+3+go1A==', [6, 15, 15, 11, 11, 26, 3, 7, 7, 7, 3, 7, 7, 7, 8, 26, 8, 24, 24, 21, 24, 6, 24, 24, 21, 7, 7, 7]], ['E0NBmr+y5mq8dddtJavMhA==', [2]], ['E0NICKD1fwIRuwwGv8gTFg==', [7, 25, 7]], ['E0NX3bPcDFD5Z8xhuBUMkg==', [24, 24, 7, 7, 7, 7, 28, 26, 26, 26, 28, 26, 28, 26, 26, 24, 16, 16, 16, 16, 16, 16, 16, 26, 7, 29, 29, 29, 29, 7, 16, 16, 26, 7, 16, 7, 24, 28, 26, 26]], ['E0O32t31P+o2WenUSu5U2w==', [24, 7, 8, 8, 8, 7, 29, 7, 8, 8, 8, 7, 7, 7, 8, 7, 8, 29, 24, 29, 8, 7, 8, 29]], ['E0OBPg0c21lSd1iDd4Vx1g==', [26, 26, 11, 26, 11, 26]], ['E0Oi6iXFJ3Z1LvdGhdaZiQ==', [11, 29, 29, 11]], ['E0P5RwqVTaIFE+gzPXUUqg==', [9, 9, 24, 24]], ['E0P7W+BwELuJqY0HNkvMJQ==', [19, 19]], ['E0PDQPCWJZo+DpND3rz25Q==', [8, 8]], ['E0Pl9J940OIuQsJkHZKVfA==', [6, 6]], ['E0PoUHukjqDluUfNMxDRzw==', [2, 8]], ['E0Poplz7vZ1j4Apkk+AKZQ==', [26, 25, 3, 26, 24, 24, 24, 24]], ['E0QA1/FkQS9emmK8t/bWRg==', [9, 9]], ['E0QF/bE4Z1/RY4gdUCL1ww==', [26, 11, 11, 24, 26, 24, 24, 24, 26, 24, 24, 24, 26, 24]], ['E0QJP1a1IGucxKB1ey90Aw==', [7, 7]], ['E0QgkvQPsYyRaSvMstOeGA==', [9, 9, 9, 9, 9, 9]], ['E0Qn7J3JxGneUQ0y3Tmchw==', [29, 14, 29, 14, 14, 14]], ['E0RBYZyeFh1xYhoqTO8DRw==', [19, 19]], ['E0RRLxZlUSx3MPlj+YmfPA==', [25, 25, 7, 7, 7, 7, 7, 7]], ['E0RgWBm7/cdOtjqr/DKgRw==', [15, 26, 15, 15]], ['E0T9QTfWE5A92fLmul8+gA==', [8, 8]], ['E0T9flRmslFokSTV0hVykw==', [27, 11, 27, 27, 27, 11]], ['E0TIIpUTgPLH14XE942g9A==', [26, 26, 26, 26]], ['E0TxFzZKJtcNIglczjctKw==', [7, 7, 7, 7, 29, 8, 7, 19, 7, 8, 7, 7, 7, 29, 8, 8, 19]], ['E0U9D2t/FbouMi7kmlnF4g==', [8, 25, 8]], ['E0UC8Iebh+zfJIY4rMC22g==', [7, 7, 7, 7, 7, 7, 7, 7, 7, 7]], ['E0UDt6J0Lrh+Iqn9oPeCTA==', [7, 7]], ['E0UI81G6aBbJPoeAHMicfg==', [7, 7, 11, 7]], ['E0UxIr/9ume88PKhNPw9iA==', [8, 8, 8, 8]], ['E0V3VVxK/nCrU2yZFAuKUA==', [16, 16]], ['E0Vhsc2NMWHe8dvcGWLFiQ==', [24, 8, 8, 8, 24, 8]], ['E0Vss5GqUwi3iHoN0bbM8A==', [26, 21, 26, 21, 8, 8, 8, 8, 8, 8]], ['E0VtgBZEkzZx8/PquqGXuQ==', [8]], ['E0W+lc6O8P83Fjw5Um5hvA==', [8, 3, 8, 8]], ['E0W9ME1v7MvV+JYquubLSw==', [7, 25, 6, 25, 6, 7, 16, 16, 6, 6, 7, 7, 7, 7]], ['E0WDgKAe7o2KVnEagcIRrQ==', [7, 7, 7, 7, 3, 3]], ['E0Wh8sIf6Tnd3K1o/dvQ0A==', [8, 8]], ['E0WloDJnAfLifEqlNvM7iA==', [16, 16]], ['E0XRSIFo88YwVEl5gBsyMg==', [8, 8]], ['E0XkXGDY3GS5MgKNjQv+5w==', [11, 11]], ['E0XlSi623XlKzLzx26Et5Q==', [23, 23, 27, 7, 3, 8, 8, 7, 27, 7, 7]], ['E0Y8xMB1JcOKnLdaLBSZAg==', [21, 7, 7, 3, 15, 15, 21]], ['E0YKG95NsGyKQ3UfvVTDTg==', [26, 8, 8, 21, 19, 3, 8, 8, 21, 19, 11, 7, 3, 26, 7, 11, 7, 11, 7, 7, 8, 8, 7, 11, 26, 24, 24, 8, 8, 21, 21, 26]], ['E0YOblzAv5H8OSqiP/2vIQ==', [27, 27, 27, 27]], ['E0YRjpo4CUvrxQlAcVGLvQ==', [27, 7, 27, 7]], ['E0Yc950gmSZTNYCDyJ1u/w==', [26, 8, 8, 8, 8, 7, 7, 7, 26, 7, 7, 7, 7, 7, 7, 7, 7, 8]], ['E0ZEtIAva6C8QJQwJM6Q5A==', [25, 7, 7, 7, 7]], ['E0ZRiUgmMopjRSR0M7g6+w==', [7, 7]], ['E0ZUS/O4VbjoqGCagRhwkA==', [8, 8]], ['E0ZadgknLkEP1nxtEw45BA==', [29, 29]], ['E0aCE9g+ZcNo0e26xuT7/g==', [7, 7]], ['E0bEkcpPB8L226VMhFl0Gw==', [9, 9]], ['E0bfV3TTc6NxT6aQwHjodQ==', [7, 7, 7, 7, 7, 7, 25, 7, 16, 7, 7, 7, 16]], ['E0c0U8QFR8sNqCw4YZ3UKQ==', [26, 19, 3]], ['E0cVOhklWhKtT3bCn3bMVg==', [25, 25, 3, 3]], ['E0dDWlIU1Qu5HJfTkZIHxQ==', [16, 8, 8, 7, 16, 7]], ['E0dDadXWHVXGYLxML1QUCw==', [26, 26, 25, 25, 25, 25, 24, 24, 25]], ['E0dc8XHU5JcV2YNkFhxUhA==', [19, 7, 7, 26, 7, 18, 18, 7, 24, 24, 26, 7, 19, 7, 7, 7, 11, 11, 11, 8, 7, 8, 7, 11, 8]], ['E0dlrku8uSc7cFpEYVxznw==', [16, 16]], ['E0eB10uGYr2y8eO0KfH47A==', [7, 23, 23, 3, 19, 7, 19, 7, 7]], ['E0eF0lva/RYiGY5FAal0wA==', [8, 11]], ['E0eXfMkmW5HWB2rqN8cx4g==', [7, 7, 9, 7, 9, 7, 7, 7]], ['E0fhiXPZYc62IlLXEDjP9Q==', [8, 8, 8]], ['E0fqoZ60NHYcmzrsFJEz9A==', [7, 7]], ['E0gOU9wjy9VKAb33lY592g==', [8, 8, 8, 3, 7, 3, 3, 7, 7, 3, 3, 24, 7, 23]], ['E0gSxVkOtRBlng1KMkea3Q==', [28, 28]], ['E0hG7V2crgoK5FKTv+alMA==', [19, 26, 19]], ['E0hrGQDwbkTUBQt8KXyrLQ==', [26, 24, 26, 24, 24, 24]], ['E0hwLJoEiOSWwyZzmMAJlQ==', [9, 24, 24, 24, 24, 7, 7, 9]], ['E0hyRIJit3aACCoZE59Igg==', [21, 22, 22, 22, 21, 22]], ['E0hzZKT3oweb1zjKl/LIgA==', [11, 11]], ['E0i0ywf/aXdbDHKKxSeTHw==', [7, 7, 7, 7]], ['E0i8z/vGrKjHqOHxham9Yw==', [28, 28]], ['E0j9YsCrsulr/MwZwXPNWg==', [7, 21, 7, 26, 21, 26, 21, 23, 23, 8, 8, 7, 16, 16, 26, 26, 7, 7, 7, 7, 8, 7, 3, 15, 7, 7, 3, 3, 27, 15, 7, 15, 15, 7, 11, 21, 3, 21, 27, 11, 21, 8, 26, 15, 8, 7, 7, 7, 16, 7, 26, 16, 7, 21, 21, 15]], ['E0jOsKXmYIwG/biTpZlj8w==', [7, 7, 7, 7, 7, 7, 7, 7, 7]], ['E0jUHZSiKZ9vqSKoMG11Bw==', [8, 8]], ['E0jaDxNk6bL3a2gbHlvI5g==', [8, 8]], ['E0jcwce1x69TC6GLBnzBiA==', [26, 26]], ['E0jcz/NihrP569BjJxbjEg==', [7, 11, 11, 11, 7, 26, 21, 26, 26, 26, 21]], ['E0jlyzc4iVPQ8xAZt4GS5A==', [26, 26, 26]], ['E0jmUmd2IddOy4qv7Dvn8g==', [16, 16]], ['E0kOCx5N2EXJvVq7Lgxy1Q==', [8, 8]], ['E0kUGe7MyaUlk4WXJ0a+ng==', [21, 21]], ['E0kzDjURONI0B7CQW2iy0Q==', [3]], ['E0l/OsBLrKHqClKNhiPilg==', [23, 23]], ['E0llyAFauc2OzZtMEKVstQ==', [26, 26]], ['E0m4JriyRC6s3AygI9xKtw==', [6, 6]], ['E0m5huwHES3Z3PKu7k7P/w==', [15, 15, 8, 8]], ['E0n0cVhIrzNsK83qg3louw==', [25, 25]], ['E0nG2TAzkDV3iLzdIm2ujw==', [8]], ['E0o19CltWGMquiEGGQFuuw==', [26, 26]], ['E0oJncU8ppk83JqHpg2p2g==', [7, 7, 27, 7, 27, 27, 7, 27]], ['E0oP75bqC6j61HHUd7rHew==', [19, 19, 19, 19, 19, 19, 19, 19, 19, 19]], ['E0oy8TprMyeU1ZyJ5ZDmpg==', [8, 6, 6, 8]], ['E0p+VSDRGrSpK8D1gElxBQ==', [3]], ['E0pcAaHyESSvgaii6VDyqg==', [7, 7, 25, 25]], ['E0pxux2/JYOKzJ7uxR3cjA==', [19, 19]], ['E0q8pxafqoo/+oQHPz6rfw==', [7, 19, 16, 7, 19, 16]], ['E0qFnhpt30au4zE4JZPvLw==', [7, 7]], ['E0qPOz4wQiDsP0DgM9h+Zg==', [3, 3, 25, 16, 16, 7, 3, 3, 7]], ['E0qd5HjMsCRjAhnpZf2xNw==', [29, 8, 8]], ['E0qw/ADTmHCo0rOwx0FC4A==', [7, 26, 7, 26, 26, 26, 26, 26, 26, 7, 26, 26, 26, 7, 11, 8, 8, 8, 8, 25, 11, 8, 11, 8, 11]], ['E0qzhNVGFSDreq7rZpK2kQ==', [19, 19, 26, 26, 26, 11, 26, 11, 11, 8, 8, 11]], ['E0rGK4Ar1UUJ4yzhQhl+2Q==', [8, 29, 29]], ['E0rKD+tudkNiMk1VUQSJrQ==', [7, 7, 7, 7]], ['E0rV5a5kR2EKMVmf/8eEdw==', [19, 7, 7, 19, 7, 7, 7, 7, 7, 7, 7, 7, 19, 7, 19, 19, 7, 19, 19, 19]], ['E0rWXWdctVAzHodH7qjHzA==', [16, 7, 16, 7, 7, 18, 7, 16, 3, 7, 7, 25, 7, 26, 3, 7, 3, 3, 3, 18, 26, 8, 8, 8, 8, 3, 3, 8, 26, 25, 7, 3, 11, 26, 26, 26, 3, 3, 19, 7, 7, 25, 7, 11, 3, 11, 8, 3, 7, 3, 7, 26, 26, 19, 28, 28, 7, 3, 11, 26, 11, 18, 8, 8, 8, 16, 8, 16, 8, 8, 3]], ['E0rZRjbha8Zjokaa5jRfDQ==', [3]], ['E0s8+hPeRwdBonRC0cjPsA==', [8, 8]], ['E0szb+nl0DpM4T9BgYoUtw==', [7, 8, 27, 8, 8, 7, 7, 7, 7, 7, 8, 8, 7, 8, 8, 8, 27, 7, 7, 19, 7, 7, 7, 8, 8, 19]], ['E0tN4xL6chENQgSKnvH5gA==', [3, 7, 7, 16]], ['E0theHUG71dicGi43x4VEw==', [19, 21, 21, 19]], ['E0tuYtnrq9xZSj0pHNVJzg==', [26, 26, 21, 19, 18, 26, 7, 7, 7, 7, 7, 21, 21, 18, 7, 26, 8, 26, 11, 8, 8, 19, 11, 26, 26]], ['E0uI4OzxE7vSdTIrRD3w5A==', [7, 7, 7, 7, 7, 26, 26, 26, 26, 7]], ['E0uLleSBL8TKEi4RMEQdLQ==', [14, 14]], ['E0uNVJEWuO4Sn7qVgPsZYA==', [28, 28]], ['E0ufbzhaWDK2K7BbbmZq2w==', [19, 17, 23, 17, 25, 19, 7, 7, 19, 3]], ['E0uop4vwtHTtI6/DRPw69Q==', [26, 26]], ['E0uzcNdZA8uOnCmrxO60vQ==', [3, 3]], ['E0vJC2VNErKdUEZXmDM7Kw==', [7, 7]], ['E0vRQdk3vUZ5g2bRO+Gq/A==', [24, 24, 24, 24]], ['E0vXweS1EG2aI6Y3HH1z+A==', [27, 27, 27, 27, 27, 27, 27, 27]], ['E0vdzqRvUHxGRn2fBUJPeQ==', [26, 8, 8, 8, 8, 8, 27, 8, 27, 26]], ['E0vjOfqknt4bEO+1mB7QEA==', [7, 7, 8, 8, 7, 8, 8]], ['E0w/kNtgGJGUuHZyaTtq/Q==', [29, 7, 7, 29]], ['E0wJdFf3ScMRgb4NwxmbRw==', [7, 7, 8, 27, 27, 7, 15, 15, 15, 27, 7, 21, 27, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 21, 21, 21, 21, 8, 21, 28, 27, 27, 8, 28, 26, 24, 24, 15, 15, 15, 15, 15, 8, 8, 8, 26, 26, 26, 26, 26, 26, 25, 25, 26, 26, 26, 25, 25, 25, 25, 25, 25, 24, 24, 15, 24, 21, 21, 16, 16, 16, 16, 16, 16, 7, 7, 7, 24, 7, 7, 7, 11, 26, 7, 26, 26, 26, 11, 7, 26, 26, 11]], ['E0wSPqy9BzrG8pcW9jYjjA==', [8, 8]], ['E0xSVRzX9q14ue1Ag3tZ2Q==', [7, 7]], ['E0xTXbyDQ1doEsn59XPx8g==', [19, 19]], ['E0xb99sO42m0eYYZANM0Cg==', [7, 7]], ['E0ypRvqBLipriYzPPGEMRg==', [27, 27]], ['E0z/u3Ce3MmpHuKZ2VDo3g==', [8, 8]], ['E0zop/KvrcvLMfhpBh5YuA==', [7, 7, 7, 7]], ['E0zqZIj3Z7eXLweqifXKHw==', [8, 8]], ['E1++j3zY9s3c+deJOb8LXg==', [19, 19, 24, 24]], ['E1+Y/nYXi5zqIwV+9ANvbw==', [8, 8, 8]], ['E1+hUFxK/fjnXF0iADvvXA==', [7, 7]], ['E1+kq4wLwda4poVx83iUsA==', [28, 28]], ['E1+ohxHYiN3uCc2FB1vazg==', [15, 15, 15]], ['E1+qYh8YRpo7TcSSdClZRg==', [6, 6]], ['E1/G8Mcy+yy8fgcIWWOmSQ==', [26, 11, 11, 21, 19, 8, 8, 21, 19, 19, 19, 25, 19, 19, 19, 19, 11, 19, 11, 11, 19, 11, 11, 19, 19, 11]], ['E1/YrDrTixbjfksMP8x9vg==', [25, 8, 26, 8, 11, 25, 11]], ['E1/jF3OAg5+in5+NMgGgeQ==', [17, 17, 24]], ['E10Zgg18DTtxlWrRWC7pTg==', [9, 9, 24, 24, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 24, 9, 24, 9, 9]], ['E10rPBmb5ZqdTp6wl/POfw==', [26, 24, 26, 24]], ['E11RGYqEAzu1WhIYQFmApw==', [27, 27]], ['E11W3fQI+IOJj00obs9qbA==', [25, 25, 26, 8, 18, 26, 25, 8, 8, 18, 25]], ['E11bpXhyjwwN4EmON8izEw==', [26, 26, 26, 26, 26, 26]], ['E11zrRPq0DoggCJigKU8Yw==', [18, 18, 18, 5, 5, 18]], ['E12Km/JPtZrkh1vs52ObmQ==', [16, 7, 7, 7, 16, 7]], ['E12PRF5Ez1lsGOAqQZa6BQ==', [26, 3, 3]], ['E12fTFXFjTPofITKlqLlAw==', [7, 7, 7, 7, 25, 25]], ['E13KT8iVXhleIEVWfU51tQ==', [7, 7, 24, 26, 24, 24, 7, 7, 24, 26, 26, 26, 26, 26, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7]], ['E146PzbgzHd3XDVp1mm5JQ==', [3]], ['E15t5B80sjGY4Y1A1eozkw==', [19, 19]], ['E16+iCcVpXyEXOTTzgtG4A==', [25, 24, 8, 3, 8, 8, 21, 21, 24, 23, 8, 3, 23]], ['E161anVT5sf2S22gLO560g==', [25]], ['E167XyzSN4S6qfBH6GzDsQ==', [6, 6, 6, 6, 6, 6]], ['E16aRXfWNyXol1H/hHKMUQ==', [23, 27, 7, 25, 7, 7, 25, 27, 7, 17, 7, 7, 23, 23, 23, 7, 24, 24, 24, 24, 7, 7, 27, 27, 7, 17, 25, 17, 17, 17, 17, 17, 17, 17, 25, 17, 17, 17]], ['E16lUKj2Kgb5fyQ1YK9TNg==', [19, 19]], ['E178G6K64mrLVw05JGg33w==', [16, 3]], ['E17DFtOJ0XFAfIjjFKaesw==', [26]], ['E17MQUK9lbfrSC0rVh4/Xg==', [23, 23]], ['E17O9KXJGbD1vot5Ex0MQA==', [19, 19]], ['E17b6gWAQvSNHp8gAcU+Gw==', [25, 26, 26, 9, 25, 26, 26, 9]], ['E17x12vks0RmTqlMA8H9gQ==', [7, 7, 7, 7, 7, 7, 7, 25, 8, 8, 8, 8, 25, 7]], ['E188AWFvD9wx0Ncs6imLcQ==', [7, 9, 9, 7]], ['E18D0QmdT58A7P+qtQJJsQ==', [8, 16, 3, 3, 15, 29, 29, 8, 15, 8, 16, 3, 16, 8, 16, 11, 8, 16, 8, 16, 8, 11, 11, 8, 7, 7, 7, 7, 7, 7, 7, 3, 11, 26, 7, 25, 26, 26, 26, 26, 26, 26, 26, 7, 26, 18, 18, 7, 26, 16, 16, 26, 16, 7, 7, 8, 8, 8, 8, 8, 8, 7, 8, 16, 16, 8, 16, 7, 7, 16, 25, 16, 26, 11, 11, 7, 11, 7, 7, 11, 26, 26, 26, 7, 26, 26, 7, 26, 26, 11, 8, 8, 8, 29, 29, 29, 29, 11, 29, 29, 29, 29, 29, 8, 8, 29, 25, 26, 8, 7, 8, 8, 18, 8, 8, 7, 8, 8, 25, 25, 25, 7, 7, 8, 7, 18, 7, 26, 26, 8, 26, 26, 7, 7]], ['E18PCIa34Undkoj+pv5JYg==', [28, 28]], ['E199EDt4f665N0/CyM3AZw==', [8, 18, 8, 19, 19, 8, 8, 18, 18, 18, 19, 19]], ['E19RtXCRIykYB+WIrvo3ug==', [11, 11]], ['E19V+g9KK39J05P5VAnWIw==', [19, 8, 8, 19, 19, 7, 19, 19, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 25, 7, 19, 7]], ['E19yzpHqy4LlV5zy+eNCnw==', [7, 7]], ['E1A4qZTPG33mzGQiWHiK7A==', [23]], ['E1ASGClnKgsri1O5Rrfepw==', [16, 16, 26, 8, 8, 26]], ['E1B8m7V+Lf+lDBJZKicTBQ==', [9, 9, 19, 16, 19, 16]], ['E1BF5DXkCPONVi2K1Vg/ng==', [11, 25, 25, 7, 11, 11, 11, 7, 7, 7, 7, 7]], ['E1BPpUU/r9nC9BzORZyJGQ==', [26, 26]], ['E1BiibIF6sFHb2cd4JETSw==', [19, 19]], ['E1Bq5R83JgyUro3XaP/TGA==', [7, 7]], ['E1ChcM6q6NK3Eg/gbzIaMA==', [26, 26]], ['E1D/VKFliFTKoon+Yo9lVA==', [16, 17, 11, 28, 28, 16, 26, 8, 26, 26, 11, 11]], ['E1DguobDfqDfGf4AQbSBVQ==', [11, 11]], ['E1DriMxZ2YrGf64x+xrlKw==', [28, 28]], ['E1E/mIoRZEO4wGL/quh7Ig==', [26]], ['E1E9vCe1bLok4ZOS3niUIg==', [25, 25]], ['E1EVBw19y1tEwxVbJ8PKnA==', [25]], ['E1EavF5LSAFCqs/u64lEnA==', [14, 14]], ['E1Ew/kb56uI+wE9r3Bb8nw==', [26, 26, 28, 25, 25, 23, 23, 26, 26, 23, 11, 11, 28, 11, 11, 23, 23, 23, 23, 7, 7, 23]], ['E1F022c6RWbMvE9BaLNkXw==', [29, 29]], ['E1F1hth7zSxlvij1Gkk1dg==', [8]], ['E1F4457fQPGj4+wL0xPm6g==', [26, 7, 7, 7, 26, 15, 15, 15, 15, 26, 16, 16, 3, 3]], ['E1F6s87d8ZKNafrxoXskjA==', [9, 9]], ['E1FGSYd0eW6QOOaJbw09+Q==', [15, 15]], ['E1FZ2bji+jSMvl9S8vU7fg==', [8, 8]], ['E1FebL/D+MHq5XcmJ76nnQ==', [7, 7, 7, 7, 12, 12, 12, 12, 26, 26]], ['E1GE8Ttjw2BkiMHLIelJWQ==', [3]], ['E1GH1yH+AD5DRnHLA9Md3A==', [26]], ['E1GR1AHhg4TgjPDIKbDrEg==', [26, 26, 26, 26, 26, 26]], ['E1HXMBrtxRzITpkkiq4H0w==', [29, 29]], ['E1Hxzp6ofkpitBuWITxuBg==', [27, 27]], ['E1I4HLvMEdmplYtvCp28KQ==', [16, 16]], ['E1IDpEu867Yu1HbBcITUHA==', [7, 7]], ['E1Ir4JEfHllqHEf4BYGZkA==', [25, 25]], ['E1IxRbVkAyw5CEr2cGJNXg==', [24, 24, 24, 24]], ['E1J+6g/P/GJwIlfiqEDJ+w==', [8, 8]], ['E1JQGjIgm5oLa4dEsU0lPQ==', [26, 8, 8, 25, 25, 26]], ['E1JYJzC0acEFDKYIGrTCGg==', [28, 28]], ['E1JdnQjNdA34OQpdZ4Dj+w==', [25, 25, 25, 7, 7, 25, 8, 8]], ['E1JeqLajXUGGXPD1MnXU1w==', [25]], ['E1Jn34GewCRmNWSRQY8RuQ==', [24, 8, 8, 11, 7, 7, 16, 8, 11, 11, 11, 7, 11, 7, 8, 8, 8, 24, 8, 16, 11, 8, 19, 11, 7, 7, 25, 19]], ['E1L/9GWwuom20tnF6QMbyg==', [8, 23, 8, 8, 8, 8, 8]], ['E1M69a0F500Tqase+q4F/w==', [23, 26, 26, 26, 26, 25, 25, 23, 23, 26, 23]], ['E1ML07FTFpnbKpH82YAwGw==', [27, 27]], ['E1MYWZjyBC3G8O5AP15Kfw==', [16, 26, 7, 7, 25, 25, 23, 23, 23, 23, 26, 3, 8, 8, 3, 16, 16, 3, 16, 8, 26, 16, 3, 11, 11, 11, 24, 24, 8, 7, 7, 23, 8, 8, 9, 9, 11, 8, 9, 9, 7, 7, 8, 26, 26, 17, 17, 11, 8, 16, 23, 23, 29, 29, 26, 26, 8, 8, 3, 8, 7, 7, 7, 7, 26, 16, 16, 3, 26, 3, 3, 16, 16, 7, 8, 3, 15, 23, 23, 23, 3, 11, 24, 23, 23, 23, 23, 23, 24, 3, 8, 16, 16, 8, 8, 25, 15, 15, 15, 24, 7, 7, 8, 11, 11, 8, 7, 7, 26, 26, 26, 16, 7, 7, 11, 8, 7, 7, 11, 7, 8, 26, 7, 7, 16, 7, 26, 11, 26, 26, 7]], ['E1MZY+NUor4Z+QRj4+IEuQ==', [19, 19]], ['E1Mebijc7P+w9eXmpXS5uA==', [7, 8, 8, 18, 7, 18, 7, 16, 11, 7, 7, 11, 16, 8, 8, 16, 8, 8, 16]], ['E1Mn/gaor7pCfcIQPnDklA==', [8, 8, 8, 1, 3, 8, 8, 8, 8]], ['E1N20GE6WKU4NY999paGjA==', [27, 27, 27, 27]], ['E1NC/nNoM+nKEMNscI8h6A==', [29, 29, 7, 7, 7, 7]], ['E1NGQuU2JtrYLmqemq2mpQ==', [3, 3, 3, 3]], ['E1NHXxZh3Uv1mkrUoDqgZQ==', [26]], ['E1NNQ4IbrhOZ85t5h1CB7A==', [9, 9, 6, 6]], ['E1NxFhvGlRk4uB5Fu97g3w==', [7, 7]], ['E1OFb0Hw33Fex6M4G2FV+Q==', [27, 27]], ['E1P6qHe57d+EeHxSFyZomg==', [11, 11, 11, 11]], ['E1PC6b+5KX+L24lEYySrkg==', [21, 21]], ['E1PN7wKq7DR3qoxXjwnTmw==', [8, 8, 8, 1, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 1, 8]], ['E1PPG3aAce7qmcqHWBg5vg==', [3]], ['E1PRtWDGIshWlJtWsz344g==', [7, 7, 24, 24, 11, 11, 24, 11, 24]], ['E1PwBXhrr/z38lba5ribzA==', [26, 26, 12, 16, 26, 19, 11, 12, 24, 7, 16, 1, 19, 19, 24, 19, 19, 26, 1, 1, 26, 1, 7, 7, 3, 19, 26, 26, 7, 17, 16, 1, 16, 11, 26, 17, 1, 26, 11, 26, 26, 26, 26, 1, 25, 16, 16, 7, 11, 3, 16, 21, 21, 11, 11, 7, 7, 1, 1, 8, 21, 21, 26, 1, 24, 11, 16, 16, 26, 26, 24, 24, 16, 16, 15, 15, 3, 3, 3, 3, 3, 17, 19, 19, 26, 7, 8, 8, 1, 1, 9, 9, 26, 29, 16, 16, 7, 8, 11, 28, 7, 7, 26, 24, 1, 16, 26, 16, 26, 1, 16, 16, 24, 1, 24]], ['E1Q/ocp4isY3zqjdsYDrSg==', [18, 18]], ['E1Q8D7SA/2kQnxwNjg6bYw==', [7, 7]], ['E1QJiJR2U6llgXU7apSSdg==', [11, 11]], ['E1QTuS45tZn+AEXTZPPrZA==', [16, 16, 16, 16, 7, 15, 7, 15, 15, 7, 15, 7, 7, 7]], ['E1R6d0rKmgKBuk81okYFxQ==', [23, 15, 15, 23, 7, 23, 23, 7, 7, 23, 23, 15, 15, 7]], ['E1RIXmnGdJ8raHPAGmIu4Q==', [25, 25]], ['E1S8OOScZWqR6uR5Ot3nrg==', [9, 9]], ['E1SD7FjJAH8Z7JRUdgWBfg==', [26, 7, 16, 8, 7, 7, 26, 7, 24, 8, 11, 11, 16, 24]], ['E1SI0eIEDaAt6/ZSKGmSqg==', [28, 28]], ['E1TGYsjIUeVWwT9zkm2mrg==', [25, 25, 11, 11, 11, 11, 8, 7, 7, 8, 8, 11, 8, 11, 8, 8]], ['E1TMUo38tMKwCY1SueXtJw==', [7, 19, 7, 19, 16, 16]], ['E1TUtBhWnIHZKLTUHYmfJQ==', [9, 5, 5, 9, 5, 5, 5, 11, 11, 5, 11, 11, 5, 5]], ['E1TwXWylGsVR5MRx0HqUdA==', [16, 7, 7, 3, 7, 26, 7, 16]], ['E1UCLtI6LH3HLXVl5nRfrQ==', [3, 3, 3, 3, 3, 3, 23, 23, 23, 23, 23, 23, 3, 3, 19, 19, 16, 16, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3]], ['E1UfPWMmwLL67KjQ+Ahppg==', [11, 8, 8, 8, 8, 8, 8, 8, 8, 11]], ['E1V5tOXqwNwD4MTG2QWDkg==', [3, 7, 16, 7, 24, 24, 3]], ['E1W7xNohHq3ufBypCgc6jg==', [26, 7, 7, 11, 8, 11, 8, 26, 26, 24, 24, 24, 24, 11]], ['E1WK3UcOo3IaawHDeyUNoQ==', [14, 14]], ['E1Wf7VO4BBZBjGR8Luq1wg==', [7, 7, 27, 27, 7, 26, 26, 26, 26, 7]], ['E1WxMuW+KameCpe9kcQ//Q==', [1, 1]], ['E1XQn5iZu52C0fW4WwapXA==', [15, 15]], ['E1Y7hm+2Rxjqm77xk5A/Cw==', [17, 17, 16, 16, 24, 24, 9, 7, 7, 9, 16, 16]], ['E1Y8TxSg9A2Sg0XjDBwvZA==', [26, 26]], ['E1YIzy+K5g3jo5Ill5SM1w==', [8]], ['E1YWfbGO8bYLbwTOB77+Zg==', [8, 8]], ['E1YhlA/GibgngKpaVh93yQ==', [7]], ['E1ZQCT9ptEFAC+XThAGK7g==', [24, 24]], ['E1ZYfwlJga4hCfvu9bfwUQ==', [26, 26, 24, 16, 24, 26, 7, 11, 11, 26, 16, 7]], ['E1aOYyHZL+DoYjofZ9VNeg==', [26, 26, 19, 8, 8, 19]], ['E1ap3aDLh2VBjy6IVQCFCg==', [26, 21, 26, 26, 26, 16, 26, 26, 7, 7, 16, 26, 7, 8, 26, 25, 8, 23, 24, 24, 8, 21, 8, 25, 25, 8, 23, 26]], ['E1bHUf8phrts34rptwKnaw==', [8, 8, 8, 8]], ['E1bQlGCEg2v5xV1GXMdARg==', [7, 25, 8, 8, 7, 25, 8, 8, 7, 25, 7, 7, 7, 25, 16, 21, 8, 8, 8, 11, 21, 16, 16, 16, 8, 21, 8, 3, 21, 21, 21, 7, 8, 8, 7, 8, 3]], ['E1bThfZGqXSm5NsBSBGobg==', [14, 14, 14, 14, 14, 14]], ['E1bWELM60/medJESAHqtag==', [16, 16]], ['E1byNaQofiU2jW1G70lGTg==', [26, 26]], ['E1dJ9GA9U9cGnsa7M+uKIQ==', [6, 6]], ['E1dplUA3qohuz8JgWZcIWQ==', [8, 7, 8, 8, 8, 8, 8, 8, 8, 7, 8, 8, 8, 26, 8, 26, 3, 8, 8, 8, 26, 26, 11]], ['E1eT8wSJ9myMRd/ANOAjdg==', [9, 9, 27, 27, 9, 9]], ['E1eU/PXR2/2ObVd1Qbw+5g==', [19, 19, 19, 19]], ['E1engSDQ6kq7gECy81f1Cg==', [27, 29, 27, 27, 29, 29, 27, 25, 7, 25, 29, 14, 14, 14, 14, 11, 11, 11, 19, 7, 11, 7, 19, 19, 19, 7, 7, 7]], ['E1ezjxw9dwjSa/0tO6PUgQ==', [18, 18, 3, 3]], ['E1f3dN3UyX/BmLVYz1VGGQ==', [26, 26]], ['E1fLIkEz5Vx1/lNc4ADV1w==', [23]], ['E1fOD4gWwZ3f7YEhIuvqLQ==', [11, 11]], ['E1fR30d2KkfPv5zNGZtnYg==', [8, 8]], ['E1ffr5P4WyO5P2t9zA1pLQ==', [7, 24, 24, 24, 24, 7, 7, 7, 7, 24, 24, 7, 7, 7]], ['E1gDVJ/DY1uL5x9O+YgF1g==', [6, 6, 6, 6]], ['E1gjfWDJRyxEUSmHuEusBA==', [7, 7, 7, 7]], ['E1hCr60eGMCw2SmNRfUUyA==', [25]], ['E1hI7ndmy3Rj1Ia02P8h3A==', [26, 26]], ['E1ismNIk27MC6MuwjN4Xjw==', [26, 26, 26, 26, 11, 11, 11, 11, 11, 26, 11, 14, 14, 26, 26, 26, 26, 11, 11, 26, 26, 26, 11, 11, 11, 26, 11, 11, 26, 26, 26, 26, 26, 26, 7, 11, 7, 7, 11, 26, 26, 7, 11]], ['E1jayDTse+JAeqMSc2oEQA==', [7, 7, 3, 7, 7, 8, 8, 8, 29, 29, 26, 7, 26, 26, 19, 16, 19, 19, 19, 19, 16, 26, 19, 7, 23]], ['E1jon500FFkbaqt0cEp5Yw==', [1, 8, 25, 25, 1, 15, 8, 15, 15, 23, 1, 1, 8, 16, 16, 16, 16, 16, 16, 15, 23, 8, 23, 23, 7, 26, 1, 1, 7, 26]], ['E1jwHNz+v/G2x/VcLGRAOA==', [8, 8]], ['E1k/7zyhqLadcTpBKpx/vw==', [11, 17, 11, 11, 19, 19, 17, 17, 11, 17]], ['E1kBfWTbAyzCpxTEtIvFKQ==', [7, 7, 7, 7, 7]], ['E1l8hWrG+gvJBE7lWgMmtw==', [9, 9, 24]], ['E1miD7iOYaNve5NG95kHhQ==', [19, 25, 25, 19, 11, 23, 7, 7, 24, 24, 24, 8, 7, 7, 7, 25, 9, 9, 11, 8, 24, 8, 8, 8, 8]], ['E1nDRvjI5j1NeiQ+pUslWA==', [27, 27, 7, 7]], ['E1nQoYQqSfzweUODervGNg==', [7]], ['E1o0bPwQuPQMe3UVU5KtFA==', [7, 11, 11, 11, 7, 11, 23, 11, 19, 23, 23, 11, 11, 11, 11, 7, 19, 7, 11, 23, 11, 11]], ['E1oBkHfCcHdRhSVFK3ojCQ==', [8, 8, 8, 8, 8, 8]], ['E1oEtZYjCPivRkzhnKLFbQ==', [3, 24, 26]], ['E1p7iGe6VtOxLyZoTc1EMg==', [7, 7, 7, 15, 24, 24, 7, 15, 7, 11, 11, 24, 24, 24, 26, 26, 7, 7, 7, 7, 24, 7]], ['E1pOJS0z3yMQ8FD+Wvqp9Q==', [26, 29, 29]], ['E1qig9SNlH0Ztibbdbm2OA==', [16, 16]], ['E1rcfySTMzgy2YMXgSKUXQ==', [9, 9]], ['E1riCUiCogwu1BeX8+XnTA==', [7, 26, 26, 7, 7, 7, 26, 26]], ['E1rzDChCZQ9ZtkjKoFppqg==', [8, 8, 19, 19, 19, 23]], ['E1sXq86XsoyRzr2o9KtTjQ==', [8, 8]], ['E1t5T2rtqTSKiUrmw0Fu2g==', [8, 8, 8, 26, 8, 11, 11, 26]], ['E1tq91G+MwlVruovT3Gz0Q==', [25]], ['E1u3PsLmnr/QK21wks8rkA==', [8, 9, 9, 8]], ['E1uJIWw4pJiU9FGB5Nt2/A==', [26, 26, 26, 26]], ['E1v/BzV4lsf8nEe6RuIFHQ==', [8, 8, 26]], ['E1vWeap3rHRlskRFAB37sw==', [3, 3]], ['E1vYdIOiPGrFUncGEucbDg==', [4, 1, 1, 24, 7, 24, 7, 7, 4, 28, 28, 23, 7, 23, 12, 16, 16, 12]], ['E1vfJnEI7dlm1qjL4yoM2A==', [8, 26, 26, 8, 16, 16]], ['E1wE3JPIG5k38JwMDi79pQ==', [9, 7, 7, 9, 7]], ['E1wsb54E9Hzqeoq+G1hhyw==', [19, 3, 19, 11, 26, 11, 11, 11, 19, 19]], ['E1xE2AOqiiTav88I1A8DNw==', [7, 3, 8, 7, 7, 7, 8]], ['E1xU29/A6ih+/h7OoIuBDA==', [9, 9]], ['E1y9nF/PSOixZ8jZGSBfiw==', [8, 8, 24, 24, 24, 26, 25]], ['E1yH2BXFQz06t6ixV+dqJg==', [8, 8]], ['E1yWoiH/t8Yp5w5p9UqDbw==', [26, 26, 8, 8, 8, 8]], ['E1zIPBPZxfjMjVdJJj3L9Q==', [3, 3]], ['E2+/Ba/jGQxH9hnbZxPGcg==', [16, 3, 15, 7, 7, 15, 26, 3, 3, 26, 3]], ['E2+ZDWTCCAjchLwplXSoUA==', [21, 8, 21, 8, 21, 21, 16, 7, 15, 7, 7, 7, 7, 7, 8, 8, 15, 16, 15, 15]], ['E2+v04UXPzGO51T3c/elXQ==', [8, 7, 7, 8, 26, 25, 3, 3, 11, 8, 8, 3, 19, 19, 26, 8, 15, 15, 25, 8, 26, 8, 11, 25, 25, 25, 25, 6, 6, 26, 26, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 7, 7, 25, 25, 25, 24, 25, 7, 7, 25, 25, 24, 25, 25, 25, 25, 25]], ['E2+wtVs4j0sdQGD/rC3b7A==', [18, 18, 12, 29, 9, 9, 29, 12, 26, 26]], ['E2/1MYlTWbRvBYTDBt2a6w==', [3, 3, 19, 19]], ['E2/9aBdIr1zXS8yEsD6LIg==', [3]], ['E2/KTL68y5Yz5xYih5teiQ==', [26, 26]], ['E2/OLOm+MyRegG1xIsmSkg==', [7, 3, 26, 7, 7, 19, 16, 16, 26, 19, 23, 23]], ['E2/qKNQCc0oijbqRG9lS4w==', [26]], ['E2/vuirXsfWzEh+d9eNDxA==', [7, 7]], ['E200xK+E9xAe0u6M+fleEA==', [11, 11, 16, 11, 7, 15, 15, 11, 11, 7, 7, 23, 15, 19, 19, 26, 26, 26, 26, 19, 7, 7, 7, 19, 7, 15, 23, 16, 18, 18, 18, 19, 19, 19, 19, 19, 18, 11, 16, 16, 19]], ['E205i9ZmuhjXprUTX08Hpg==', [25, 25, 25]], ['E206bkvPgEkjCFlxIDYj2g==', [25, 25, 25, 25]], ['E20zuMon9sZ65AXS7eikAQ==', [19, 19, 26, 26, 26, 26, 11, 26, 26, 19, 11, 19, 26, 3, 16, 16, 7, 26, 7, 3]], ['E214Q+DtOmcLJW/IIHlpbw==', [8, 8, 7, 8, 27, 27, 7, 11, 8, 8]], ['E21QNthvGuokmHiDXhvfhw==', [7, 7, 7, 7]], ['E21RNrq+p3LC3urLD2+vvg==', [23, 23, 23]], ['E21UqGcA067aTnlDwaj+8Q==', [3]], ['E22p5fWHzEmQuSl9TDUXOQ==', [7, 26, 7, 21, 7, 7, 3, 21, 26]], ['E23eOdwS2MqXtt5dPIu0dQ==', [7, 7, 7, 7, 7, 7]], ['E247O6HR6pTlZL75I2akUA==', [23, 19, 19, 19, 19, 19, 7, 7, 7, 7, 7, 23, 19, 7]], ['E24Asf1vBPg584yvwQFfVw==', [19, 8, 8, 8, 23, 17, 7, 7, 7, 7, 7, 7, 9, 9, 19, 11, 11, 23, 23, 23, 23, 25, 25, 23]], ['E24TD1ZbUAErYzoeqDdIBg==', [25, 26, 26, 26, 26, 25]], ['E24bA5JoG9cyz/IA2q6zHA==', [24, 24, 24, 24]], ['E24xpQYrD7Pkb0wztsWyng==', [19, 19]], ['E254Mg1Wa3ydxrkc2VWDpg==', [18, 8, 8, 26, 18, 18, 18, 26]], ['E25KqmvRyXGeu60IIqSv6Q==', [8, 7, 7, 8]], ['E25TDiB0cEh5OUSoEtL5rQ==', [21, 21]], ['E25UcPOwHLb8rkExP17mSg==', [29, 29]], ['E25VmZHq48i8+UsHhlMYdw==', [8, 8, 8, 8]], ['E25wI9KOKqm4Yf8745ThqA==', [26, 26, 7, 7, 25, 25, 21, 7, 8, 26, 8, 3, 3, 7, 3, 7, 26, 21, 15, 15, 7, 7, 8, 8, 28, 28, 7, 7, 15, 28, 15, 7, 7, 28, 28, 26, 7, 28, 26, 25, 25, 25, 25, 25, 19, 19, 19, 21, 19, 19, 19, 21, 25, 11, 21, 26, 26, 21, 21, 25, 23, 25, 8, 16, 16, 8, 8, 23, 16, 16, 26, 26, 26, 21, 21, 23, 23, 21, 8]], ['E26A1etPXWTHr941hFpeJQ==', [7, 7]], ['E26WnYuAb4u0gwleVBk7aA==', [7, 26, 7, 26, 7, 7]], ['E26bCD7ZTC8xPn2D2Znaxg==', [15, 15, 15, 15]], ['E26x1owYderf/4RIR7+qig==', [19, 11, 11, 27, 19, 27, 9, 27, 9, 27]], ['E26yLZHVy53GQD0ReATBEA==', [11, 11, 11]], ['E27LzJXURLuayKRW3sgSbw==', [7, 7]], ['E27cVN8DjARRlCJpWaf4QA==', [26, 26]], ['E27gv098rtz+XoSRbUiHBA==', [7, 27, 7, 27]], ['E280SiHgWCtw7cLf5vaSsA==', [7]], ['E29KiICmcg11SzTG6Ejwog==', [1, 16, 16]], ['E29X5FA3S0654UKAce9mbg==', [11, 11, 11, 11, 11, 11, 11, 11]], ['E29o+doqyKRJncsiOf8EcQ==', [3]], ['E29p0f3Df1nwbN3ivUVY5Q==', [26]], ['E2ADRTuPjHpuYdqYBfZ9TQ==', [29, 29, 26, 26]], ['E2AHrdX0N2KH1/LbM1EF4w==', [8, 8]], ['E2AacydUyoOIdtL2oCqsYQ==', [26, 26]], ['E2BiVp3x14e4J1pPNJoBLg==', [23]], ['E2BkgM7YayLrK0/MrTMuzg==', [4, 4]], ['E2BuWwW6AOz/D1TSA4Od8Q==', [3, 3, 3, 3, 26, 26, 19, 25, 26, 26, 27, 27, 11, 11, 11, 16, 25, 16, 16, 26, 16, 19, 16, 16, 11, 25]], ['E2C8xa9i6XQGw4G6u6BY7g==', [26, 26, 26, 26, 26, 16, 16, 16, 16, 16, 11, 16, 26, 11, 26, 11, 25, 26]], ['E2Cab9Nz5fZ92Of6cPkxmQ==', [14, 14]], ['E2CvheQe8T31/1cNeYrwhQ==', [7, 7, 26, 11, 11, 17]], ['E2DBt/N8/MvQcCHnar+9YA==', [7, 25, 7, 7, 25, 7]], ['E2DWG4DIPmSTqQ/Kb9OO6A==', [25, 25]], ['E2DaVKXOYGU1idzngptg8g==', [23]], ['E2DmWGP8KfCC4PPneGbW0A==', [7, 7]], ['E2E5G2ShnlYaJ0KtTW1b8Q==', [26, 16, 26, 29, 16, 29, 16, 16, 26]], ['E2EGawt6TYZPwYF1RMsOrA==', [11, 11]], ['E2EiJNrJUQ4+hhiIJ/xAQg==', [29, 7, 7]], ['E2EjfzMncps+eRbpu6H9FA==', [8, 8, 8, 8]], ['E2EvAtR5jmGurJ7yRuqXjg==', [28, 28]], ['E2FCzDxFLUoNzubYxPmCnw==', [19, 26, 19, 26, 19, 19, 9, 8, 9, 9, 9, 26, 16, 16, 16, 16, 26, 8, 8, 7, 7, 7, 8, 8, 8, 7, 7, 7, 7, 7, 8, 8]], ['E2FepsT3P+bwsDC1pZnJhQ==', [27, 27]], ['E2H05ZBqcZvM0KJLXAWZzQ==', [26]], ['E2H0vKXZybgTw1DLfaAQQA==', [28, 28]], ['E2HG8NbcKVFoGYZ6v24pKg==', [3, 3]], ['E2HhQK+ihPFS0JGXzNZxMw==', [7, 7]], ['E2HsZoVmC7CVLFY8UZgvVw==', [16, 19, 19, 16]], ['E2Hug8jX0GYA7cIyHTQVOQ==', [8, 8, 23]], ['E2Hyp3uH/ohLhad270g//w==', [19, 21, 3, 3, 19, 21]], ['E2IJLfAxlXBkQL887xTG1g==', [8, 7, 7, 7, 7, 7, 7]], ['E2IO+qjf603IxUxtqcsThg==', [7, 7, 7, 19, 19, 7]], ['E2IOLHUxk3ySgSVqBqs4+Q==', [19, 19, 17, 17, 8, 8, 8, 8, 8, 8, 8, 7, 7, 7, 7, 27, 27, 27, 8, 27, 19, 11, 7, 19, 7, 26, 26, 7, 7, 26, 11, 26, 29]], ['E2IVf4QEze2FOHp7Fsg6mQ==', [7, 7]], ['E2JUaonHp0/7dHB8p9Ykjw==', [7, 7, 19, 7, 19, 7]], ['E2JdovG9MH2meWiPoogOlg==', [26, 26, 7, 7]], ['E2K3mev6h1zYv2N1tc7t3Q==', [25, 25]], ['E2K5TUVySA1Nf1UjQ3lL6g==', [29]], ['E2K6K2wtNP1+2y1fZeJZYg==', [7, 7, 7, 16, 7, 8, 8, 8, 8, 11, 11, 11, 11, 26, 26, 17, 17, 19, 16, 7, 7, 7, 7, 7, 19, 8, 7, 8, 7, 8, 7, 7, 8, 7]], ['E2KgiE1dakq/75p+Wsj5pQ==', [26, 26, 26, 26, 8, 8, 8, 26, 26]], ['E2L22f9nzKsO3PMqlpr4SA==', [6, 6, 6, 6]], ['E2L98Dw6abREUdAwz6WGew==', [24, 24, 7, 7, 23, 23]], ['E2LnDAhELHMg1jVoe6wfNg==', [25, 8]], ['E2MEQf32Sg+mMlhzbJmNtQ==', [8, 16, 17, 14, 25, 25, 14, 3, 26, 8, 8, 15, 8, 23, 26, 8, 16, 16, 16, 29, 16, 16, 8]], ['E2NPI1DnDJ3P867rdsYt5g==', [25, 14, 14, 14, 25, 14, 25, 25, 7, 7, 7, 7, 25, 25]], ['E2NekXpIP32M1sHgU/ZLCQ==', [9, 9, 9, 7, 7, 9]], ['E2NlQz96Bgat36cjX4+wCg==', [26, 26, 26, 23, 26, 11, 26, 26, 3, 26, 18, 18, 7, 7, 18, 29, 18, 26, 29]], ['E2NwpCIQ8RKe8p+2rycRPQ==', [3, 3, 8, 8, 26, 7, 8, 8, 29, 7, 7, 7, 16, 16, 29]], ['E2OBcMQKbKnKsKOC4Sfm+A==', [16, 26, 16]], ['E2PkVeDtD/GpnI/H/lH1bA==', [16, 11, 11, 11, 11, 16, 9, 9]], ['E2Ptuqd0SnNYu+A6ghNTPA==', [28, 28]], ['E2PxsXZR5mxRirOkr9rbGQ==', [8, 19, 8, 19]], ['E2Q1Kepn8YXk8+lrufWLrQ==', [7, 7]], ['E2Q7R5NA4Yf7Mo+Y8rMLWQ==', [16, 16, 7, 7]], ['E2QNpauoDjHsra7Yn9eunA==', [25]], ['E2QPsJd8+ajJHcN/0aFmOw==', [8, 14, 7, 14, 25, 7, 25, 7, 8, 7, 29, 29, 7, 7]], ['E2Qf6VAjpkYxAYTL9P8ocg==', [8, 8]], ['E2R/OfSiXNTYzXFjOcsjYw==', [6, 6]], ['E2RJhTGWO8bmsI0HsriwPw==', [7, 7, 8, 8, 7, 1, 8, 7, 7, 11, 7, 7, 8, 7, 7, 25, 25, 25, 25, 7, 7, 7, 7, 8, 7, 8, 3, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 3, 8, 8, 8, 8, 8, 8, 7, 24, 24, 8, 3, 8, 8, 11, 8, 11, 8, 8, 26, 7, 7, 8, 8, 11, 8, 1, 8, 8, 8, 7, 8, 8, 8, 8, 3, 8, 8, 8, 8, 8, 25, 25, 8, 8, 7, 7, 26, 26, 25, 11, 11, 11, 11, 25, 25, 7, 8, 8, 26, 8, 7, 8, 8, 8, 8, 24, 24, 8, 8, 8, 8, 8, 8, 8, 8, 8]], ['E2RhZYYhVc3lEVcU+710Bg==', [8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8]], ['E2SWIuU5uCKV9AI3AkZF7w==', [11, 11]], ['E2T0nL2KA6p5Y59cF4PePw==', [7, 7]], ['E2TCoM1RcSe32bTb5oqyCw==', [11, 7, 11, 11, 7, 7, 7, 11, 11, 11, 24, 3, 24, 24, 24, 3]], ['E2TO7zrn0x13b1r9sdr53A==', [27, 7, 7, 7, 7, 8, 18, 18, 8, 25, 27, 7, 26, 25, 7, 26, 26, 16, 16, 26, 7, 7]], ['E2TPRE1kTmk4tjRO2CT2tg==', [19, 26, 29, 29, 25, 19]], ['E2TdFJj6ew4j9U/sOXXTNw==', [7, 7]], ['E2TiPSIJkKqKPgoJYfvCQg==', [8, 8]], ['E2UPFS99bZ9khEU4wbWx1g==', [16, 16, 16, 16, 16, 16, 27, 27]], ['E2UjnHpZMBVLW69L2hg1Fw==', [7, 24, 26, 26, 7, 24]], ['E2UtsSuZQo3ab04VNL5ItA==', [26]], ['E2WIt1LTHCjQ6Zp2ahd2bg==', [27, 27]], ['E2WMiY6Yxo5sPO8FhATUVQ==', [5, 17, 5, 3, 3, 6, 17, 15, 17, 17, 6, 15]], ['E2WY7nWegHErEeUdTgG5oA==', [3, 3]], ['E2Way3OV+ZLmq7wJciWb8g==', [27, 27]], ['E2WcDQuHDdQ1bIaQQG/BUg==', [25, 25]], ['E2Wslgj+kVaOMbCimd88NQ==', [26, 26]], ['E2X0JXi1f6FOBWTSM6DLkA==', [21, 9, 9, 9, 9, 21, 21, 21]], ['E2Xozv5O010txbEzgvbU1g==', [7, 16, 18, 7, 18, 3, 3, 8, 7, 11, 7, 7, 7, 7, 26, 7, 7, 7, 26, 9, 9, 7, 7, 11, 16, 16, 16, 26, 26, 7, 16, 8, 25, 7, 7, 8, 25, 8, 16, 16, 16]], ['E2Y+p0bMYD217q4OfeyK3A==', [7, 19, 8, 7, 7, 16, 16, 19, 19, 8, 19, 19, 19, 24, 7, 24, 24, 19, 19, 8, 7, 7, 7, 7, 7, 7, 7, 21, 21, 8, 8, 8, 21, 21, 8, 19, 19, 8, 11, 8, 11, 8, 26, 27, 7, 16, 16, 16, 3, 8, 26, 26, 26, 19, 24, 19, 3, 3, 3, 26, 24, 8, 8, 8, 8, 7, 7, 7, 7, 26, 16, 16, 24, 3, 17, 3, 3, 3, 24, 11, 8, 11, 17, 24, 24, 27, 27, 27, 11, 24, 16, 11, 3]], ['E2Y7hyJBLre/DrPZfibj4Q==', [8, 25, 23, 28, 8, 28, 23, 7, 7, 7, 16, 8, 7, 8, 8, 16]], ['E2YqB7LdjDbiGthWLUQvBg==', [24, 24, 7, 3, 23, 23, 7, 7, 26, 26, 8, 8, 8, 7, 8, 26, 26, 7, 7, 7, 7, 8]], ['E2YsbMzL5APaDS8A7yL0wA==', [19, 19, 19, 19]], ['E2Z+LpQPrwUcnSs9j2cfBw==', [8, 26, 8, 8, 3, 26]], ['E2Zhk6C7h8SLsdO44y3qZw==', [7, 7]], ['E2a/AcnPoAIbFOVLdf2qyQ==', [7, 7, 26, 16, 4, 16, 3, 26, 14, 19, 3, 26, 26, 26, 14, 19, 11, 11, 11, 11, 26, 11, 26, 26, 11, 3, 26, 16, 26, 16, 8, 8, 26, 26, 26]], ['E2aS9BpSy/gKa42CSU8nag==', [8]], ['E2ak15mSqm8xqBLs30sKNg==', [21, 21, 16, 19, 16, 11, 11, 11, 11, 16, 21, 19, 16, 21]], ['E2bHZyob0YoSJ6DM1347eg==', [19, 19]], ['E2bMQ5VfeCjrc4vrVuaRNg==', [3, 3, 16, 18, 16, 16, 8, 16, 16, 3, 16, 18, 23, 23, 25, 23, 23, 23, 23]], ['E2brx6wgZqIqZRkRfgYAZA==', [24, 7, 7, 24]], ['E2cF40AA3pAaKDxbAR+oRw==', [7, 19, 19, 7, 7, 7]], ['E2cKwU2RLBcr/vpWHVyMVA==', [7, 7, 7, 7, 29, 29]], ['E2cNZncFslIsBhJcciha1A==', [8, 19, 16, 7, 26, 11, 24, 24, 11, 25, 23, 19, 8, 11, 23, 25, 11, 8, 11, 8, 8, 7, 8, 8, 8, 8, 16, 7, 16, 16, 7, 7, 7, 7, 11, 11, 7, 8, 8, 21, 8, 26, 21, 11]], ['E2cTjucJ8VA+foT09e0EsQ==', [7, 7, 3, 15, 19, 19, 7, 15, 15, 26, 8, 16, 16, 19, 26, 26, 3, 11, 16, 11, 11, 3, 11, 25, 25, 11, 26, 11, 11, 25, 8, 24, 24, 24, 7, 8, 11, 11, 3, 16, 16, 7, 7, 19, 19, 26, 26, 26, 26, 11, 11, 7, 26, 3, 7, 23, 24, 21, 21, 21, 3, 26, 26, 3, 24, 24, 23, 23, 7, 19, 21, 21, 21, 21, 26, 21, 7, 16, 7, 25, 11, 11, 8, 8, 8, 8, 11, 11, 26, 25, 26, 3, 3, 8, 8, 7, 8, 3, 8, 16, 16, 7, 16, 16, 7, 7, 7, 26, 8, 8, 26, 7, 11, 11, 8, 26, 23, 3, 16, 7, 7, 8, 24, 8, 23, 8, 23, 16, 16, 11, 24, 8, 11, 24, 7, 26, 16, 11, 15, 26, 8, 3, 15, 15, 15, 26, 8, 8, 2, 15, 8, 26, 8, 24, 15, 19, 19, 27, 3, 25, 3, 19, 7, 3, 27, 19, 7, 25, 19, 19, 8, 16, 25, 16, 8]], ['E2ccYHOWDRYlT0cRdbDWUQ==', [8, 8, 8, 8]], ['E2cwwfgo/VYEkg1HEhI4cg==', [25]], ['E2czkQ6c8cAclmByTqPAxg==', [8, 8, 19, 8, 19, 8]], ['E2dE8ftNf6E8YX0FjnzD9g==', [7]], ['E2dYrO7ZlR9K2AQafTXd/w==', [15, 15, 15]], ['E2dwNwpFhiMI9iVr9lF0dA==', [28, 19, 26, 19, 7, 19, 26, 7, 7, 19, 19, 24, 19, 7, 24, 7, 7, 7, 7]], ['E2dxpNIudXN5OftZYSTJvQ==', [3, 3]], ['E2e2ZTxPbRWqZbaDJX2zyw==', [19, 26, 26, 26, 19, 11, 26, 26, 7, 7, 26, 26, 15, 19, 26, 26, 26, 15, 15, 15, 19, 19, 7, 19, 26, 11, 7, 7, 7, 11, 26, 11]], ['E2eYT9EGA7bZ6q8BHaYQIw==', [6, 6]], ['E2efMofLOvS/lXu9neuAxQ==', [21, 21]], ['E2ex6EFcbccKXWLmaC8ahw==', [3, 3, 7, 3, 7]], ['E2fKlDe4mnqYEl2eKgfH5A==', [8, 24, 8, 24]], ['E2feQt2Fq4D/5VzaHCTmKA==', [17, 29, 29, 27, 27, 27, 27]], ['E2flz86Fmwf+ZODESLZU2w==', [8, 16, 7, 7, 17, 8, 11, 11, 11, 11, 7, 26, 7, 7, 7, 7, 7, 7, 7, 7, 7, 23, 26, 23, 26, 16]], ['E2g+y37CZxt6fYcCnM7OhQ==', [27, 27]], ['E2g74WFTz3GJ0626uERuOQ==', [28, 28]], ['E2gDPLNtWK+U2uYSA1Qv9A==', [24, 8, 24, 25, 6, 25, 6, 25, 8, 25]], ['E2gDoPUGiGvW1O3yAqVBkQ==', [7, 7]], ['E2gJBwv1dEeppV3p8tgvdg==', [7, 7]], ['E2hIIoBX1D4dpJcDq1Nbxw==', [11]], ['E2hW/mLSpg/zoFkXOguqLQ==', [8, 19, 25, 21, 23, 21, 19, 8, 26, 29, 29, 7, 3, 8, 25, 8, 7, 16, 8, 8, 25, 16, 25, 16, 8]], ['E2hop1DG+FnXh3iv8RznqQ==', [3, 8, 25, 3, 26, 26, 8]], ['E2i+Q8YTnanpYSCV2VQxoQ==', [26, 7, 7, 7, 7, 7, 26, 7]], ['E2iDv/oiKrtXEqDXUabXbg==', [26]], ['E2iNpBQGu3eQU7o++MOV8g==', [26, 26]], ['E2iOzrxoRgEy5Yq6znSasQ==', [11, 11]], ['E2ilaow7KcNqT8nsBrtnLA==', [8, 8, 7, 7, 8, 8, 8, 7, 7, 8, 8, 7, 7, 8, 8, 7, 7, 7, 7, 19, 19, 7, 8, 7, 11, 8, 8, 7, 19, 8, 19, 8, 7, 8, 8, 8, 8, 25, 11, 11, 16, 23, 16, 7, 7, 3, 16, 7, 7, 7, 27, 7, 11, 11, 27, 7, 17, 25, 25, 23, 7, 7, 26, 7, 26, 7, 8, 6, 8, 8, 15, 15, 17, 6, 7, 15, 7, 3, 25, 7, 25, 25, 8, 21, 21, 11, 11, 11, 8, 8]], ['E2im6gO/ukcOeqRhHm8zhQ==', [7, 8, 3, 24, 24, 3, 7, 7, 7, 7, 7, 7, 9, 7, 16, 7, 7, 8, 8, 8, 8, 16, 9, 8, 8, 25, 8, 8, 8, 8, 15, 26, 26, 7, 7, 15, 7, 8, 8, 8, 7, 7]], ['E2jrmbGh89zusQtuArsG3A==', [7, 7]], ['E2kD3rEd+sYjEfYHMMO5jA==', [28, 28]], ['E2kD55kLwVAlM3RUqu2D9g==', [18, 18, 18, 18, 19, 19, 19, 19]], ['E2kI/nVk50vIvlVFgS08NA==', [26, 9, 26, 27, 7, 27, 27, 27, 26, 27, 27, 9, 26, 7, 11, 21, 11, 19, 21, 28, 28, 7, 21, 21, 7, 11, 11, 7, 7, 29, 29, 11, 11, 11, 19]], ['E2kNrYTgkB4AgzjS1yLOAg==', [3, 3]], ['E2l6ZoVrL5Ha6Qzt3IOKhA==', [11, 11, 11, 8, 11, 11, 11, 11, 11, 11, 9, 9, 11, 8, 26, 11, 21, 11, 11, 19, 19, 11, 8, 8, 26, 7, 7, 25, 25, 24, 21, 24, 7, 7, 7, 27, 27, 27, 18, 8, 11, 8, 8, 8, 27, 8, 8, 21, 11, 11, 21, 7, 11, 11, 18, 16, 7, 7, 7, 16]], ['E2lNa2mH3AthoeuZx75R7w==', [19, 19]], ['E2lYWc0gYBDHBSSEF71eyg==', [16, 16]], ['E2lp2tLk1VJvVrdEUxVz9A==', [11, 11, 11, 11]], ['E2m47hJWCzkC1BWxgY3tKw==', [11, 11, 11, 7, 7, 11, 11, 11]], ['E2mL2NTbUsPXBlAEw7u8qQ==', [7, 7]], ['E2mjd+dbQj4mzWZuoz4MDA==', [16, 25, 25, 25, 25, 16, 16, 11, 11, 25, 16, 25, 25, 26, 26, 16, 16, 25, 16, 25, 25, 16]], ['E2mkBBes8j4XkwHxpvDlyQ==', [3, 24, 3, 27, 27, 24]], ['E2o6vvcdTElGaExzvjLV3A==', [8, 17, 17, 7, 19, 19, 19, 19, 7, 7, 11, 7, 3, 3, 11, 11, 19, 19, 26, 26, 11, 26, 26, 3, 26, 3, 26, 8, 27, 3, 3, 27, 23, 7, 26, 26, 26, 26, 7, 23, 23, 23, 11, 11, 26, 26, 8, 23, 25, 8, 23, 25, 3, 3, 3, 18, 3, 18, 26, 19, 3, 7, 8, 19, 3, 3, 3, 8, 3, 8, 3, 23, 7, 7, 7, 26, 26, 27, 27, 28, 7, 3, 3, 8, 3, 7, 3, 24, 3, 3, 28, 8, 8]], ['E2oCzyNcAJp4qMelGVg/hQ==', [29, 29, 29, 29, 29, 29]], ['E2oKChbsk2qd+aIIdD3hiA==', [27, 27, 7, 7, 26, 26, 27, 27]], ['E2oRIDQMjiR5QL3ZbP/rnQ==', [15, 15, 7, 7, 7, 7]], ['E2qfvERYZ7lh2uIhCru0fA==', [24, 24, 24, 24]], ['E2rHK3+BDW8/Z3mmzAi2OQ==', [8, 8]], ['E2rXpSqMgFzFIA+rgG6wow==', [27, 27, 27, 7, 7, 7, 7, 27]], ['E2ru3ljK6bjAmWnKla9UeA==', [7, 26, 26, 11, 11, 7]], ['E2ry00vh+MGY3NO+h37n3Q==', [7, 7, 24]], ['E2sIz3h03znj0R97WgtZwQ==', [19, 19]], ['E2sjCcF+a5C7NiarNPCG7w==', [25, 7, 25, 7]], ['E2tWBNX204nCtFY0bvzeDw==', [8, 8]], ['E2tb8E/sjZII2keRtFJ/zg==', [16, 16, 16, 24, 24, 7, 7, 19, 19, 19, 19, 19, 19, 25, 16, 16, 16, 16, 7, 7, 7, 8, 8, 7, 11, 11, 16, 16, 26, 26, 26, 26, 16, 25]], ['E2tmYoUrFK+8tzLLb4oE5Q==', [25, 9, 9, 8, 8, 8, 7, 7, 23, 23, 8, 8, 3, 8, 26, 8, 8, 25, 25, 9, 7, 19, 18, 19, 18, 26, 26, 7, 7, 9, 8, 9, 25, 7, 7, 8, 7, 8, 8, 8, 9]], ['E2u4IKZzhYBLUP2iGf+r2g==', [19, 7, 25, 26, 7, 16, 3, 7, 16, 17, 16, 16, 16, 25, 3, 7, 7, 16, 17, 7, 24, 26, 19, 24, 25, 25, 25, 25, 25, 24, 7, 7, 25, 7, 8, 8, 26, 7, 7, 7, 24]], ['E2u8izQY9TOgCZ2QIzgxTw==', [26, 26, 7, 26, 26, 7]], ['E2uJmAXuyduD4LHcEzBzmQ==', [3, 3]], ['E2uR2Q8nwAQzNa+TO7V86A==', [19, 19]], ['E2ubtZELR4vTanXojTexAA==', [11, 11, 11, 11, 11, 11, 11, 11, 11, 11]], ['E2v/9bKEgs36yzRKq+HDog==', [7, 7]], ['E2v5LXFdfTVnNbjAsuZ5vw==', [11, 11, 7, 7, 11, 8, 26, 19, 26, 3, 19, 3, 26, 8, 11, 11]], ['E2vLoPx2VsxHk1CT85LKbg==', [16, 8, 16, 8]], ['E2vUv3OE9XlyKM4PuWn4Mg==', [26, 7, 6, 11, 11, 7, 7, 7, 8, 6, 8, 8, 11, 8, 11, 11, 23, 6, 6, 6, 11, 11, 11, 6]], ['E2vWPJJvIMNBFsuaFUo2hg==', [26, 26, 26]], ['E2vZtrA6yXkfLxqdUZBcGQ==', [7, 25, 25, 7, 3, 26, 26, 23, 25, 26, 3, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 7, 7, 7, 7, 8, 19, 19, 7, 7, 3, 8, 7, 7, 7, 25, 26, 23, 19, 3, 25, 25, 3, 8, 25, 8, 8, 8, 8, 8, 8, 11, 11, 3, 16, 16, 16, 16, 26, 25]], ['E2vfqwWZ4DY0fP4N50WTyw==', [19, 19, 3, 3, 19, 19]], ['E2y8sNzJz2EEfv8WDu9PrA==', [26, 11, 26, 26, 26, 26, 26, 26, 26, 26, 8, 8, 26, 26]], ['E2yPBwVvhHsSCqD1PusHzA==', [16, 16]], ['E2yVji116rKrw1X83rF9Tg==', [25, 25]], ['E2ydlFw5745o1tNanSNxZw==', [17, 7, 7]], ['E2ywLAw3TvGDTdlU8O5zlg==', [7, 7, 26]], ['E2z+9JwmqRtzOoXyo1nejw==', [27, 27]], ['E2zTj5zcVIoFrW9cqYMlbA==', [8, 8, 8, 8, 3, 8, 8, 3, 8, 7, 25, 25, 8, 8, 7, 7, 8, 3, 3, 7]], ['E2zaEnhyVGX6QJphY/KbSw==', [7, 7, 7]], ['E2zbDUbApXFcGlmEBvcxjA==', [16, 24, 16, 7, 24, 24, 7, 16, 24, 16]], ['E2zs8/40Hxw7kVQJSc4Edg==', [8, 8]], ['E3+Cf49gVwQ3fyCuEmyVWQ==', [27, 27]], ['E3+HemGqrQcdWkxuzTIbUw==', [7, 7]], ['E30jsjD+Qb1sj2itMZuBeg==', [28, 28]], ['E30mJ316woD08ED7dz0K9w==', [8, 7, 7, 8, 8, 8, 8, 8, 21, 21, 8, 7, 8, 7, 8, 8]], ['E30n4JZ+S0fRLtsZv+n1Nw==', [16, 16, 3, 27, 27, 3]], ['E31E4n3kAYid8710d24GHA==', [16, 16, 29, 29, 16, 25, 16]], ['E31SU7IGs4fEcJb3bKKm6w==', [8, 8]], ['E31q42qFGpUhTlf1LKkH3A==', [25]], ['E31tB7R2ADwKhX8V5vwDQQ==', [8, 8, 8, 7, 27, 27, 7, 7, 7]], ['E326Sys/WrrBiDQbzFApng==', [3]], ['E32ez4XI0tW16Y1gHyn/MA==', [7, 7, 7, 24, 24, 24, 24, 26, 23, 7, 26, 23, 3, 3, 16, 16, 26, 16, 3, 16, 3, 1, 1, 26, 16, 16]], ['E32ky5/CYks/GcjTFs+5hA==', [8, 24, 3, 7, 24, 24, 8, 24, 24, 8, 8, 19, 7, 19]], ['E32tcLicrxCFXckIK0ecxA==', [15, 15, 7, 26, 26, 7, 25, 7, 7, 25]], ['E33Gq2dci5eqOs4524DTRA==', [7, 7]], ['E33LXUk0Yg+/qvv+AmBgJw==', [16, 8, 16, 8]], ['E33tl1nEca+WM34+PywjrA==', [7, 24, 25, 25, 11, 19, 19, 19, 19, 7, 19, 11, 8, 7, 11, 24, 8, 8, 19, 7, 7, 8, 7, 26, 26, 25, 25, 25, 25, 8, 26, 8]], ['E340ZaeByczj33nmZdlkaw==', [5, 19, 7, 19, 7, 19, 5, 19]], ['E342P+YmXSoat3NF9UNpvg==', [27, 27]], ['E34IX5BYT2aKUnGeSvYSnw==', [8, 8]], ['E34OPkvVw2hXuWCJkwPQLg==', [7, 7, 7, 7]], ['E34QzX3qp2VWFZxrcb3bhA==', [27, 27, 27, 27]], ['E35+y26OjXXR8KUmJNEYYA==', [23, 23, 16, 16, 25, 26, 26, 8, 8, 25, 11, 11]], ['E35/zoeU5ZeAwdEoEMT2Ug==', [7, 23, 23, 7]], ['E35OkJH7g8Cr2r1P0dhTsw==', [3, 11, 11, 11, 11, 25]], ['E35btH71GvSaH59kyK61aw==', [8, 7, 7, 11, 8, 24, 24, 24, 8, 8]], ['E35fSokWpgQOaBHge8zR9Q==', [28, 28]], ['E35lpwxi8lZ+rCiGBdnnAA==', [3, 3, 16, 16, 3, 3, 3, 3]], ['E35p4lu4ssDDh6FbszjjPQ==', [8, 7, 7, 7, 7, 3, 3, 8, 7, 15, 11, 11, 7, 19, 19, 26, 26, 15, 19, 19, 11, 7, 7, 26, 8, 8, 8, 8, 23, 8, 8, 8, 8, 26, 23, 7, 23, 23]], ['E36RvCC/uBIgfQsNLKROiQ==', [27, 27]], ['E36hQlwGTArnYVxKglkcBQ==', [11, 11, 7, 7, 7, 7, 18, 18, 7, 7]], ['E37f6JK2+4uinIZnwUo6FA==', [8, 8]], ['E37z70das5gbJFhMvq/Cjw==', [26, 7, 7, 28, 28, 24, 24, 24, 24]], ['E3846ZDPGrW7rvY92rPdDQ==', [8, 3, 3, 8, 3, 3, 3, 3]], ['E38vA6BABAYkh+kmswP5nw==', [3]], ['E39+so8buw9jqn1z/9+6dw==', [8, 8, 7, 3, 24, 24, 26, 26, 16, 16, 7, 7, 7, 7, 3, 8, 7, 7, 7, 18, 7, 7, 7, 24, 24, 19, 19, 8, 8, 7, 8, 16, 3, 3, 7, 7, 7, 7, 26, 26, 19, 19, 3, 7, 16, 3, 11, 19, 21, 26, 24, 24, 26, 19, 26, 19, 9, 9, 7, 7, 12, 8, 19, 9, 9, 21, 19, 9, 9, 19, 19, 19, 9, 9, 15, 15, 3, 8, 26, 8, 8, 19, 26, 19, 12, 16, 16, 25, 3, 7, 7, 11, 11, 11, 11, 11, 25, 11, 26, 11, 19, 16, 19, 16, 19, 15, 15, 3, 19, 19]], ['E394RA/8u1+WIwhNLMN2/A==', [19, 7, 19, 19, 19, 19, 25, 25, 7, 26, 26]], ['E3996mmwg6ekw6x5IjmKfw==', [25]], ['E39sDfrVGhMKKF/Vvvit/w==', [11, 26, 26, 7, 7, 11]], ['E3AJyIQUMBT7+2gOxftTSA==', [8, 8, 7, 7, 7, 7, 7, 7]], ['E3AZEfOs1LS/tNb/D3cGpw==', [11, 11]], ['E3Amd8W64cHSoC7kiCWXBQ==', [11, 11]]] CPU times: user 1.95 s, sys: 2.38 ms, total: 1.95 s Wall time: 1.96 s
In [0]:
test = pd.DataFrame(vector)
test[0:3]
Out[0]:
| 0 | 1 | |
|---|---|---|
| 0 | [] | |
| 1 | E++3MFP7eV7wR3rVM6N7sw== | [27, 27] |
| 2 | E++RIWdYSkwflDB68hnIbw== | [26] |
In [0]:
Comutation time for
3.1. RNN model¶
the code is from https://github.com/datasciencesociety/E-Commerce-Case-Recommender-Systems-/blob/master/ShopUp%20recommender%20system.ipynb
In [0]:
# need to prepare the data in the format shown below
final_text = test
#final_text()
In [0]:
from fastai.text import *
data_lm = (TextList.from_df(final_text, cols=[0, 1])
.split_by_rand_pct()
.label_for_lm()
.databunch())
In [0]:
data_lm.save(base_dir +'/data_lm.pkl')
#data_lm.load(base_dir +'/data_lm.pkl')
In [0]:
# see tokenizers
data_lm.vocab.itos[:12]
Out[0]:
['xxunk', 'xxpad', 'xxbos', 'xxeos', 'xxfld', 'xxmaj', 'xxup', 'xxrep', 'xxwrep', ',', '7', '8']
In [0]:
# show how the data looks
data_lm.show_batch()
| idx | text |
|---|---|
| 0 | [ 8 , xxwrep 5 23 , 8 , 23 , 23 , 23 ] xxbos xxunk [ 7 , 7 ] xxbos xxunk [ 16 , 16 , 16 , 3 ] xxbos xxunk [ 19 , 19 , 19 , 15 , 7 , 15 , 16 , 19 , 19 , 15 , 19 , 15 , 23 , 7 , 7 , 7 , 29 , |
| 1 | 7 , 7 , xxwrep 4 3 , 5 , 5 , 18 , 18 , 19 , 3 , 19 , 19 , 7 ] xxbos xxunk [ 7 , 23 , 26 , 7 , 7 , 26 , 7 , 7 , 27 , 27 , 26 , 8 , 26 , 8 ] xxbos xxup xxunk + xxunk [ 11 , 28 , xxwrep 5 11 |
| 2 | , 26 , 7 , xxwrep 6 8 , 7 , 8 , 8 , 8 , 11 , 11 , 26 , 26 , 8 , 25 , 11 , 25 , 11 , 8 , 8 , 11 , 7 , xxwrep 4 8 , 8 ] xxbos xxunk [ 3 , 3 ] xxbos xxunk [ 19 , 1 , 24 , 23 , xxwrep 4 24 |
| 3 | / xxunk [ 8 , 27 , 26 , 11 , 11 , 27 , 26 , 8 ] xxbos xxunk [ 19 , 26 , 26 , 7 , 26 , 26 , 7 , 7 , 19 , 19 , 19 , 7 ] xxbos xxunk [ 3 ] xxbos xxunk / xxunk [ 24 , 24 , 24 ] xxbos xxunk [ 8 ] xxbos xxunk [ |
| 4 | 27 , 7 , 7 , 7 , 27 , 8 , 8 , 26 , 8 , 8 , 8 , 11 , 8 , xxwrep 4 11 , 7 , 8 , 8 , 11 ] xxbos xxunk [ 8 , 8 ] xxbos xxunk [ 28 , 8 , 8 ] xxbos xxunk [ 27 , 27 , 27 , 27 ] xxbos xxunk [ 15 , |
In [0]:
bs=48
learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.3)
Downloading https://s3.amazonaws.com/fast-ai-modelzoo/wt103-fwd.tgz
In [0]:
learn.lr_find()
96.00% [24/25 00:26<00:01]
| epoch | train_loss | valid_loss | accuracy | time |
|---|---|---|---|---|
| 0 | 3.340195 | #na# | 00:01 | |
| 1 | 3.263628 | #na# | 00:01 | |
| 2 | 3.272171 | #na# | 00:01 | |
| 3 | 3.261902 | #na# | 00:01 | |
| 4 | 3.273185 | #na# | 00:01 | |
| 5 | 3.291749 | #na# | 00:01 | |
| 6 | 3.300609 | #na# | 00:01 | |
| 7 | 3.289651 | #na# | 00:01 | |
| 8 | 3.280879 | #na# | 00:01 | |
| 9 | 3.264146 | #na# | 00:01 | |
| 10 | 3.256909 | #na# | 00:01 | |
| 11 | 3.237149 | #na# | 00:01 | |
| 12 | 3.233130 | #na# | 00:01 | |
| 13 | 3.180503 | #na# | 00:01 | |
| 14 | 3.104903 | #na# | 00:01 | |
| 15 | 3.003042 | #na# | 00:01 | |
| 16 | 2.890815 | #na# | 00:01 | |
| 17 | 2.768616 | #na# | 00:01 | |
| 18 | 2.648154 | #na# | 00:01 | |
| 19 | 2.542473 | #na# | 00:01 | |
| 20 | 2.442693 | #na# | 00:01 | |
| 21 | 2.356030 | #na# | 00:01 | |
| 22 | 2.303447 | #na# | 00:01 | |
| 23 | 2.341840 | #na# | 00:01 |
75.00% [3/4 00:00<00:00 2.7030]
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
In [0]:
learn.recorder.plot(skip_end=15)
In [0]:
learn.fit_one_cycle(20, moms=(0.8,0.7))
| epoch | train_loss | valid_loss | accuracy | time |
|---|---|---|---|---|
| 0 | 2.717542 | 2.492427 | 0.373884 | 00:01 |
| 1 | 2.688393 | 2.411179 | 0.378571 | 00:01 |
| 2 | 2.660772 | 2.311985 | 0.395759 | 00:01 |
| 3 | 2.596229 | 2.149124 | 0.410268 | 00:01 |
| 4 | 2.524637 | 1.987439 | 0.431696 | 00:01 |
| 5 | 2.445551 | 1.857130 | 0.456808 | 00:01 |
| 6 | 2.365314 | 1.757391 | 0.488616 | 00:01 |
| 7 | 2.285447 | 1.670905 | 0.519643 | 00:01 |
| 8 | 2.213687 | 1.602738 | 0.543192 | 00:01 |
| 9 | 2.149950 | 1.564555 | 0.553125 | 00:01 |
| 10 | 2.090645 | 1.524688 | 0.566964 | 00:01 |
| 11 | 2.037449 | 1.501929 | 0.573772 | 00:01 |
| 12 | 1.991056 | 1.484127 | 0.578571 | 00:01 |
| 13 | 1.948414 | 1.469810 | 0.583259 | 00:01 |
| 14 | 1.911308 | 1.460003 | 0.585938 | 00:01 |
| 15 | 1.886878 | 1.461029 | 0.586272 | 00:01 |
| 16 | 1.864608 | 1.446378 | 0.587612 | 00:01 |
| 17 | 1.834917 | 1.452825 | 0.589174 | 00:01 |
| 18 | 1.809143 | 1.452881 | 0.588504 | 00:01 |
| 19 | 1.786782 | 1.454805 | 0.587054 | 00:01 |
In [0]:
learn.save(base_dir +'/fine_tuned')
Let's play
In [0]:
# give one cluster and see what it comes
TEXT = "19 , 26 , 26 , 7 , 26 , 26 , 7 , 7 , 19 , 19 , 19 , 7"
N_WORDS = 4
N_SENTENCES = 1
In [0]:
print("\n".join(learn.predict(TEXT, N_WORDS +1 , temperature=0.75) for _ in range(N_SENTENCES)))
19 , 26 , 26 , 7 , 26 , 26 , 7 , 7 , 19 , 19 , 19 , 7 , 7 , 24 ,
In [0]:
Simulated Result¶
IT is recommeding for this example articles 19 , 26 , 26 , 7 , 26 , 26 , 7 , 7 , 19 , 19 , 19 , 7 the next 2 are 7 and 24
In [0]:
4. Need to combine article cluster and user cluster recommendation¶
This is not done
5. Evaluation Metrics¶
In [0]:
In [0]:
def mean_av_pres(y, y_hat, factor):
"""
Compute mean average precision https://www.youtube.com/watch?v=pM6DJ0ZZee0
Input: list1 (list) and list2 (list) plus how many factors (int)
Output percentage (float)
"""
count = 0
recom = []
for i in y[:factor]:
# print(i)
if i in y_hat:
recom.append(1)
else:
recom.append(0)
# print(recom)
coef = map_support_computation(recom)
return (coef)
def distribution_coef(y, y_hat, factor):
"""
Distribution coeficent percentage of what matched divided by all
Example:
12,1,2,3,4
12,6,4,8,9
Compute: 1,0,0,0,1 -> 2/5 -> 0.40
Input:
list1 (list) and list2 (list) of attributes
factor how many factors (int) to consider
Output percentage (float)
"""
count = 0
for i in y[0:factor]:
if i in y_hat:
count += 1
coef = count / len(y)
return coef
def compute_mrr(y, y_hat):
""" Compute MRR https://machinelearning.wtf/terms/mean-reciprocal-rank-mrr/
Input:
y (list) of attribute for particular person
y_hat (list) of attributes
Output:
score (float) like: 0.6 = 60%
"""
rank = []
for i in y:
for index, j in enumerate(y_hat):
if i == j:
rank.append(1 / (index + 1))
Q = len(rank)
if Q == 0:
result = 0
else:
result = (1 / Q) * np.sum(rank)
return result
In [0]:
Conclusion
This case is a great example of what our future thinking should be: the best personalized options for the people, wanting to learn more of the things they care about.
In the process we faced some problems: the 48h time interval is motivating but also not enough; finding people to work with is also a great challenge; the dataset is big, but still possible to compute in a personal computer; the collaboration with the teammates on a distance was also something we had to coup with. However, it all was worth it and it was a good improvement.
There were many problems in the steps but overall the chosen solution gives good results. The whole training process is not going to take a long time, but it all depends on the given time interval.
If you like what we did please follow our ShopUp blog for more details and explanations with video and code or our YouTube channel.
9 thoughts on “ShopUp Datathon2020 – Article recommender case”
Nice idea to use BERT – I would be happy to see what are results from Russian model on Bulgarian text
Please share results from RNN model
I think there were some in the code. Around 58% for RNN after 20 epoch.
Wow we uploaded wrong file. I will update it later today. Thanks
This is a nice article. I like it that there is focus on recent BERT-like transformer models. The approach makes sense, but it is unclear to me what it is exactly. It feels like content-based recommendation. Why not also collaborative? E.g., (shameless self-promotion): http://proceedings.mlr.press/v28/georgiev13.html
1. Why use Russian for BERT?
There is multi-lingual BERT that has Bulgarian:
– https://github.com/google-research/bert/blob/master/multilingual.md
There are also other transformers that support Bulgarian:
– https://github.com/facebookresearch/XLM
There is also Slavic BERT (not soo suitable, as it is for NER):
– https://github.com/deepmipt/Slavic-BERT-NER
2. I do not understand the last evaluation measure…
3. Why is there a need to create clusters? Why not compare user-user or page-page directly?
4. Why use RNN for clustering?
5. I like it that there is visualization, but I could not read much… It is too crowded and unreadable…
6. What are the evaluation results?
7. BTW, are the evaluatiomn measures really “metrics” in mathematical sense? Do they satisfy the 4 axioms to be a “metric”? https://en.wikipedia.org/wiki/Metric_(mathematics)
8. Is time information used for anything?
Preslav as usual such a deep and detailed feedback. Thank you so much 🙂
Directly to your points.
1) Good to know that there is BG. Will use it.
2) Mean Reciprocal Rank – I would prefer MAP, MRR is used to when you have ranking in a list measures the first correct item, so normally is used if you want to predict only one, which is next and use ranking to see where is in the list, if it is 2nd or 3d it makes the magic which is in the formula. I will update the article.
3. I need clusters – because I want to have user interests representations somehow, which can be easier if it is in clusters, especially for sites with more than 50K articles.
4. RNN to predict next cluster of interest, not article they have 2-3 days life, and because it is sequence data. Our patterns as humans are changing so for me clusters are “topic of interests” somehow.
5. If I upload the updated file probably you will see more.
6. Only for RNN is 58% you will see it on new file but not whole logic is implemented with the three metrics mentioned below. I will need like 2-3 hours more 🙂
7. Two of them are – MRR and MAP for sure the other one is made by me 🙂 Here is some info https://stats.stackexchange.com/questions/127041/mean-average-precision-vs-mean-reciprocal-rank
8. It is supposed to be. Mentioned into the diagram. not in code
Excellent work, guys. I like how you formed a whole solution for the problem, and even went the extra mile to include scraping of the original article text.
Bulgarian is indeed missing from most of the pretrained language models, and to retrain one takes a while. Also usage of transfer learning from an existing or similar language (such as russian) would have taken time.
Nice idea to use fastAI language model for the RNN, and thus creating recommendation based on the previously visited ones. Training that would also take a long time – unless you have access to strong GPUs. So just for the record – a newer, faster (x4-x16), version of it, called MultiFit, is available with FastAI v. 2, but the linked notebook is still using the older version (https://nlp.fast.ai/classification/2019/09/10/multifit.html).
Also, very nice and clear writing and the illustrations helped a lot.
Hey thank you Liad 🙂
The article is updated.
Hi guys, I wasn’t able to find the output.csv prediction file – do you know where I can find it. I tried to understand the code (don’t have much experience with Python), and based on my limited understanding of the code the predictions are in the array called “vector”. But that seems to sometimes have multiple predictions. I am likely not understanding something.
Hi, Sergi & Desi,
The think i like most in your work is that you enriched the data with the content of the articles. The data enrichment is a very important step, which, if is made in the beginning of any project would give an advantage.
Including the time in the consideration – the sequence of readings is also interesting for me. Actually, here the hypothesis is that there is a pattern in readings: usually an article raises particular interest and it would be expected that the reader would like to proceed in particular (reading) direction. So, with RNN you account for this hypothesis. Here I would be happy to see more about the investigation – whether accounting for the reading sequence really improves the recommendations…