Telecommunications Traffic Behavior: Analysis and Implication - Gordana Djankovic

Gordana Djankovic, Data Science Lead and PO, gdjankovic@telesign.com
Marko Mitic, Data Scientist, mmitic@telesign.com
with Data Science
Telecommunications Traffic Behavior: Analysis and Implications

© 2017 TeleSign
• Why Data Science in TeleSign?
• TeleSign internal and external data.
• Math mechanisms used for end-user classification: Supervised vs Unsupervised.
• Universal Risk Score - TS Fraud Detection System.
• New telecommunication data in TeleSign.
• Malicious vs Genuine behavior detection.
• Algorithms, Tools and Examples.
Main goals
2

© 2017 TeleSign
• DS team is a part of Product department in TS, in charge for the
– analysis of internal and external data
– design, implementation and pre-sales of data (science) products
– design, implementation and automation/improvement of internal processes.
• DS team tightly collaborate with
– business teams
– engineering department
– sales teams
– client services teams
– TS customers and potential customers
– external data vendors…
• DS Team is R&D, business driven, customer oriented team.
DS organization in TS
3

© 2017 TeleSign
• to formulate ideas for potential projects based on the data available internally,
• to make decisions about the potentially meaningful and valuable projects
(decisions about the investments),
• to reject unrealistic assumptions and “obvious failures”,
• to detect existing defects in the business and as well as room for the business
improvements.
Business purposes of DS in TS:
4

© 2017 TeleSign
• to define set of tools, languages, techniques, etc.
• to collect and evaluate internal data (internal data inventory and ranking),
• to discover, collect and evaluate external data (external data inventory and
ranking),
• to design and implement data products and solutions,
• to design and implement testing, monitoring and validation processes and tools
around the data solutions…
In order to get the best results DS team is focused on combination of expert
knowledge and data-driven methods.
R&D purposes of DS in TS:
5

© 2017 TeleSign
During the process of implementation of the Data Science, the most
common problems, which we observed, were
• non-existence of the „proper“ data,
• non-existence of the „proper“ experts and data-driven decisions makers to
produce high-quality output from the data,
• non-existence of modern technologies, which allow us to extract structured
attributes/variables/predictors from the raw data in order to make the data
usable in the Data Science algorithms,
• lack of understanding of the complementary properties of all the previous
resources.
Most common issues during the “implementation” of DS
6

© 2017 TeleSign
1. Data!!!
2. Ability to detect useful information from the data.
3. Ability to adopt and adapt new DS trends internally.
4. Ability to present new DS company’s products externally.
Conclusion: Necessary to make initial investments in all the resources listed above to
build data-driven system that will be beneficial for the company on a long term.
“Must have” for successful application of DS
7

© 2017 TeleSign
• TS provides services of A2P communication via SMS and Voice traffic
(e.g. 2FA).
• Core data points in TS are phone number attributes and behavior.
• Additional important data points are e.g. IP and email address attributes
and behavior.
• Labeled data.
Data in TS
8

© 2017 TeleSign
• The raw data is presented in the form of CDRs (Call Detail Record), containing
basic information about transaction and subscriber, such as
– original phone number
– type of transaction
– time of transaction…
• Enriched raw data contain additional info like
– dial plan data
– phone number and country black lists data…
Internal and external data in TS
9

© 2017 TeleSign
During the initial analysis of TS traffic data we’ve observed a significant amount of
fraudulent transactions and malicious attempts in the form of
• fake and bulk account creation,… - using false identity to subscribe a service
without intention to pay for the service,
• account takeover - using the service without having necessary authority,
• call/SMS spamming and phishing,
• VoIP fraud,
• international revenue share fraud (IRSF),
• promo abuse…
Types of Telco and/or Internet fraud observed in TS traffic
10

© 2017 TeleSign
What is the cost of the fraud?
11
Large profits, expansion of information technology and modern ways of
communication caused the growth of a well–organized and well–informed
community of fraudsters, causing huge financial losses every year all over the
world.
*Communication Fraud Control Association

© 2017 TeleSign
• Based on all the available data, initial research results and market needs TS has
made the decision to invest in FDS development.
• FDS represents a set of mechanisms to identify fraud in the shortest possible
time interval from the moment it was committed.
• FDS in TS has become continuously evolving project, based on analysis of
massive data, data mining, and application of various mathematical, statistical
and ML models and methods, because
– when a detection method is disclosed, fraudsters will adapt and/or adopt new
methods for attack and
– in addition to existing, new fraudsters are continuously appearing, and unaware of
the existing detection methods, apply the "old" methods of attack.
– Thus, FDS = old/existing fraud detection methods+ new/updated fraud detection
methods.
Fraud Detection System (FDS)
12

© 2017 TeleSign
• If the target variable that needs to be described by DS methods is available in
historical data, we’re dealing with the supervised problem.
• Supervised techniques provide tools to predict target variable.
• Supervised methods used in TS: various classification algorithms (LR, DT, RF, NN,
SVM...)
• If the target variable is not available, we’re dealing with the unsupervised
problem.
• Unsupervised techniques produce splitting of the data into the subgroups based
on particular similarities in the data, which do not guarantee value for observed
business problem!
• Unsupervised methods used in TS: clustering, profiling, exploratory data
analysis...
Supervised vs Unsupervised methods
13

© 2017 TeleSign
URS is a combination of
• unsupervised methods (clustering) used to deliver end-user segmentation, based on the
traffic behavior/patterns, fraud type, industry type, region, etc.
• supervised methods (multinomial classifiers) used to deliver final probability/score that
the end user is fraudulent.
URS provides probability that the end-user is fraudster scaled to the interval
(0,1000]:
• (800,1000] – high risk
• (600,800] – medium-high risk
• (400,600] – medium risk
• (200,400] – medium-low risk
• (0,200] – low risk
Universal Risk Score - TS fraud detection system
14

© 2017 TeleSign
What happens in the background
15
TeleSign
Transactions

© 2017 TeleSign
• The key investment in the DS (beside the data and “expert” resources) is the
investment in the target variable data, i.e. labeled data!
• Expenses of labeling are common and significant, whether it is
– internal labeling (spending time and human resources), or
– external labeling (getting the labeled data from the customers).
• Labeled data should be always double checked in order to prevent model
development with inaccurate labels (e.g. customer is not able to confirm
legitimate behavior, customer could not detect a significant portion of fraud,
etc.).
• Custom Score models, based on various supervised classification methods, are
developed only for the customers with reliable continuous (weekly/monthly)
labeled feedback.
Universal Risk Score vs Custom Score Solution
16

© 2017 TeleSign
• detects the universal fraudulent patterns common for all customers
• detects behavior patterns using ML methods and techniques
• decreases false positive rate (FPR)
• provides solution for
– all potential/new customers during the test phase
– small/non-enterprise customers
– customers not able/not willing to provide labeled data for further customization
• provides adaptive frame for further improvements, such as
– application and implementation of new data sources/data points/patters (e.g.
new telco data)
– fast and clear integration of custom models
– implementation of legitimate end-user detection…
URS advantages:
17

© 2017 TeleSign
• In 2017 TS was acquired by Belgian company BICS, which operates in the
(international) telecommunication industry (leading voice carrier,
roaming and signaling provider, etc.).
• Huge amount of telco traffic data (international SMS, Voice, mobile data,
signaling traffic, roaming traffic, premium and special services phone
number DBs) has become available to TS.
New Telco data in TS
18

© 2017 TeleSign
Analysis of BICS telco traffic behavior resulted with the following subscriber
classification:
• Non-human or bad-human subscribers
– machine-generated, applicative or call center traffic
– fraudulent traffic (IRSF, VoIP fraud, Voice/SMS spamming…).
• Legitimate subscribers.
• Unknown subscribers.
Classification of Telco traffic
19

© 2017 TeleSign
• Machine Learning algorithms:
 Logistic regression,
 Tree based methods,
 Neural Networks,
 Class-imbalanced algorithms
 Clustering methods (k-means, GMM, PAM,…)
 Advanced graph based solutions…
Algorithms and tools
20
Var 1 < 1.2
Var 2 < 2.5
Var 3 > 5 Var 4 < 10.1
Var 5 > 5.4 Var 6 > 2
• Tools used:
 AWS, DataBricks (Spark)
 MS SQL, S3
 R, Python, Scala, SQL

© 2017 TeleSign 21
DATABRICKS NOTEBOOK – COLLABORATIVE WORKSPACE
USAGE OF MULTIPLE PROGRAMMING LANGUAGES
Interactively query large-scale data sets in R, Python, Scala, or SQL.
MACHINE LEARNING: Mllib, GraphX and GraphFrames
Traffic cleansed data
Variables
DATA FLOW
1. Ingest 2. store 3. Prep 4. Data
Exploration

© 2017 TeleSign
Graph based traffic analysis
23
• Degree based analysis • Triangle counting
• Link distance to fraud • Graph cliques
• Strongly connected components, …
Fraud

© 2017 TeleSign
• Implementation of DS is hard but very valuable investment.
• Data is a gold mine if the data informativeness is extracted and used in the
right way.
• Fraud is not a "needle in a haystack” problem anymore.
• Legitimate end-user detection upgrade.
• Common DS tools and techniques used in TS.
Summary
25

Telecommunications Traffic Behavior: Analysis and Implication - Gordana Djankovic

Recommended

Recommended

More Related Content

Similar to Telecommunications Traffic Behavior: Analysis and Implication - Gordana Djankovic

Similar to Telecommunications Traffic Behavior: Analysis and Implication - Gordana Djankovic (20)

More from Institute of Contemporary Sciences

More from Institute of Contemporary Sciences (20)

Recently uploaded

Recently uploaded (20)

Telecommunications Traffic Behavior: Analysis and Implication - Gordana Djankovic

Editor's Notes