Cyber crime cases: Confusion matrix or its two types of error.
With the modern advancing age, where the whole technology changes in just a span of 5 years, cyber crimes are also on rise. The introduction of Dark Web and onion sites have hastened the cyber crimes and illegal trading. Also, stealing of sensitive information through cyber attacks is getting prominent in 21st Century. With each passing day, the work force becomes insufficient in fighting against cyber incidents and new solutions are sought. Solutions such as autonomous cyber defense systems, smart cyber security assistant architecture and intrusion detection systems are investigated in the fight against cyber-attacks and crimes. Now, Machine Learning is being used to identify these cyber attacks as it doesn’t rely on a predefined dictionary of viruses like in an anti-virus.
Many algorithms like K-Nearest Neighbour, Artificial Neural Network, Support Vector Machine, Bayesian Net, etc. have been tried and a decent accuracy has been achieved. Now, although accurate but these models were trained on probabilistic output and so A Confusion Matrix is obtained, similar to one in Logistic Regression.
Well, although it is impossible to achieve 100% accuracy but even if one achieves it, in a probabilistic Classification Model- has a drawback. Well, the confusion matrix displays accuracy in 2x2 Matrix
It contains 2 types of errors : Type I error and Type II error.
A type I error occurs when the null hypothesis is actually true, but was rejected as false by the testing.
A type II error occurs when the null hypothesis is false, but was accepted as true by the testing.
Type I error is the most dangerous and compromising.
Here’s a research paper published in 2021 on Anomaly based Intrusion detection by Machine Learning.
Even Here, there are cases when the anomaly is actually there, but rejected by machine depicting everything’s fine (Type I error).
Another case is that when drugs are being tested for a particular disease in medical science using Machine Learning a type I error would cause the appearance that a treatment for a disease has the effect of reducing the severity of the disease when, in fact, it does not. When a new medicine is being tested, the null hypothesis will be that the medicine does not affect the progression of the disease.
Thus, We must not solely rely on Machine Learning for Binary Classification and search for methods to reduce Type I error.