Group 3 SE1941 CSI104 SP2024 Decision Tree Classification Algorithm​

Decision Tree Classification Algorithm​

GROUP 3

Vo Truong Phu

Tran Phuoc Tai

Chung Gia Bao

Tran Hoang Phuc

Vu Duy

INDEX

1. Abstract

2. Introduction

3. Method & Material

4. Results

5. Discussion

6. Conclusion

7. References

ABSTRACT

1. ABSTRACT

• The application of classification techniques in data mining has significantly impacted the field of economics, offering insights into market trends, consumer behavior, and financial risk management.
• This report explores the varied applications of classification in economics, emphasizing its role in enhancing decision-making processes, forecasting economic conditions, and identifying key factors influencing market dynamics. Through the use of algorithms such as Decision Trees, Support Vector Machines, and Neural Networks, economists and financial analysts can predict economic outcomes, segment consumers, and assess credit risks more accurately.

INTRODUCTION

2. INTRODUCTION

K-nearest Neighbors

Decision Trees

Naive Bayes

Here, the classification of data mining is robust. The flowchart it includes resembles the structure of a tree with the classes hanging on leaf nodes with labels. The internodes contain decision classification algorithms in data mining. These are routed to the neighboring leaf node.

It includes non-linear prediction boundaries. This is due to K-nearest Neighbors falling into the category of non-linear classifier. The classifier uses the k nearest neighbours class to make classifications and predictions in data mining concerning a new test data point.

This algorithm assumes that each parameter standing on an individual foot will have equal effects on the results. That is why they are also equally important. Naive Bayes estimates the event probability that it is to occur.

2. INTRODUCTION

Random Forests

Support Vector Machines

Neural Networks

Also called SVM, it reflects the in-space training data. These are segregated into distinctive categories using large gaps. Then, new data points are identified in this space, following which the prediction of categories is conducted with a focus on the gap side they belong to.

This classification of data mining method obtains the input and then learns to identify its patterns. This helps neural networks to make output predictions for similar new inputs.

This one is compatible with numerous decision trees on diverse subsamples of databases. Here, the average is implemented to improve the accuracy of its predictions and administer overfitting.

2. INTRODUCTION

Ensemble Method

It combines diverse models to empower the outcomes of machine learning. This process leverages better predictive performance production when in comparison to a singular mode.

METHOD & MATERIAL

3. METHOD & MATERIAL

Root Node: The root node is from where the decision tree starts. It represents the entire dataset, which further gets divided into two or more homogeneous sets.

Click the image for fullscreen

3. METHOD & MATERIAL

Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated further after getting a leaf node..

Click the image for fullscreen

3. METHOD & MATERIAL

Splitting: Splitting is the process of dividing the decision node/root node into sub-nodes according to the given conditions.

Click the image for fullscreen

3. METHOD & MATERIAL

Branch/Sub Tree: A tree formed by splitting the tree.

Click the image for fullscreen

3. METHOD & MATERIAL

Pruning: Pruning is the process of removing the unwanted branches from the tree.

Click the image for fullscreen

3. METHOD & MATERIAL

Parent/Child node: The root node of the tree is called the parent node, and other nodes are called the child nodes.

Click the image for fullscreen

How does the Decision Tree algorithm work?

• In a decision tree, for predicting the class of the given dataset, the algorithm starts from the root node of the tree. This algorithm compares the values of root attribute with the record (real dataset) attribute and, based on the comparison, follows the branch and jumps to the next node.
• For the next node, the algorithm again compares the attribute value with the other sub-nodes and move further. It continues the process until it reaches the leaf node of the tree.

How does the Decision Tree algorithm work?

How does the Decision Tree algorithm work?

The complete process can be better understood using the below algorithm:

• Step-1: Begin the tree with the root node, says S, which contains the complete dataset.
• Step-2: Find the best attribute in the dataset using Attribute Selection Measure (ASM).
• Step-3: Divide the S into subsets that contains possible values for the best attributes.

How does the Decision Tree algorithm work?

• Step-4: Generate the decision tree node, which contains the best attribute.
• Step-5: Recursively make new decision trees using the subsets of the dataset created in step -3. Continue this process until a stage is reached where you cannot further classify the nodes and called the final node as a leaf node.

RESULT

4. RESULT

In the banking sector, Decision Trees are utilized for a variety of complex financial decisions, extending beyond traditional risk management and risk analysis. They enable compact decision-making in various customer service scenarios, addressing challenges like data protection threats, lack of personalization, limited personnel, and absence of self-service options. For instance, banks face significant customer service challenges, such as apprehensions about data sharing and agents not having the right solutions. Decision trees aid in overcoming these challenges by enabling positive interactions for both customers and banking agents, thus elevating the banking customer experience (CX).

4. RESULT

A practical application of decision trees in banking is illustrated by the process of innovating new banking products. For example, developing a new product involves multiple stages, each with its own set of risks and uncertainties. Decision trees help in quantifying the risks and returns at each phase, providing a structured framework for decision-making. This involves breaking down the process into phases, defining decisions at the end of each phase, estimating probabilities of outcomes, computing cash flow, and then calculating the value of each path by "folding back the tree" to get an expected value for the entire process. This methodology not only aids in managing the risk but also in optimizing the decision-making process for product development within banks​​.

Click the image for fullscreen

4. RESULT

Furthermore, decision trees facilitate self-service in banking, reducing wait times and enabling independent service for customers. For example, self-service portals powered by decision trees can guide customers through processes like sending money or applying for credit cards without the need to interact with a bank agent. This approach builds customer trust, as it allows for a more streamlined and efficient service, contributing to a better overall customer experience​​.

4. RESULT

These examples demonstrate the practical application of Decision Tree Classification Algorithms in the banking sector, showcasing how they contribute to both risk management and enhancing the customer experience by providing structured, efficient solutions to complex decision-making problems.

DISCUSSION

Disadvantages The decision tree contains lots of layers, which makes it complex. It may have an overfitting issue, which can be resolved using the Random Forest algorithm. For more class labels, the computational complexity of the decision tree may increase.

Advantages It is simple to understand as it follows the same process that a human follows while making any decision in real life. It can be very useful for solving decision-related problems. It helps to think about all the possible outcomes for a problem. There is less requirement for data cleaning compared to other algorithms.

VS

CONCLUSION

6. CONCLUSION

The Decision Tree Classification Algorithm is a powerful tool in the realm of data mining, offering a straightforward and intuitive means for solving classification problems across various sectors. This algorithm excels in its ability to break down complex decision-making processes into simpler, manageable parts, making it invaluable for both predictive modeling and data analysis tasks. Its broad applicability spans from the financial sector, where it aids in risk assessment and customer segmentation, to healthcare, where it helps diagnose diseases and predict patient outcomes.

6. CONCLUSION

Key strengths of the Decision Tree Classification Algorithm include its simplicity for users to understand and interpret the generated models, its flexibility in handling different types of data, and its capability to model non-linear relationships. However, it's crucial to acknowledge potential limitations, such as the risk of overfitting and the challenge of dealing with very large datasets, which can be mitigated through techniques like pruning and ensemble learning.

6. CONCLUSION

In conclusion, the Decision Tree Classification Algorithm remains a cornerstone technique within data science, continually proving its worth through its adaptability, ease of use, and effectiveness in extracting meaningful insights from data. As the field of data science evolves, the decision tree's foundational principles will undoubtedly continue to influence the development of new methodologies and technologies, securing its place as a vital asset in the data analyst's toolkit.

REFERENCES

7. REFERENCES

• Han, J., Pei, J., & Kamber, M. (2011). Data Mining: Concepts and Techniques. Elsevier.
• Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning. Springer.
• Varian, H. R. (2014). Big Data: New Tricks for Econometrics. Journal of Economic Perspectives.
• SouthState Correspondent Division. (2021). "Using Decision Trees in Banking (And Why Innovation Makes Economic Sense)."
• Knowmax. (2023). "Decision Trees In Banking: Elevate Banking CX With Guided Workflows."

THANKS!