1. Introduction

Social media platforms have become integral to modern communication, yet the proliferation of automated accounts, commonly referred to as bots, poses a significant threat to the integrity of online discourse [1]. Bots are employed for a wide range of malicious purposes, including dissemination of misinformation, artificial inflation of engagement metrics, and manipulation of public opinion [2]. According to recent estimates, automated accounts may generate up to 47 % of traffic on major social media platforms [3].

Existing approaches to bot detection can be broadly categorized into rule-based, feature-engineering-based, and deep learning methods [4]. While deep learning approaches, particularly graph neural networks, have demonstrated strong performance on structured benchmarks such as TwiBot-22 [5], they often lack interpretability, which is critical for practical deployment where community administrators must justify moderation decisions.

Furthermore, cross-dataset generalization remains a persistent challenge: models achieving 98 % accuracy on older benchmarks may perform near randomly on unseen data distributions [6]. This means that reported quality metrics often reflect the idiosyncrasies of a particular sample rather than the model's ability to detect automated behaviour as such.

The present work addresses these limitations by proposing an explainable bot detection approach based on gradient boosting (CatBoost) combined with SHAP analysis. The key contributions are:

1. a systematically designed feature space of 16 indicators spanning profile, behavioral, and social categories;
2. a two-threshold classification scheme producing actionable verdicts;
3. SHAP-based explanations providing per-account feature attributions; and
4. experimental evaluation on two benchmark datasets demonstrating competitive performance against both classical and deep learning baselines.

2. Related Work

2.1. Machine Learning Approaches to Bot Detection. Supervised machine learning has been the dominant paradigm for bot detection since the early work of Cresci et al. [7]. Aljabri et al. [4] provide a comprehensive survey identifying four generations of bot detectors, from simple classifiers to graph-based deep learning. Nguyen et al. [6] benchmarked 35 supervised learning models across multiple datasets, finding that ensemble methods, particularly Random Forest and gradient boosting variants, demonstrate the most robust cross-dataset generalization, whereas deep learning models showed higher within-dataset accuracy but substantially degraded performance under distribution shift.

Gradient boosting methods have been successfully applied to bot detection in several studies. Shevtsov et al. [8] developed an XGBoost-based framework achieving an F1-score of 0.916 on US 2020 election data. Li et al. [9] combined Particle Swarm Optimization with CatBoost for social spammer detection. More recently, Cheng et al. [10] proposed CB-MTE, a CatBoost framework incorporating semantic and graph features, achieving a macro-F1 of 80.84 % on TwiBot-22.

2.2. Explainability in Bot Detection. The need for explainability in bot detection has been recognized as essential for building trust in automated moderation systems [11]. Shevtsov et al. [8] pioneered the combined use of XGBoost and SHAP, revealing that account age and social activity levels are the most critical discriminators. Tabassum et al. [12] demonstrated that only 6 of 19 commonly used features are sufficient for effective detection, with no statistically significant performance loss. Lopez-Joya et al. [13] applied SHAP to analyze misclassified bots, identifying that low URL usage in tweets is a primary evasion strategy.

2.3. Bot Detection on Russian Social Networks. Research on bot detection in Russian-language social networks is limited but growing. Samokhvalov [14] applied CatBoost to VKontakte data, achieving AUC of 0.91 on a dataset of over 280,000 accounts. Kolomeets and Chechulin [15] introduced novel bot quality metrics using the MKMETRIC2022 dataset, demonstrating that sophisticated VK bots with high-quality profiles are virtually indistinguishable from humans by manual inspection. The latter result underlines the practical demand for automated detection tools supplemented with an explanation of the decision taken.

3. Proposed Approach

3.1. Feature Space Design. For each account a under analysis, a feature vector x(a) ∈ ℝ¹⁶ is computed from data obtained through the social network API. The features are organized into three groups: profile (P1–P6), behavioral (B1–B7), and social (S1–S3). The composition of the feature space is presented in Table 1.

Table 1

Feature space of the analyzed account

Profile features capture static account characteristics. They are robust across time periods and have been consistently identified as strong discriminators in prior work [4, 12]. Behavioral features capture dynamic activity patterns. Low entropy of temporal activity (B4) is a particularly informative signal, as automated accounts tend to exhibit concentrated posting patterns [8]. Social features characterize the social embeddedness of an account, which is difficult for simple bots to replicate [15].

3.2. Classification Model. The classification task is formulated as follows. Given a training set D = {(xᵢ, yᵢ)}, i = 1, …, N, where xᵢ is a 16-dimensional feature vector and yᵢ ∈ {0, 1} is a binary label (0 for genuine, 1 for bot), the goal is to learn a function f: ℝ¹⁶ → [0, 1] mapping feature vectors to bot probabilities.

CatBoost is selected as the primary classifier due to its native support for categorical features, built-in class imbalance handling via scale_pos_weight, and competitive performance on tabular data [10]. The model is an additive ensemble of decision trees:

F(x) = Σ γₜ · hₜ(x), t = 1, …, T, (1)

where T is the number of trees, hₜ is the t-th tree, and γₜ is its coefficient. The bot probability is obtained via the sigmoid function:

p(bot | x) = σ(F(x)). (2)

Training minimizes the weighted log-loss with L2 regularization. Hyperparameters are tuned via 5-fold cross-validation using Optuna, with the search space: tree depth in [4, 8], learning rate in [0.01, 0.1], iterations in [500, 2000] with early stopping patience of 50, and L2 regularization strength in [1, 10].

3.3. Two-Threshold Verdict Scheme. The raw probability output is converted to a categorical verdict using two configurable thresholds: τ₁ (default 0.80) for the bot threshold and τ₂ (default 0.50) for the suspicious threshold:

if p ≥ τ₁ — “bot”; if τ₂ ≤ p < τ₁ — “suspicious”; if p < τ₂ — “genuine”. (3)

This scheme provides community administrators with a nuanced assessment rather than a binary decision, allowing them to prioritize review of suspicious accounts. Threshold configurability matters in practice: communities with different costs of false positives and false negatives require different detection sensitivity.

3.4. SHAP-Based Explanations. For each classified account, SHAP values φⱼ are computed for all 16 features, satisfying the additivity property:

f(x) = φ₀ + Σ φⱼ, j = 1, …, 16, (4)

where φ₀ is the base model value corresponding to the average prediction over the training set. The top-5 features by absolute SHAP value are included in the classification report, with their direction of influence (increasing or decreasing bot probability) and the account's actual feature value. This enables administrators to understand why a specific account was flagged, supporting informed manual review decisions.

4. Experimental Evaluation

4.1. Datasets. Experiments are conducted on two established benchmarks. TwiBot-22 [5] is the current gold-standard dataset, comprising approximately 1 million Twitter accounts with graph structure and multiple annotation types. Cresci-2017 [7] is a widely used earlier benchmark containing genuine accounts and several categories of social bots. Using both datasets makes it possible to assess not only classification quality but also the model's ability to transfer to data of a different origin.

4.2. Baselines and Metrics. CatBoost is compared against Random Forest and SVM with RBF kernel as classical baselines, and against published results for BotRGCN [16] and RGT [17] as deep learning references. All models are evaluated using Accuracy, Precision, Recall, F1-score, and AUC-ROC. F1-score serves as the primary comparison metric, as it jointly captures detection completeness and precision, which is essential under pronounced class imbalance.

4.3. Results. Table 2 presents the classification results on both datasets. On TwiBot-22, CatBoost achieves an F1-score of 0.842, outperforming Random Forest (0.793) and SVM (0.776), and approaching the performance of the graph-based method RGT (0.867), which, unlike the proposed approach, requires full social graph data. On Cresci-2017, CatBoost achieves an F1-score of 0.961.

Table 2

Classification results on the benchmark datasets

Note: “n/a” denotes a value that was not measured.в

Notably, CatBoost demonstrates the strongest cross-dataset generalization: when trained on TwiBot-22 and tested on Cresci-2017 without retraining, F1 drops to 0.723, while Random Forest drops to 0.681 and SVM to 0.614. This confirms the finding of Nguyen et al. [6] that ensemble tree methods offer superior transfer properties, and supports the practical applicability of the approach when the data distribution of the target platform differs from the training one.

4.4. SHAP Feature Importance Analysis. Fig. 1 presents the global SHAP feature importance ranking averaged across the TwiBot-22 test set.

Fig. 1. Global SHAP feature importance ranking (TwiBot-22 test set)

The top-5 most influential features are:

1. account age (P1),
2. standard deviation of inter-post intervals (B2),
3. follower-to-following ratio (P4),
4. entropy of hourly activity (B4), and
5. mean daily posting frequency (B1).

Profile features dominate the top-3, confirming findings of Tabassum et al. [12], while behavioral features occupy positions 2, 4, and 5, indicating that temporal regularity is a strong bot indicator.

Analysis of individual SHAP explanations reveals interpretable patterns. For accounts classified as bots with high confidence (p > 0.95), the most common explanation patterns involve very young account age combined with high posting frequency and near-zero entropy of activity distribution, indicating concentrated automated activity. For suspicious accounts (0.50 < p < 0.80), explanations typically highlight anomalous follower-to-following ratios combined with moderate posting regularity, suggesting more sophisticated automation.

5. Architecture Integration

The proposed model is deployed within BotDetect, a web-based platform for social media bot analysis. The system follows a microservice architecture with four containers: a web frontend (React SPA), a backend API server (Python FastAPI), an ML inference service (CatBoost + SHAP), and a PostgreSQL database. The ML service is isolated to allow independent scaling and GPU utilization for batch inference.

The inference pipeline processes batches of accounts in seven steps:

1. receive account data from the backend;
2. compute 16-dimensional feature vectors;
3. normalize features using training-set statistics;
4. obtain CatBoost probability predictions;
5. apply the two-threshold verdict scheme;
6. compute SHAP values;
7. return results to the backend.

Processing 100 accounts requires approximately 50–200 ms of model inference time, satisfying the platform's non-functional requirement of under 10 minutes for 100 accounts including API data collection.

6. Discussion

The results demonstrate that gradient boosting with engineered features remains competitive with graph neural networks for bot detection, while offering substantially greater interpretability. The key advantage of the proposed approach lies in the combination of strong classification performance with per-account SHAP explanations, which are absent from GNN-based methods. A further practical advantage is that no full social graph is required, since collecting one through public APIs is in many cases infeasible.

A notable limitation is the reliance on features extractable from public API data, which may not capture all behavioral signals available from full platform access. Additionally, as Ferrara [2] notes, the emergence of LLM-generated bot content is likely to diminish the discriminative power of content-based features over time, suggesting that the profile and behavioral features emphasized in this work may prove more durable.

The configurable two-threshold scheme addresses a practical gap identified in the analysis of existing solutions (Brand Analytics, Kribrum, YouScan), none of which provide adjustable detection sensitivity or transparent feature-level justifications for verdicts. Yet it is precisely these properties that determine whether automatic classification results can serve as grounds for moderation decisions.

7. Conclusion and Future Work

This paper presented an explainable approach to social media bot detection combining CatBoost gradient boosting with SHAP-based interpretability. A 16-feature space spanning profile, behavioral, and social indicators was designed, and a two-threshold verdict scheme was proposed for nuanced classification. Experimental evaluation on TwiBot-22 and Cresci-2017 benchmarks demonstrated F1-scores of 0.842 and 0.961 respectively, with superior cross-dataset generalization compared to classical baselines.

SHAP analysis revealed that account age, posting interval regularity, and follower-to-following ratio are the most discriminative features, providing interpretable explanations that support administrator decision-making. The approach is integrated into the BotDetect platform, satisfying requirements for performance, transparency, and configurability.

Future work includes: (1) expanding the feature set with graph-based indicators to bridge the gap with GNN methods; (2) evaluating on VKontakte-specific datasets (MKMETRIC2022); (3) investigating adversarial robustness against feature-aware evasion strategies; and (4) incorporating temporal feature drift detection for model retraining triggers.

References:

1. Cresci S. A Decade of Social Bot Detection // Communications of the ACM. — 2020. — Vol. 63, no. 10. — P. 72–83. — DOI: 10.1145/3409116.
2. Ferrara E. Social Bot Detection in the Age of ChatGPT: Challenges and Opportunities // First Monday. — 2023. — Vol. 28, no. 6. — DOI: 10.5210/fm.v28i6.13185.
3. Rauchfleisch A., Kaiser J. The False Positive Problem of Automatic Bot Detection in Social Science Research // PLOS ONE. — 2020. — Vol. 15, no. 10. — Art. e0241045. — DOI: 10.1371/journal.pone.0241045.
4. Machine Learning-Based Social Media Bot Detection: A Comprehensive Literature Review / M. Aljabri [et al.] // Social Network Analysis and Mining. — 2023. — Vol. 13. — Art. 20. — DOI: 10.1007/s13278–022–01020–5.
5. TwiBot-22: Towards Graph-Based Twitter Bot Detection / S. Feng [et al.] // Advances in Neural Information Processing Systems 35: Datasets and Benchmarks Track. — 2022. — DOI: 10.48550/arXiv.2206.04564.
6. Supervised Learning Models for Social Bot Detection: Literature Review and Benchmark / H.-D. Nguyen [et al.] // Expert Systems with Applications. — 2024. — Vol. 238. — Art. 122217. — DOI: 10.1016/j.eswa.2023.122217.
7. The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race / S. Cresci [et al.] // Proceedings of the 26th International Conference on World Wide Web Companion. — 2017. — P. 963–972. — DOI: 10.1145/3041021.3055135.
8. Identification of Twitter Bots Based on an Explainable Machine Learning Framework: The US 2020 Elections Case Study / A. Shevtsov [et al.] // Proceedings of the International AAAI Conference on Web and Social Media. — 2022. — Vol. 16. — P. 956–967. — DOI: 10.1609/icwsm.v16i1.19349.
9. Social Spammer Detection Based on PSO-CatBoost / S. Li [et al.] // Lecture Notes in Computer Science. — Cham: Springer, 2021. — Vol. 12382. — DOI: 10.1007/978–3–030–68851–6_28.
10. CB-MTE: Social Bot Detection via Multi-Source Heterogeneous Feature Fusion / M. Cheng [et al.] // Sensors. — 2025. — Vol. 25, no. 11. — Art. 3549. — DOI: 10.3390/s25113549.
11. Social Botomics: A Systematic Ensemble Machine Learning Approach for Explainable and Multi-Class Bot Detection / I. Dimitriadis [et al.] // Applied Sciences. — 2021. — Vol. 11, no. 21. — Art. 9857. — DOI: 10.3390/app11219857.
12. How Many Features Do We Need to Identify Bots on Twitter? / F. Tabassum [et al.] // Information for a Better World: Normality, Virtuality, Physicality, Inclusivity (iConference 2023): Lecture Notes in Computer Science. — Cham: Springer, 2023. — Vol. 13971. — DOI: 10.1007/978–3–031–28035–1_22.
13. Why a Bot is Undetectable? An Explainability-Based Study of Misclassified Automated Accounts / S. Lopez-Joya [et al.] // Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU 2024): Lecture Notes in Computer Science. — Cham: Springer, 2024. — DOI: 10.1007/978–3–031–73997–2_5.
14. Samokhvalov D. I. Machine Learning-Based Malicious Users' Detection in the VKontakte Social Network // Proceedings of the Institute for System Programming of the RAS. — 2020. — Vol. 32, no. 3. — DOI: 10.15514/ISPRAS-2020–32(3)-10.
15. Kolomeets M., Chechulin A. Experimental Evaluation: Can Humans Recognise Social Media Bots? // Big Data and Cognitive Computing. — 2024. — Vol. 8, no. 3. — DOI: 10.3390/bdcc8030024.
16. BotRGCN: Twitter Bot Detection with Relational Graph Convolutional Networks / S. Feng [et al.] // Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). — 2021. — P. 236–239. — DOI: 10.1145/3487351.3488336.
17. Heterogeneity-Aware Twitter Bot Detection with Relational Graph Transformers / S. Feng [et al.] // Proceedings of the AAAI Conference on Artificial Intelligence. — 2022. — Vol. 36. — P. 3977–3985.
18. Botometer 101: Social Bot Practicum for Computational Social Scientists / K.-C. Yang [et al.] // Journal of Computational Social Science. — 2022. — Vol. 5. — P. 1511–1528. — DOI: 10.1007/s42001–022–00177–5.