Df ML Anomaly Detection
Streaming Anomaly Detection Solution by using Pub/Sub, Dataflow, BQML & Cloud DLP
About
This section of the repo contains a reference implementation of an ML based Network Anomaly Detection solution by using Pub/Sub, Dataflow, BQML & Cloud DLP. It uses an easy to use built in K-Means clustering model as part of BQML to train and normalize netflow log data. Key part of the implementation uses Dataflow for feature extraction & real time outlier detection which has been tested to process over 20TB of data. (250k msg/sec). Finally, it also uses Cloud DLP to tokenize IMSI (international mobile subscriber identity) number as the streaming Dataflow pipeline ingests millions of netflow log form Pub/Sub.
Open Source Health
- Stars
- 191
- Forks
- 49
- License
- Apache-2.0
- Last commit
- 9 months ago
Resources & Links
Related Categories
Vendor
Google Cloud Platform
Open-source projects on GitHub: Kubectl AI, KHI and Cloud SQL Proxy
Quick Links
Open Source
Stars
191
Forks
49
Contributors
0
Open Issues
16
More by Google Cloud Platform
Related Products
Cloud Run MCP
Cloud Run MCP deployment tool
More from this vendor
Jit Groups
JIT Groups is an open source application that lets you implement secure, self-service access management for Google Cloud using groups.
More from this vendor
EvilnoVNC
Ready to go Phishing Platform
Shared categories
Symbiote
Your target's phone's front and back cameras📸 can be accessed by sending a link🔗.
Shared categories
Reverse Skill
Reverse Engineering / Authorized Penetration Testing / Security Research Skill Router Pack AI-powered routing + On-demand toolchain bootstrapping + Self-evolving knowledge base Supports Claude Code, Kiro, Cursor, Cline, and other AI coding clients 逆向/渗透/安全技能路由包 - AI 自动路由 +…
Shared categories