Splunk

Splunk is a data analytics and monitoring platform that ingests, indexes, and analyzes machine-generated data (logs, metrics, traces) for security (SIEM), IT operations (ITSM) .

Key Components

Component	Role
Forwarder	Collects and forwards data (no processing).
Indexer	Parses, indexes, and stores data.
Search Head	Executes SPL queries and visualizes results.
Deployment Server	Manages configurations for forwarders.

Work with Splunk

1. Data Ingestion

• Sources: Logs (files, APIs, syslog), metrics (CPU, memory), and streaming data (Kafka).

• Forwarders: Lightweight agents (Splunk Universal Forwarder) collect and send data to Splunk.

2. Indexing

• Parsing: Splunk extracts key fields (timestamps, hostnames, event types).

• Indexing: Data is stored in time-series indexes for fast retrieval.

3. Search & Analysis

• Search Processing Language (SPL): Splunk’s query language (e.g., index=“wazuh-alert” | stats count by src_ip).

• Correlation: Detects patterns (e.g., brute-force attacks).

4. Visualization & Alerts

• Dashboards: Custom charts/tables in Splunk Web.

• Alerts: Trigger actions (email, webhook) when thresholds are breached.

5. Storage & Retention

• Hot/Warm/Cold Buckets: Automatically moves data to cheaper storage over time.

SPL (Splunk Query Language) Complete Reference

1. BASIC SEARCHES

error                         # Simple text search  
"connection timeout"          # Phrase search  
sourcetype=access_*           # Wildcard source  
status=404                    # Exact field match  
bytes>1000                    # Numeric comparison  

2. BOOLEAN OPERATORS

(failed OR error)             # OR condition  
status=200 AND method=POST    # AND condition  
NOT client_ip=192.168.1.*     # Exclusion  

3. FIELD EXTRACTION

# Regex extraction  
| rex "user=(?<username>\w+)"  

# JSON extraction  
| spath input=json_field  

# Create new field  
| eval mb=bytes/1024/1024  

4. STATISTICAL COMMANDS

# Count events by field  
| stats count by user  

# Time-based aggregation  
| timechart span=1h count by status  

# Top values  
| top 10 client_ip  

5. TIME FILTERS

# Relative time  
earliest=-24h latest=now  

# Absolute time  
earliest="01/01/2025:00:00:00" latest="19/05/2025:00:00:00"  

# Time bucketing  
| bin _time span=15m  

6. ADVANCED EXAMPLES

Security Alert (Brute Force):

sourcetype=auth failed  
| stats count by src_ip  
| where count>5  
| sort -count  

Application Performance:

sourcetype=nginx response_time>2000  
| stats avg(response_time) by app_name