Splunk Engineers

9:00 - 9:15 | Welcome & Orientation

• • Welcome & instructor introductions
• • Course overview & objectives
• • Logistics (breaks, lunch, bathrooms, WiFi)
• • Hybrid format expectations
• • In-person: Participation guidelines
• • Virtual: Camera on/off policy, chat usage
• • Icebreaker: Quick poll
• • What's your Splunk role? (Admin, Developer, Analyst, Architect, ES/SIEM)
• • AI experience level (1-10)
• • One AI concern you have about your Splunk work

9:15 - 9:45 | What is AI?

• • What is AI, really? (demystified)
• • AI is not magic -- it is pattern matching
• • Splunk analogy: Just like Splunk finds patterns in machine data across terabytes of logs, AI finds patterns in training data across terabytes of text
• • Another way to think about it: `| stats count by pattern` on the entire internet
• • AI vs ML vs LLM (hierarchy)
• • Three types of AI (overview)
• • Predictive
• • Generative
• • Agentic
• • Common misconceptions debunked
• • "AI understands my logs" -- no, it predicts statistically likely next tokens
• • "AI will write all my SPL" -- it can help draft queries, but you still need to validate field names, index references, and logic
• • "AI is always right about Splunk commands" -- it will confidently describe commands that do not exist
• • When AI helps vs. when it doesn't

9:45 - 10:15 | Predictive AI Deep Dive

• • Predictive AI explained (weather forecasting analogy)
• • How it works (simplified -- no math!)
• • Real-world examples:
• • Netflix recommendations
• • Spam detection
• • Credit card fraud detection
• • **For YOUR Splunk work:**
• • Anomaly detection in time-series metrics (CPU spikes, memory leaks, request latency drift)
• • Forecasting license usage and ingestion volume
• • Predicting alert storms before they overwhelm the SOC
• • Identifying abnormal login patterns in security event data
• • Detecting new log patterns that deviate from established baselines

10:15 - 10:45 | Generative AI Deep Dive

• • Generative AI explained (creative AI)
• • How LLMs work (simplified)
• • Training on massive text (including technical docs, Stack Overflow, SPL examples, Splunk documentation)
• • Pattern recognition
• • Next-word prediction
• • Real-world examples:
• • ChatGPT
• • GitHub Copilot
• • AI writing assistants

10:45 - 11:00 | Setup Verification & Break Prep

11:30 - 11:40 | Lab 1 Introduction

• • Lab 1 overview: "Your First AI Conversation -- Splunk Edition"
• • Learning objectives:
• • Chat with AI model about Splunk topics
• • Generate SPL queries from natural language descriptions
• • Get AI to explain complex existing SPL
• • Create dashboard panel descriptions from raw searches
• • Catch AI hallucinating about fake SPL commands
• • Demo: Instructor walkthrough (5 min)
• • "Show me failed SSH logins" -> AI generates SPL -> validate against real Splunk
• • Q&A (3 min)
• • Get started!

11:40 - 12:10 | Lab 1: Your First AI Conversation

12:10 - 12:25 | Lab 1 Debrief & Discussion

12:25 - 12:55 | Agentic AI & Introduction to Agents

• • Agentic AI explained
• • Chatbot (passive) vs Agent (active)
• • Chatbot: Tells you SPL syntax. You copy, paste, run, interpret.
• • Agent: Generates SPL, reads the results, classifies severity, suggests next steps, recommends correlation searches -- all in one conversation.
• • Components of an agent:
• • Goal: "Investigate this security alert"
• • Reasoning: "I need to check auth logs, then correlate with network data"
• • Tools: `generate_spl()`, `parse_log_pattern()`, `classify_event_severity()`, `suggest_correlation()`
• • Action: Execute each tool in sequence
• • Result: "Here is the full investigation summary with recommended next steps"
• • Modern tooling: MCP (Model Context Protocol) for standardized tool integration

12:55 - 1:20 | Prompt Engineering Workshop

1:20 - 1:30 | Week 1 Wrap-Up & Homework

• • Recap: What we learned today
• • Three types of AI (with Splunk examples for each)
• • How to use Generative AI for SPL generation and log interpretation
• • Prompt engineering basics for Splunk workflows
• • Preview: Next Saturday (tease exciting content)
• • Build RAG system with YOUR Splunk documentation
• • Create a log analysis agent with real tools
• • Cost analysis for AI-assisted SIEM operations at scale