Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Sr. Security Researcher at Microsoft Author of Visual Threat Intelligence https://SecurityBreak.io & https://unprotect.it @Fr0gger_

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Challenges in Threat Intelligence Introduction to Prompt Engineering Techniques & Common Pitfalls Case Studies & Practical Applications (with code examples) Future Insights

Slide 5

Slide 5 text

Leverage AI for enhanced capabilities. Time-consuming understanding Overwhelming data New analysts' learning curve Fast-evolving threats

Slide 6

Slide 6 text

Prompt Engineering is the the discipline and science of crafting effective prompts to guide AI models, especially LLMs, to desired outputs.

Slide 7

Slide 7 text

Clarity: Clearly define the task you want the model to perform. Specificity: Be as detailed as necessary to avoid ambiguous results. Iteration: Continuously refine prompts based on AI feedback.

Slide 8

Slide 8 text

Over-complexity: Avoid making prompts overly detailed. Ambiguity: Vague prompts can lead to generic responses. Blind Trust in the Model: Over-reliance on the model's capabilities without verification. No Examples: not providing example inputs and outputs. Misplaced Belief in Model's Understanding: Assuming the model can intuitively understand your meaning. Ignoring Obsolescence: Failing to update prompts in line with model updates or changes in data.

Slide 9

Slide 9 text

Precision and specificity Close ended question You are a threat intelligence analyst, analyze the provided data to identify MITRE ATT&CK techniques, and present in Markdown with columns for ID, Description, and Comments. Here are my data: Role Definition / Contextual Awareness Clear Objectives Iterative refinement

Slide 10

Slide 10 text

Self-Consistency Generate Knowledge Tree of Thoughts ReAct Graph Prompting Role Prompting Zero-shot Prompting Few-shot Prompting Directional Stimulus Prompting

Slide 11

Slide 11 text

What is Few-Shot prompting? Guides ML models using example prompts for task understanding. Zero-Shot vs Few-Shot Zero-Shot: No example prompts. Few-Shot: Uses example prompts for clarity. Why Use Few-Shot Prompting? Enhances task-specific accuracy. Ideal for complex or nuanced tasks.

Slide 12

Slide 12 text

Source: https://store.securitybreak.io/theintelbrief

Slide 13

Slide 13 text

RAG is a technique used to combine retrieval-based and generative models Two Phases: Retrieval & Generation Retrieval: Searches Database Generation: Context-Relevant Response Customize with Your Own Data! Prepare your data Tokenization Split in smaller Chunks Embeddings and Vector Model is ready for input

Slide 14

Slide 14 text

Source: https://otrf.github.io/GPT-Security-Adventures/experiments/ATTCK-GPT/notebook.html#generate-knowledge-base-embeddings

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

What is ReAct? A logic framework for creating intelligent agents. Allows agents to perform complex tasks through a series of actions. Core Components Reason: The agent's thought process to decide the next action. Act: The actual action taken by the agent based on reasoning. How Does It Work? Action → Observation → Thought Cycle The agent performs an action. Observes the result. Thinks (Reasons) about the next step. https://peterroelants.github.io/posts/react-repl-agent/

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

HTTPS://GITHUB.COM/FR0GGER/AWESOME-GPT-AGENTS

Slide 23

Slide 23 text

AI in Threat Intelligence Generative AI can address challenges in threat intelligence. Crafting the right prompt The importance of clearly defining objectives in prompt engineering. Mastering Techniques Understanding various methods like Few-Shot Learning, RAG, ReAct, etc.

Slide 24

Slide 24 text

https://hbr.org/2023/06/ai-prompt-engineering-isnt-the-future

Slide 25

Slide 25 text

Thomas Roccia @fr0gger_ SecurityBreak.io