Language Model Concepts Part 2 Part 6 Generative AI & Live Demo RAG & Offline Documents Hands-On Hands-On Jan.ai - local models Goose AI - local agents Part 3 Part 7 Governance, Ethics & Regulation Probabilistic vs Deterministic Part 4 Hands-On AI Standards (from ISO) NotebookLM
Dartmouth Conference Birth of the term 'Artificial Intelligence' (McCarthy, 1956) 1966 ELIZA by Joseph Weizenbaum First chatbot; pattern-matched responses; fooled many users 1980s Expert Systems Rules-based AI: if-then logic encoding human expertise 1990s Machine Learning & Image Processing Statistical models, SVMs, early neural nets, computer vision 2012 Deep Learning Revolution AlexNet wins ImageNet; GPUs enable large neural networks 2017 Transformer Architecture 'Attention Is All You Need' - the paper that changed everything 2022+ Large Language Models & Agents GPT, Claude, Gemini; agentic systems that can act and plan and code
- they generate new output token by token, based on patterns learned from vast training data. Text Generation Image Generation Code Generation Claude, GPT-5, Gemini - answer questions, write, summarise, reason Midjourney, DALL-E, Stable Diffusion pixels from prompts GitHub Copilot, Cursor - write, explain, debug code Audio / Speech Video Multi-modal Eleven Labs, Whisper - voices, transcription, music Sora, Runway - generate or edit video from text Models that handle text, images and audio together
the AI has already received a set of instructions. This is the system prompt. System Prompt Set by the developer/operator. Defines persona, rules, context, tone. User often cannot see this. User Message What you type. The model sees both system prompt AND user message together. Model Response Generated output constrained by both layers above.
Jan.ai is a desktop application that lets you download and run open-weights language models entirely on your own machine. No API key. No data sent to the cloud. What you can do Requirements Download models (Llama, Mistral, Phi, Gemma), chat locally, use a local OpenAI-compatible API endpoint on port 1337 At least 8GB RAM for small models (3–7B). 16GB+ recommended. GPU optional but speeds things up significantly. Good for Limitations Privacy-sensitive use cases, offline operation, testing prompts locally, building local RAG pipelines Smaller models are less capable than GPT-4/Claude. Slower without GPU. Model choice matters - quantisation reduces quality.
do the magic that they do. • Tokens are the currency of generative AI. • Typically, 4 characters make up a token. Characters include everything in a sentence: space, punctuation, letters, numbers, emojis. • Cost of tokens will tend to ZERO https://tiktokenizer.vercel.app/ https://www.claudetokenizer.com/ https://platform.openai.com/tokenizer https://lunary.ai/deepseek-tokenizer
et al Since inputs to a language model is human language via text, audio, video, numbers, these can be weaponised to attack systems Prompt Injection Red Teaming https://tiktokenizer.vercel.app/ - let’s visualise it Combination of Prompt Injection and injection of zero width characters Pumping invisible text Limitations https://gibberifier.com Unless clean and sanitised text is able to be consistently used, expect systems to be attacked
cause real-world harm: bias, misinformation, privacy violations, automation of harmful tasks Unlike traditional software, AI outputs are probabilistic and hard to audit or predict Power is concentrated in a few large organisations - access, accountability, and transparency are concerns Regulation is lagging behind capability - governments are scrambling to catch up EU AI Act (2024) US Approach Other Countries First major AI-specific law. Risk-tiered: minimal → limited → high-risk → unacceptable. Fines up to €35M or 7% of global revenue. Executive Orders, NIST AI RMF framework. Sector-specific. More voluntary than EU. Ongoing Congressional debate. [US GOVT CORRUPTION IS KILLING ALL VALUE] UK: pro-innovation, light-touch. China: strict state oversight. Singapore, Canada, India are developing national frameworks.
manipulation, real-time facial recognition in public spaces High Risk CV screening, credit scoring, medical devices, critical infrastructure, law enforcement Limited Risk Chatbots - must disclose they are AI. Deepfakes - must be labelled. Minimal Risk Spam filters, AI in games. Largely unregulated but encouraged to follow voluntary codes.
vocabulary and measurable accountability framework - essential for procurement, audit, and board-level governance. ISO 42001 - AI Management System ISO 5338 - AI Lifecycle Processes The top-level management standard. Defines how an organisation governs AI development and deployment. Certifiable. Think ISO 9001 or ISO 27001 but for AI. Covers the full lifecycle of AI systems: planning, development, deployment, monitoring, retirement. ISO 5339 - AI Use Case Guidance ISO 8183 - AI Data Life Cycle Practical guidance on identifying and scoping AI use cases, including risk and impact considerations. Focused on data: collection, quality, labelling, storage, and governance as it relates to AI systems.
the state of the art (e.g. GPT-4o, Claude 3.7, Gemini Ultra). Closed You access the model via an API. Weights are not released. OpenAI, Anthropic, Google. Open Weights The model weights are downloadable. You can run locally. Not the same as open source (training data & code may be proprietary). Examples: Llama, Mistral, Phi. Open Source Weights + training code + data all released. Rare. Pythia, BLOOM as examples. Distillation Training a smaller model to mimic a larger one's outputs. Makes capable models that run on consumer hardware. Thinking / CoT Models that reason step-by-step before answering. Slower, more accurate on complex tasks. Claude's 'extended thinking', o1, o3, DeepSeek-R1. Instruction Follow Fine-tuning a base model to follow human instructions rather than just predict next tokens. Base model → Instruct model.
to training data, you know exactly what was used to train the model • You can recreate the model from the training data - even though it can be costly • You will NOT be at the mercy of the model trainers who could have placed code in the training data that could only be triggered very selectively • The cut-off date of the training determines the currency of the model ◦ • Check out Mr Chatterbox Discover and explore the, in excess of, 2.8 million models at huggingface.co
using your documents - without retraining the model and without the documents leaving your control. 1 2 3 4 5 6 Your Documents Chunk & Embed Vector Store User Question Retrieve LLM + Context PDFs, notes, manuals, reports Text split into chunks; converted to vectors (numbers representing meaning) Chunks stored in a local database, searchable by similarity Your query is also embedded into a vector Top matching chunks pulled from the store Model answers using retrieved chunks as context Key point: with a local vector store (e.g. Postgres/ChromaDB, Ollama + local LLM), your documents never leave your machine.
AI agent you run locally. It can browse files, run code, search the web, and complete multi-step tasks using tools - all on your machine. What makes Goose different from a chatbot: • • • • Agents can take actions, not just answer questions It uses tools: file system, shell commands, web search, APIs You describe a goal; the agent plans and executes steps autonomously Can connect to MCP (Model Context Protocol) servers for extended capabilities Use with Install Jan.ai local model OR a cloud model API (Anthropic, OpenAI, Ollama) brew install block/goose/goose (Mac) or via pip. CLI-first, GUI in development.
running with large language models such as gpt-oss, Gemma 4, DeepSeek-R1, Qwen3 and more. What makes Ollama useful: • • • Pull down models (open source and open weight) to local systems and run them It can use tools: file system, shell commands, web search, APIs It is constantly being enhanced for extended capabilities Use with Install Jan.ai local model OR a cloud model API (Anthropic, OpenAI, Ollama) curl -fsSL https://ollama.com/install.sh | sh
🎲 The Language Model • • • • • • • Switch ON → bulb lights. Every time. No exceptions. Switch OFF → bulb off. Every time. Given the same input, you always get the same output You can predict the result before running it Examples: calculators, databases, traditional software • • • Same prompt → different answer each time Model picks next token based on probability distribution You cannot fully predict output in advance Output is somewhere 'in the ballpark' - not exact This is a feature AND a limitation: creative but not reliable for exact facts
materials you provide. It doesn’t browse the open web by default, and it cites where each answer comes from. Source grounding & citations Upload PDFs, Google Docs, websites, YouTube videos and audio. Clickable citations jump to the original passage. Audio Overviews A podcast-style dialogue between two AI hosts. In interactive mode you can interrupt and ask questions. Studio multimodal outputs Slide decks, infographics, mind maps, data tables, quizzes and flashcards, all grounded in your sources. Deep Research Adds web information to your notes to produce cited research reports on broader topics. Gemini & Workspace integration Use it inside Gemini chats with @Gemini Notebook. Included in Google Workspace plans for organizations and education.
addresses the hallucination problem in research. Students, educators, researchers and content creators can turn materials into study aids, presentation, listenable summaries with traceability to the original text. Bottom line: a closed-loop research workspace where your documents are the main source of truth, and AI turns them into podcasts, slides and study tools on demand.
attempt to replicate Claude's capabilities or interface. Key issues: Hermes (by Nous Research) is a family of fine-tuned models trained for agent/tool-use tasks. • Not affiliated with Anthropic - no safety guarantees • Model behaviour may be unpredictable • Potential licensing and IP issues depending on implementation • Useful for experimentation but not production or sensitive data • Runs locally via Ollama or Jan.ai • Strong instruction-following and function-calling • Good choice for local agentic workflows where privacy matters • Open weights - inspect and run yourself Practical guidance: • • • Always verify the provenance and licence of any open-weights model before using in an organisational context Check whether fine-tuning data has appropriate licensing For sensitive data, prefer models you can fully audit and run offline
To ensure that you can get to use language models consistently, a form of memory between invocations is needed. • • That’s what Memory.MD is able to offer. This is part of the “harness” that wraps around language models to help them have context Practical guidance: • • Harnesses are evolving rapidly We have seen them being used in OpenClaw, Hermes and many others
its core, an enormous number of matrix multiplication operations. GPUs were designed to do exactly that - massively in parallel. CPU vs GPU Why Matrices? CPU: 8–32 powerful cores, optimised for sequential logic. GPU: Thousands of smaller cores, optimised for parallel mathematical operations (originally: rendering triangles in games). Every layer of a neural network multiplies input vectors by weight matrices. A 7B parameter model does billions of these multiplications per token. GPUs can do many simultaneously. The Unexpected Overlap Practical Takeaway Crypto mining is also matrix-heavy parallel computation - the same GPU architecture works for both. This is why GPU prices spiked during the crypto boom and again during the AI boom. Consumer NVIDIA GPUs (RTX 3080/4090) can run 7–13B models. Apple Silicon (M2/M3/M4) uses unified memory and works well for local inference without a discrete GPU.
of research culminated in today's LLMs 2 GenAI is probabilistic: expect variation, not precision 3 Governance is real: EU AI Act, ISO 42001, and national frameworks apply now 4 You can run capable AI entirely offline - privacy is achievable 5 RAG is how you make AI work with your own documents without retraining 6 Agents go beyond chat - they plan and act autonomously 7 GPUs matter because AI is parallel matrix maths, not sequential logic
harishpillay
emna__ayadi
aleyda
twada
codingconduct
joshbly
sachag
marcduiker
jeffersonlam
katiecoart
csswizardry
charlesmeaden
