Forms++: Generating forms that match actual requirements

Creating forms is a tiring task. Why not use LLMs for that? In this session, Max Marschall demonstrated how to automate such repetitive tasks.
You learn how to use the form requirements to generate the actual form. The result can be complex, accessing data and including actual models, relations, and references in your web app—effectively lifting the workload from developers.
However, developers are just some of the people who benefit from this approach.
Join Max to see how you can use Generative AI to enable your web app to ease the daily business of everyone, including its users.

Transcript

1. Forms++ Generating forms that match actual requirement Max Marschall @MaxOSchulte Consultant / Developer
2. Arti fi cial Intelligence Machine Learning Data Science NLP A Deep Learning Generative AI Large Language Models Image / Video Generation
3. Bla bla bla … Intention
4. Bla bla bla … Generation of… Intention Summarisation of… Automation of… Simpli fi cation of…
5. Bla bla bla … “Augmenting an App with an assistant, that helps the user based on displayed context. Allowing an assistant to act for me in that context.” Intention
6. Intention Bla bla bla … “Augmenting an App with an assistant, that helps the user based on displayed context. Allowing an assistant to act for me in that context.” Problem: Integration GenAI Metainformation Tool calls Leaking secrets System integration Solutions AST parsing TS Decorators Client-side "agents" Barebones Angular OpenAI
7. Intention Bla bla bla … “Augmenting an App with an assistant, that helps the user based on displayed context. Allowing an assistant to act for me in that context.”
8. Tools & Functions Process Think Display Bla bla bla …
9. User Prompt • Voice input • Text input • Image input • Unstructured • Universal interface • Natural language as fi rst-class input
10. • Prompts • RAG • Sanitising • Guards • Vector-Database • Tool selection • Additional data (Processing)
11. • System prompt • Base information • Rules • Intention • Output format Processing You are an friendly helpfull assistent proficient in creating and managing forms. You will receive information on the state of an app. If you feel there a … System Prompt https://docs.anthropic.com/en/release-notes/system-prompts#oct-22nd-2024 You are a data analyst API capable of sentiment analysis that responds in JSON. The JSON schema should include { "sentiment_analysis": { "sentiment": "string (positive, negative, neutral)", "confidence_score": "number (0-1)" # Include additional fields as required } https://console.groq.com/docs/text-chat Prompts
12. • State • Additional information • Generation examples Context /** Additional Information: basic list of static pages shown in the side navigation. */ [ { title: 'Contributions', icon: 'question_answer', link: ['/contributions'] }, { title: 'Conferences', icon: 'podium', link: ['/conferences'] }, { title: 'Speakers', icon: 'person', link: ['/speakers'] }, { title: 'Collections', icon: 'category', link: ['/collections'] }, ];
13. Tools & Functions /** * Accepts a resolved sitemap entry and navigates to it. * * e.g. path: /collections/:id * ":id" is replaced by an actual entity id. */ public navigate(resolvedPath: string): void { this.router.navigateByUrl('/' + resolvedPath); } { "type": "function", "function": { "name": "navigate", "description": "Accepts a resolved sitemap entry …, "parameters": { "type": "object", "properties": { "resolvedPath": { "title": "resolvedPath", "description": "", "type": "string" } } }, "required": ["resolvedPath"], } }
14. Large Language Model • Transformer Neural Network • Context & relation aware • Pattern matching & recognition • “Natural” language understanding • Generating / creating content • Summarisation
15. Large Language Model • It cannot make decisions • It cannot understand non-linear content • It cannot draw conclusions
16. Large Language Model • Not deterministic • Can hallucinate • Prompt-Injection
17. Result Bla bla bla …
18. Result = Bla bla bla …
19. Tools & Functions Process Work Display
20. • Pattern matching • Context extraction • Language understanding • Transformation “language to language” Creating Components Used LLM capabilities
21. • “Contract” • Output Language • Interpreted in Code • Guiding Rails • Output Validation Creating Components Schema De fi nition
22. Creating Components { "type": "object", "properties": { "steps": { "type": "array", "items": { "type": "object", "properties": { "title": { "type": "string", "description": "Wizard Step Title." }, "description": { "type": "string", "description": "Wizard Step Short Description. }, "fields": { "type": "array", "items": { "type": "object", "properties": { "type": { "type": "string", "enum": [ "title", "subtitle", "description", "text", "textarea", "select", "radioGroup", "date", "dateRange", const WizardStepSchema = z.object({ steps: z .object({ title: z.string().describe('Wizard Step Title.'), description: z.string().describe('Wizard Step Short Description.'), fields: z .object({ type: z.nativeEnum(FormFieldType).describe('Type of form'), description: z.string() .describe('Description of the Field. What it is and… ‘), label: z.string().describe('Form field label…'), required: z.boolean().describe('Whether the field …’), key: z.string().describe('Key to store the value under.'), }) .array(), }) .array(), }); Zod Schema de fi nition Schema De fi nition
23. Creating Components { "HeaderFormField": { "description": "Header shown in a form. The label is… "type": "object", "properties": { "type": { "description": "Type of form field.", "type": "string", "const": "title" }, … "required": { "description": "Whether the field is required… "type": "boolean" }, … }, "required": ["key", "label", "type"] }, /** Base type for all form field types. */ interface BaseFormField { /** Type of form field. */ type: FormFieldType; /** Description of the Field. What it is and what values should be there. */ description?: string; /** Form field label shown to the user. */ label: string; /** Whether the field is required. Defaults to false. */ required?: boolean; /** Key to store the value under. */ key: string; } /** Header shown in a form. The label is used as its text. */ interface HeaderFormField extends BaseFormField { type: FormFieldType.Title; } AST Parsing Schema De fi nition
24. Creating Components Create a Wizard to collect information about a car accident. The fi rst step is to collect a clients information as well as the insurance ID. The second step is about collection information on the accident, What type of accident, which other party is involved, which fault it was. The third and fi nal step is about accepting the terms of use and a check if it is ok to process these data. steps: [ { title: 'Client Information', description: 'Please provide your personal and insurance info fields: [ { type: 'text', description: 'Enter your legal first name as it appea label: 'First Name', required: true, key: 'firstName', }, … ], }, ], [ { title: 'Accident Information', description: 'Please provide details about the accident', fields: [ { type: 'select', description: 'Select the type of accident', label: 'Type of Accident', required: true, key: ‘accidentType', options: [ { value: 'collision', label: 'Collision' }, { value: 'theft', label: 'Theft' }, { value: 'vandalism', label: 'Vandalism' }, { value: 'other', label: 'Other' }, ],
25. Creating Components Create a Wizard to collect information about a car accident. The fi rst step is to collect a clients information as well as the insurance ID. The second step is about collection information on the accident, What type of accident, which other party is involved, which fault it was. The third and fi nal step is about accepting the terms of use and a check if it is ok to process these data. steps: [ { title: 'Client Information', description: 'Please provide your personal and insurance info fields: [ { type: 'text', description: 'Enter your legal first name as it appea label: 'First Name', required: true, key: 'firstName', }, … ], }, ], [ { title: 'Accident Information', description: 'Please provide details about the accident', fields: [ { type: 'select', description: 'Select the type of accident', label: 'Type of Accident', required: true, key: ‘accidentType', options: [ { value: 'collision', label: 'Collision' }, { value: 'theft', label: 'Theft' }, { value: 'vandalism', label: 'Vandalism' }, { value: 'other', label: 'Other' }, ],
26. Creating Components Create a Wizard to collect information about a car accident. The fi rst step is to collect a clients information as well as the insurance ID. The second step is about collection information on the accident, What type of accident, which other party is involved, which fault it was. The third and fi nal step is about accepting the terms of use and a check if it is ok to process these data. steps: [ { title: 'Client Information', description: 'Please provide your personal and insurance info fields: [ { type: 'text', description: 'Enter your legal first name as it appea label: 'First Name', required: true, key: 'firstName', }, … ], }, ], [ { title: 'Accident Information', description: 'Please provide details about the accident', fields: [ { type: 'select', description: 'Select the type of accident', label: 'Type of Accident', required: true, key: ‘accidentType', options: [ { value: 'collision', label: 'Collision' }, { value: 'theft', label: 'Theft' }, { value: 'vandalism', label: 'Vandalism' }, { value: 'other', label: 'Other' }, ],
27. Tools & Functions Process Work Display
28. Process Work Tools & Functions
29. Process Work Tools & Functions { "HeaderFormField": { "description": "Header shown in a form. The label is… "type": "object", "properties": { "type": { "description": "Type of form field.", "type": "string", "const": "title" }, … "required": { "description": "Whether the field is required… "type": "boolean" }, … }, "required": ["key", "label", "type"] }, /** Base type for all form field types. */ interface BaseFormField { /** Type of form field. */ type: FormFieldType; /** Description of the Field. What it is and what values should be there. */ description?: string; /** Form field label shown to the user. */ label: string; /** Whether the field is required. Defaults to false. */ required?: boolean; /** Key to store the value under. */ key: string; } /** Header shown in a form. The label is used as its text. */ interface HeaderFormField extends BaseFormField { type: FormFieldType.Title; }
30. Tools & Functions Process Work /** * Creates form fields for a provided list of formField definition. */ createFormFields(fields: FormField[]): FormField[] { … } /** Union of all supported form field types. */ export type FormField = | HeaderFormField | SubheaderFormFiel | …
31. /** * Creates form fields for a provided list of formField definition. */ @AiTool createFormFields(fields: FormField[]): FormField[] { … } /** Union of all supported form field types. */ export type FormField = | HeaderFormField | SubheaderFormFiel | … Tools & Functions Process Work { "type": "function", "className": "FormBuilderService", "classDocumentation": "", "function": { "name": "createFormFields", "title": "createFormFields", "type": "function", "description": "Creates form fields for a provided list… "parameters": { "type": "object", "properties": { "fields": { "title": "fields", "description": "Union of all supported form… "anyOf": [ { "$ref": "#/definitions/HeaderFormField" }, …
32. Process Work Tools & Functions /** * Creates form fields for a provided list of formField definition. */ @AiTool createFormFields(fields: FormField[]): FormField[] { … } /** Union of all supported form field types. */ export type FormField = | HeaderFormField | SubheaderFormFiel | … { "type": "function", "className": "FormBuilderService", "classDocumentation": "", "function": { "name": "createFormFields", "title": "createFormFields", "type": "function", "description": "Creates form fields for a provided list… "parameters": { "type": "object", "properties": { "fields": { "title": "fields", "description": "Union of all supported form… "anyOf": [ { "$ref": "#/definitions/HeaderFormField" }, …
33. Process Work Tools & Functions /** * Creates form fields for a provided list of formField definition. */ @AiTool createFormFields(fields: FormField[]): FormField[] { … } /** Union of all supported form field types. */ export type FormField = | HeaderFormField | SubheaderFormFiel | … { "type": "function", "className": "FormBuilderService", "classDocumentation": "", "function": { "name": "createFormFields", "title": "createFormFields", "type": "function", "description": "Creates form fields for a provided list… "parameters": { "type": "object", "properties": { "fields": { "title": "fields", "description": "Union of all supported form… "anyOf": [ { "$ref": "#/definitions/HeaderFormField" }, … tool_calls": [ { "id": "call_wqxp1eYfhDTZrkuXfPqFP7uc", "type": "function", "function": { "name": "createFormFields", "arguments": { "fields": [ { "type": "title", "label": "Event Information", "description": "Form to collect information about an upcoming event.", "key": "eventInformationTitle" }, { "type": "text", "label": "Event Name", "description": "Enter the name of the event.", "key": "eventName" }, { "type": "select", "label": "Speaker", "description": "Select an existing speaker for the event.", "options": { "labelKey": "fullName", "collectionId": "speakers" }, "key": "speaker" } ] } …
34. Structured Output Process Work Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don't need to worry about the model omitting a required key, or hallucinating an invalid enum value. https://platform.openai.com/docs/guides/structured-outputs?context=without_parse
35. Structured Output Process Work const completion = await this.client.chat.completions.create({ tool_choice: 'auto', model: 'gpt-4o-2024-08-06', temperature: 0, messages, tools, }); const completion = await this.client.chat.completions.create({ model: ‘gpt-4o-2024-08-06', messages, structured_output, }); "fields": [ { "type": "title", "label": "Event Information", "description": "Form to collect information about an upcoming event.", "key": "eventInformationTitle" }, … ] tool_calls": [ { "id": "call_wqxp1eYfhDTZrkuXfPqFP7uc", "type": "function", "function": { "name": "createFormFields", "arguments": { "fields": [ { "type": "title", "label": "Event Information", "description": "Form to collect information about an upcoming eve "key": "eventInformationTitle" …
36. Structured Output Process Work • Providing only an output schema, no function • Some limitation • “Use for presenting to the user” • Reliable type-safety • Explicit refusals
37. Structured Output Process Work • String: minLength, maxLength, pattern, format • Numbers: minimum, maximum, multipleOf • Objects: patternProperties, unevaluatedProperties, propertyNames, minProperties, maxProperties • Arrays: unevaluatedItems, contains, minContains, maxContains, minItems, maxItems, uniqueItems Limitations on strict
38. • “Talking to fast learning & smart junior" • No default assumptions • Documentation is key • Examples are great Development Mindset
39. • Avoid auto-commit • Enable feedback • Highlight probability • Clear entry point into AI work fl ows • Progress / generation indicators • Suggest common actions UI & UX
40. • Client-side for PoC • Easy transition to server-side • Write docs for dummies • Unambiguous tools • Unambiguous context • An Agent is just a loop LLMs do not actually call functions - you do! Lessons Learned
41. It's a wrap! • Slides: www.thinktecture.com/max-marschall • @MaxOSchulte, max.marschall@thinktecture.com