Making Sense of HS Codes: HSense AI System for Automated Tariff Classfication

Track 3_1615_1_Lablup Conf 2025_Sergey

Transcript

1. Making Sense of HS Codes: HSense AI System for Automated Tariff Classfication Leksikov Sergey
3. Outline • Korea’s Tariff Classification Landscape • Solution: Hsense • HSense Team - workflow • Tariff Officer • GRI rule expert • Legal Framework Expert • HS Code Navigator • Technical details • Tech Stack • Core ideas • Accuracy performance • Challenges • Code Snippet Time
4. Korea’s Tariff Classificaition Landscape
5. The HS Code Tariff System • HS (Harmonized System) Code is a 10-digit numbers that classify every traded product locally and globally • Determine tariffs: 0% vs 8% can mean millions in costs for imported goods • Required by law: Every import/export needs correct classification • 11,000+ possible codes in Korea's system
6. Korea's Classification Landscape https://www.hs-tariff.com/main/hs_mti_ai_mai n/
7. Korea's Classification Landscape • 97 product chapters (01-97, excluding 77) • Each chapter branches into hundreds of subcategories • From mountain animals “01” to artwork “97” • Complex hierarchy requires systematic navigation
9. A Complex Decision Problem https://www.hs-tariff.com/main/hs_mti_ai_mai n/
10. GRI Rules: HS해석에 관한 통
11. Typical workflow base d on GRI-1
12. Typical workflow based-2
13. Typical workflow based-3
14. Solution: HSense
15. Solution: HSense • HSense - AI System for Automated Tariff Classification • Multi-Agent collaboration • Coordinated specialists: • Tariff Officer + GRI Rule Expert + Codes Definition Expert + HS Code Navigator • Making sense of 11,000+ codes through systematic evidence synthesis
16. Benefits of Agentic AI Team System • Rule-based approach based on legal law framework • Robustness for new products and inventions: • Product cases databases cannot keep with new products released on the market • UNIPASS product case dataset covers only about 30% of all possible hs codes • No need to fine-tune LLM on domain or product cases -> cost and time saved • Can explain it’s decision making vs black-box Deep Learning approach
17. HSense Team The Coordinat or The Interpreter The Scholar The Navigator
18. Tariff Officer - The Coordinator • Coordinates work between team member agents • Guides the search • Makes final decision
19. GRI Rule Expert - The Interpreter • Explaings the rules for classification • Provides interpretation and guidance • Rule-based reasoning
20. Legal Framework Expert - The Schola r • Expert in Legal Law document: • 2000+ pages law document • ~ 3 million tokens of legal text • Uses Retrieval Tools • Retrieval Augmentation Generation (RAG) tool over vectorstore for semantic meaning • SQLite database for keyword search over legal framework
21. HS Code Navigator - The Pathfinder • Navigates the code hierarchies from 2 digit to 10 digits • Tool: • SQLite database of codes hierarchies with keyword description
22. HSense Team - workflow
23. Technical details
24. Core ideas • LLM Context Management: • keep context size small • specialization • Ensemble and MoE methods were effective in Machine Learning field • Collaborative Decision Making: • multiple perspectives • cross-validation • consensus building • Task Decomposition: • Divide and Conquer • systematic evidence gathering
25. Tech stack • agentic framework - Agno • semantic database, vectorstore - FAISS (Facebook AI Similarity Search) • Korean PDF with text to structured Markdown - https://github.com/datalab-to/marker • tried about 5 options • another good markdown structure was from https://github.com/microsoft/markitdown • LangChain for chunking based on MarkDown Header content • SQLite with FTS5 extension - full-text search support
26. Accuracy performance • Work in progress and accuracy range within other commercial models • “Benchmarking Harmonized Tariff Schedule Classification Models” Bryce Judy, 11 November 2024 • Accuracy: 44-89% on 10 digits, top-1 • Optimization is needed on multiple levels: • retrieval • prompt • team composition • Inference scalability: • Add RAG pipeline with product cases
27. Prompt Optimization System - 1
28. Prompt Optimization System - 2
29. Challenges • OpenAI API rate limits - prevents optimization and multiple evaluation • Chunking and vectorstore are not right tools for hierarchical document structure • Limited number of open source fast, smart and good at Korean LLM models • Transparency and monitoring • Hard to debug • HSense Manager makes wrong decision -> all members work hard in wrong direction until the end • Need a system to backtrack decision and see the alternatives
30. Scalability of Team of Agents • Multi-agent systems are naturally slower due to LLM processing time • Practical Solutions: • Overnight Batch Processing: Run multiple agent teams on product of 100 catalo gs • Quality-First Approach: Accuracy matters more than real-time speed • Market Context: • Existing slow systems prove value and are in demand: • AI research and report writers (Perplexity, Gemini Deep Research): 20+ minutes per report • OpenAI reasoning pro models: PhD-level problem solving takes time • Companies in specific industries have limited set of product categories • the domain and search space can be narrow downed
32. Basic Agent with Reasoning Tool - 1 • Pydantic schema for structured output • Reasoning Tool - Agent decided when to enter reasoning mode. Example: a complex product with different components or materials
33. Basic Agent with Reasoning Tool - 2 • Continue Agent() object initialization • Add memory to keep session and conversations • Can be used with Sqlite or Postgresql for persistance • Setting the respons_model with pydantic schema for structured output • logging enablers with detailed tools and intermediate steps • multi-modal support for images out of box if model is multi-modal
34. Team definition snippet with modes • Team modes: • collaborate • coordinate • route • Team also itself acts as an Agent with own prompt • Team coordination, decision steps, iterations are all hidden behind Agno framework
36. Appendix
37. HS Code structure
38. HS Code hierarchies database table
39. HS Code Classificaiton full workflow
40. 템플릿 안내사항 • 권장이지만 꼭 이렇게 맞춰야 하는 것은 아닙니다. • 글꼴 : Pretendard (한글/영문) 및 Inter (영문 이탤릭용) Prentendard는 이 파일에 포함되어 있지만, 아래 주소에서 직접 다운로드하여 활용하셔도 무방합 니다. • https://cactus.tistory.com/306 • https://fonts.google.com/specimen/Inter • 코딩 글꼴 : JetBrains Mono • https://www.jetbrains.com/lp/mono/